distilbert-base-uncased-ner-finer

Model description

This model is a fine-tuned version of distilbert-base-uncased on the Finer-139 dataset.

It achieves the following results on the evaluation set:

• Loss: 0.0293
• Precision: 0.8768
• Recall: 0.9064
• F1: 0.8914
• Accuracy: 0.9901

Training and evaluation data

The training data consists of the top 4 ner_tags having the most occurence from the Finer-139 dataset plus the outside tag "O".

Training results

Epoch	Training Loss	Validation Loss	Precision	Recall	F1	Accuracy
1	0.035700	0.035880	0.847873	0.890125	0.868486	0.987242
2	0.023700	0.029618	0.867055	0.906431	0.886306	0.989505
3	0.017000	0.029322	0.876898	0.906431	0.891420	0.990180

Valiadtion results

ner_tag	precision	recall	f1-score	support
O	1.00	0.99	1.00	229573
I-DebtInstrumentInterestRateStatedPercentage	0.94	0.94	0.94	5412
I-LineOfCreditFacilityMaximumBorrowingCapacity	0.82	0.88	0.85	4288
I-DebtInstrumentBasisSpreadOnVariableRate1	0.89	0.97	0.93	4788
I-DebtInstrumentFaceAmount	0.79	0.76	0.78	3398

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0