scibert-base-uncased-ner

This model is a fine-tuned version of allenai/scibert_scivocab_uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0191
• Cases: {'precision': 0.9767981438515081, 'recall': 0.967816091954023, 'f1': 0.972286374133949, 'number': 435}
• Country: {'precision': 0.9751332149200711, 'recall': 1.0, 'f1': 0.9874100719424461, 'number': 549}
• Date: {'precision': 0.9706896551724138, 'recall': 0.9690189328743546, 'f1': 0.9698535745047373, 'number': 581}
• Deaths: {'precision': 0.9529411764705882, 'recall': 0.9501466275659824, 'f1': 0.9515418502202643, 'number': 341}
• Virus: {'precision': 0.9963235294117647, 'recall': 0.998158379373849, 'f1': 0.9972401103955841, 'number': 543}
• Overall Precision: 0.9760
• Overall Recall: 0.9796
• Overall F1: 0.9778
• Overall Accuracy: 0.9923

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Cases	Country	Date	Deaths	Virus	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
No log	1.0	291	0.0411	{'precision': 0.90744920993228, 'recall': 0.9241379310344827, 'f1': 0.9157175398633258, 'number': 435}	{'precision': 0.9699646643109541, 'recall': 1.0, 'f1': 0.9847533632286996, 'number': 549}	{'precision': 0.9149305555555556, 'recall': 0.9070567986230637, 'f1': 0.9109766637856526, 'number': 581}	{'precision': 0.8830769230769231, 'recall': 0.841642228739003, 'f1': 0.8618618618618619, 'number': 341}	{'precision': 0.9889908256880734, 'recall': 0.992633517495396, 'f1': 0.9908088235294119, 'number': 543}	0.9385	0.9408	0.9396	0.9861
0.1005	2.0	582	0.0291	{'precision': 0.9733656174334141, 'recall': 0.9241379310344827, 'f1': 0.9481132075471699, 'number': 435}	{'precision': 0.9699646643109541, 'recall': 1.0, 'f1': 0.9847533632286996, 'number': 549}	{'precision': 0.9512195121951219, 'recall': 0.9397590361445783, 'f1': 0.9454545454545454, 'number': 581}	{'precision': 0.9161849710982659, 'recall': 0.9296187683284457, 'f1': 0.9228529839883551, 'number': 341}	{'precision': 0.9889908256880734, 'recall': 0.992633517495396, 'f1': 0.9908088235294119, 'number': 543}	0.9628	0.9608	0.9618	0.9910
0.1005	3.0	873	0.0221	{'precision': 0.9764705882352941, 'recall': 0.9540229885057471, 'f1': 0.9651162790697674, 'number': 435}	{'precision': 0.9751332149200711, 'recall': 1.0, 'f1': 0.9874100719424461, 'number': 549}	{'precision': 0.9706896551724138, 'recall': 0.9690189328743546, 'f1': 0.9698535745047373, 'number': 581}	{'precision': 0.9552238805970149, 'recall': 0.9384164222873901, 'f1': 0.9467455621301775, 'number': 341}	{'precision': 0.9963235294117647, 'recall': 0.998158379373849, 'f1': 0.9972401103955841, 'number': 543}	0.9763	0.9755	0.9759	0.9929
0.0237	4.0	1164	0.0216	{'precision': 0.9789719626168224, 'recall': 0.9632183908045977, 'f1': 0.9710312862108922, 'number': 435}	{'precision': 0.9751332149200711, 'recall': 1.0, 'f1': 0.9874100719424461, 'number': 549}	{'precision': 0.9740034662045061, 'recall': 0.9672977624784854, 'f1': 0.9706390328151987, 'number': 581}	{'precision': 0.9502923976608187, 'recall': 0.9530791788856305, 'f1': 0.951683748169839, 'number': 341}	{'precision': 0.9944954128440368, 'recall': 0.998158379373849, 'f1': 0.9963235294117647, 'number': 543}	0.9764	0.9788	0.9776	0.9921
0.0237	5.0	1455	0.0191	{'precision': 0.9767981438515081, 'recall': 0.967816091954023, 'f1': 0.972286374133949, 'number': 435}	{'precision': 0.9751332149200711, 'recall': 1.0, 'f1': 0.9874100719424461, 'number': 549}	{'precision': 0.9706896551724138, 'recall': 0.9690189328743546, 'f1': 0.9698535745047373, 'number': 581}	{'precision': 0.9529411764705882, 'recall': 0.9501466275659824, 'f1': 0.9515418502202643, 'number': 341}	{'precision': 0.9963235294117647, 'recall': 0.998158379373849, 'f1': 0.9972401103955841, 'number': 543}	0.9760	0.9796	0.9778	0.9923

Framework versions

• Transformers 4.51.3
• Pytorch 2.5.1+cu121
• Datasets 3.5.0
• Tokenizers 0.21.1