openai-fineweb-edu-scorer-xlm-multilabel-xlm-roberta-base-lr5e-05-20250411_122059

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3645
• Precision: 0.5930
• Recall: 0.4813
• F1 Macro: 0.5091
• Accuracy: 0.6488

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 128
• seed: 0
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 Macro	Accuracy
No log	0	0	8.1215	0.0002	0.2	0.0004	0.0010
0.2936	0.7812	1000	0.2877	0.5095	0.4544	0.4485	0.6647
0.2516	1.5625	2000	0.3197	0.5299	0.3886	0.4136	0.6293
0.1737	2.3438	3000	0.2922	0.5089	0.4296	0.4518	0.6516
0.0996	3.125	4000	0.3233	0.5012	0.4379	0.4585	0.6369
0.1051	3.9062	5000	0.3108	0.5042	0.4401	0.4609	0.6496
0.0601	4.6875	6000	0.3501	0.4840	0.4501	0.4614	0.6411
0.0588	5.4688	7000	0.3554	0.4758	0.4585	0.4658	0.6327
0.0385	6.25	8000	0.3527	0.4853	0.4518	0.4647	0.6331
0.0312	7.0312	9000	0.3475	0.4912	0.4580	0.4714	0.6415
0.0314	7.8125	10000	0.3442	0.4983	0.4557	0.4679	0.6551
0.0211	8.5938	11000	0.3543	0.4823	0.4740	0.4769	0.6449
0.0193	9.375	12000	0.3585	0.4816	0.4621	0.4697	0.6438
0.0208	10.1562	13000	0.3730	0.4825	0.4588	0.4659	0.6207
0.0203	10.9375	14000	0.3578	0.4903	0.4748	0.4818	0.6468
0.0154	11.7188	15000	0.3513	0.4994	0.4591	0.4744	0.6557
0.0126	12.5	16000	0.3623	0.4920	0.4488	0.4649	0.6460
0.0087	13.2812	17000	0.3632	0.4940	0.4512	0.4675	0.6449
0.0091	14.0625	18000	0.3518	0.6976	0.4793	0.5121	0.6533
0.009	14.8438	19000	0.3569	0.5918	0.4886	0.5128	0.6527
0.0056	15.625	20000	0.3672	0.5532	0.4882	0.5081	0.6457
0.0045	16.4062	21000	0.3655	0.5391	0.4870	0.5060	0.6460
0.0035	17.1875	22000	0.3646	0.4955	0.4634	0.4765	0.6489
0.0032	17.9688	23000	0.3631	0.6942	0.4841	0.5150	0.6503
0.0035	18.75	24000	0.3625	0.5986	0.4805	0.5103	0.6504
0.0019	19.5312	25000	0.3645	0.5930	0.4813	0.5091	0.6488

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.1