gemma-fineweb-edu-scorer-xlm-multilabel-lr5e-05-20250411_131303

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2492
• Precision: 0.6627
• Recall: 0.6146
• F1 Macro: 0.6294
• Accuracy: 0.7497

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 128
• seed: 0
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 Macro	Accuracy
No log	0	0	3.9808	0.0433	0.25	0.0737	0.1730
0.2093	0.7812	1000	0.2214	0.7611	0.5177	0.5153	0.7086
0.1698	1.5625	2000	0.1857	0.5631	0.5555	0.5575	0.7651
0.1121	2.3438	3000	0.2308	0.6434	0.6269	0.6259	0.7284
0.064	3.125	4000	0.2097	0.6739	0.5911	0.6102	0.7503
0.0754	3.9062	5000	0.2096	0.6880	0.6116	0.6331	0.7550
0.043	4.6875	6000	0.2199	0.6600	0.6042	0.6193	0.7440
0.0339	5.4688	7000	0.2400	0.6427	0.6289	0.6301	0.7311
0.0229	6.25	8000	0.2384	0.6554	0.5982	0.6134	0.7470
0.0239	7.0312	9000	0.2401	0.6446	0.6180	0.6268	0.7423
0.0222	7.8125	10000	0.2902	0.6406	0.6020	0.6039	0.7003
0.0189	8.5938	11000	0.2368	0.6471	0.6025	0.6177	0.7368
0.0154	9.375	12000	0.2357	0.6453	0.6222	0.6284	0.7392
0.0115	10.1562	13000	0.2361	0.6856	0.5721	0.5974	0.7519
0.0117	10.9375	14000	0.2431	0.6438	0.6418	0.6413	0.7412
0.0126	11.7188	15000	0.2460	0.6443	0.6101	0.6184	0.7461
0.0079	12.5	16000	0.2571	0.6397	0.6446	0.6342	0.7325
0.006	13.2812	17000	0.2454	0.6572	0.6111	0.6244	0.7428
0.0055	14.0625	18000	0.2430	0.6588	0.6164	0.6246	0.7506
0.0044	14.8438	19000	0.2491	0.6504	0.6272	0.6333	0.7417
0.0052	15.625	20000	0.2445	0.6580	0.6095	0.6229	0.7486
0.0053	16.4062	21000	0.2429	0.6654	0.6054	0.6225	0.7496
0.0021	17.1875	22000	0.2485	0.6740	0.6172	0.6327	0.7496
0.0037	17.9688	23000	0.2432	0.6582	0.6198	0.6307	0.7523
0.0018	18.75	24000	0.2527	0.6575	0.6149	0.6275	0.7451
0.0026	19.5312	25000	0.2492	0.6627	0.6146	0.6294	0.7497

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.1