multilingual-e5-base-edu-scorer-lr3e4-bs32

This model is a fine-tuned version of intfloat/multilingual-e5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0854
• Precision: 0.4959
• Recall: 0.3633
• F1 Macro: 0.3589
• Accuracy: 0.3909

Model description

More information needed

Intended uses & limitations

More information needed

Test results

Binary classification accuracy (threshold at label 3) ≈ 82.00%

Test Report:

              precision    recall  f1-score   support

           0       0.81      0.42      0.55       100
           1       0.30      0.34      0.32       100
           2       0.36      0.55      0.44       100
           3       0.32      0.55      0.41       100
           4       0.43      0.26      0.32       100
           5       0.75      0.06      0.11        50

    accuracy                           0.39       550
   macro avg       0.50      0.36      0.36       550
weighted avg       0.47      0.39      0.38       550

Confusion Matrix:

[[42 45 10  2  1  0]
 [10 34 38 17  1  0]
 [ 0 24 55 21  0  0]
 [ 0  6 27 55 12  0]
 [ 0  2 18 53 26  1]
 [ 0  1  3 22 21  3]]

Test metrics

  epoch                   =       20.0
  eval_accuracy           =     0.3909
  eval_f1_macro           =     0.3589
  eval_loss               =     1.0854
  eval_precision          =     0.4959
  eval_recall             =     0.3633
  eval_runtime            = 0:00:05.35
  eval_samples_per_second =    102.729
  eval_steps_per_second   =      3.362

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 Macro	Accuracy
No log	0	0	3.1979	0.0587	0.1667	0.0869	0.3524
0.788	0.3368	1000	0.7656	0.4083	0.3366	0.3319	0.4744
0.7595	0.6736	2000	0.7674	0.4107	0.3293	0.3194	0.4378
0.7858	1.0104	3000	0.9050	0.4264	0.2889	0.2769	0.4908
0.8086	1.3473	4000	0.7185	0.4152	0.3450	0.3437	0.4842
0.7441	1.6841	5000	0.7373	0.4055	0.3347	0.3297	0.455
0.7725	2.0209	6000	0.7261	0.4190	0.3488	0.3463	0.4696
0.7339	2.3577	7000	0.7775	0.3969	0.3423	0.3450	0.5168
0.7256	2.6945	8000	0.7042	0.4018	0.3581	0.3608	0.5058
0.6947	3.0313	9000	0.7011	0.4093	0.3528	0.3520	0.492
0.6911	3.3681	10000	0.6902	0.4211	0.3554	0.3536	0.5078
0.6966	3.7050	11000	0.6893	0.4131	0.3591	0.3602	0.5018
0.7511	4.0418	12000	0.7161	0.3995	0.3431	0.3449	0.5196
0.6848	4.3786	13000	0.6771	0.4144	0.3656	0.3661	0.5114
0.6837	4.7154	14000	0.6807	0.4017	0.3523	0.3539	0.522
0.6755	5.0522	15000	0.6824	0.4033	0.3512	0.3529	0.5154
0.6759	5.3890	16000	0.6775	0.4029	0.3616	0.3619	0.5186
0.678	5.7258	17000	0.6773	0.4111	0.3552	0.3499	0.4952
0.656	6.0626	18000	0.6745	0.5747	0.3821	0.3902	0.518
0.6639	6.3995	19000	0.6655	0.4072	0.3626	0.3618	0.5188
0.6982	6.7363	20000	0.6723	0.4023	0.3615	0.3618	0.51
0.6286	7.0731	21000	0.6849	0.4157	0.3541	0.3565	0.5288
0.6191	7.4099	22000	0.6646	0.4197	0.3694	0.3695	0.517
0.6338	7.7467	23000	0.6879	0.4192	0.3589	0.3601	0.5272
0.6744	8.0835	24000	0.7007	0.4348	0.3456	0.3462	0.5248
0.6241	8.4203	25000	0.6631	0.4148	0.3680	0.3694	0.5286
0.6361	8.7572	26000	0.6732	0.4172	0.3553	0.3551	0.5236
0.6326	9.0940	27000	0.6729	0.4073	0.3657	0.3669	0.526
0.6072	9.4308	28000	0.6578	0.4149	0.3717	0.3738	0.5166
0.6539	9.7676	29000	0.6636	0.4165	0.3584	0.3596	0.526
0.6353	10.1044	30000	0.6615	0.4359	0.3817	0.3865	0.5246
0.6018	10.4412	31000	0.6612	0.4663	0.3828	0.3890	0.5166
0.609	10.7780	32000	0.6718	0.4172	0.3612	0.3624	0.5316
0.6027	11.1149	33000	0.6944	0.4655	0.3924	0.3995	0.5
0.6006	11.4517	34000	0.6739	0.4235	0.3551	0.3569	0.526
0.5649	11.7885	35000	0.6651	0.4379	0.3763	0.3819	0.522
0.5799	12.1253	36000	0.6574	0.4128	0.3661	0.3681	0.519
0.577	12.4621	37000	0.6555	0.4187	0.3717	0.3712	0.5274
0.5935	12.7989	38000	0.7002	0.4236	0.3755	0.3761	0.4846
0.5726	13.1357	39000	0.6885	0.4202	0.3796	0.3812	0.4986
0.5966	13.4725	40000	0.6773	0.4242	0.3811	0.3838	0.5058
0.5923	13.8094	41000	0.6599	0.4103	0.3598	0.3594	0.5218
0.5922	14.1462	42000	0.6677	0.4348	0.3807	0.3840	0.5154
0.574	14.4830	43000	0.6891	0.4173	0.3565	0.3567	0.533
0.5791	14.8198	44000	0.6604	0.4111	0.3740	0.3764	0.526
0.5699	15.1566	45000	0.6624	0.4074	0.3705	0.3719	0.5212
0.5476	15.4934	46000	0.6730	0.4113	0.3546	0.3539	0.5272
0.5963	15.8302	47000	0.6700	0.4097	0.3622	0.3636	0.5302
0.5678	16.1671	48000	0.6608	0.4102	0.3622	0.3621	0.5242
0.5599	16.5039	49000	0.6739	0.4114	0.3590	0.3598	0.5294
0.5458	16.8407	50000	0.6613	0.4135	0.3672	0.3687	0.5274
0.5157	17.1775	51000	0.6742	0.4032	0.3588	0.3597	0.5282
0.5649	17.5143	52000	0.6614	0.4036	0.3661	0.3672	0.5166
0.5729	17.8511	53000	0.6627	0.4018	0.3632	0.3645	0.5096
0.5705	18.1879	54000	0.6613	0.4076	0.3656	0.3667	0.522
0.5329	18.5248	55000	0.6601	0.4103	0.3665	0.3685	0.5196
0.551	18.8616	56000	0.6615	0.4162	0.3675	0.3692	0.5262
0.5571	19.1984	57000	0.6643	0.4113	0.3662	0.3680	0.528
0.5443	19.5352	58000	0.6620	0.4104	0.3660	0.3680	0.5228
0.5463	19.8720	59000	0.6614	0.4078	0.3677	0.3692	0.5186

Framework versions

• Transformers 4.53.2
• Pytorch 2.7.1+cu126
• Datasets 4.0.0
• Tokenizers 0.21.2