train_mnli_1753093711

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.1169
• Num Input Tokens Seen: 347859920

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3645	0.5	44179	0.3380	17403808
0.2859	1.0	88358	0.2381	34786008
0.1129	1.5	132537	0.1936	52165240
0.1375	2.0	176716	0.1687	69564424
0.1106	2.5	220895	0.1551	86951080
0.136	3.0	265074	0.1450	104352808
0.1519	3.5	309253	0.1369	121746504
0.1935	4.0	353432	0.1335	139123792
0.1464	4.5	397611	0.1286	156526672
0.1063	5.0	441790	0.1246	173916408
0.0987	5.5	485969	0.1229	191309592
0.1157	6.0	530148	0.1214	208701328
0.1015	6.5	574327	0.1207	226098768
0.1043	7.0	618506	0.1187	243493272
0.1012	7.5	662685	0.1183	260881240
0.1025	8.0	706864	0.1175	278276232
0.0814	8.5	751043	0.1173	295687496
0.1493	9.0	795222	0.1169	313062872
0.1139	9.5	839401	0.1170	330444056
0.0418	10.0	883580	0.1169	347859920

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1