train_mnli_1753271392

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the mnli dataset. It achieves the following results on the evaluation set:

• Loss: 0.0867
• Num Input Tokens Seen: 347859920

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2676	0.5	44179	0.0973	17403808
0.164	1.0	88358	0.0907	34786008
0.0055	1.5	132537	0.1002	52165240
0.118	2.0	176716	0.0867	69564424
0.0015	2.5	220895	0.1152	86951080
0.0935	3.0	265074	0.1064	104352808
0.0105	3.5	309253	0.1427	121746504
0.0011	4.0	353432	0.1456	139123792
0.0789	4.5	397611	0.1977	156526672
0.0003	5.0	441790	0.1803	173916408
0.0001	5.5	485969	0.2100	191309592
0.0002	6.0	530148	0.2051	208701328
0.0001	6.5	574327	0.2591	226098768
0.0	7.0	618506	0.2438	243493272
0.0	7.5	662685	0.2963	260881240
0.0	8.0	706864	0.3234	278276232
0.0	8.5	751043	0.3484	295687496
0.0	9.0	795222	0.3621	313062872
0.0	9.5	839401	0.3940	330444056
0.0	10.0	883580	0.3969	347859920

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1