train_multirc_1753094164

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1492
• Num Input Tokens Seen: 132272272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2959	0.5	3065	0.1770	6639424
0.1984	1.0	6130	0.1572	13255424
0.0889	1.5	9195	0.1520	19871232
0.1146	2.0	12260	0.1492	26471216
0.1268	2.5	15325	0.1545	33075856
0.1007	3.0	18390	0.1629	39694112
0.1521	3.5	21455	0.1603	46313216
0.006	4.0	24520	0.1501	52929744
0.3003	4.5	27585	0.1589	59549072
0.1177	5.0	30650	0.1592	66152480
0.0486	5.5	33715	0.1672	72765696
0.0755	6.0	36780	0.1772	79389648
0.0772	6.5	39845	0.1912	86008784
0.0286	7.0	42910	0.1884	92621824
0.1522	7.5	45975	0.1887	99237152
0.0034	8.0	49040	0.1856	105830544
0.0042	8.5	52105	0.1977	112458064
0.0036	9.0	55170	0.1930	119047920
0.1249	9.5	58235	0.1925	125686064
0.0941	10.0	61300	0.1924	132272272

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1