train_multirc_1753094162

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1357
• Num Input Tokens Seen: 132272272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2946	0.5	3065	0.1869	6639424
0.2248	1.0	6130	0.1595	13255424
0.0889	1.5	9195	0.1496	19871232
0.1594	2.0	12260	0.1460	26471216
0.0945	2.5	15325	0.1446	33075856
0.1203	3.0	18390	0.1438	39694112
0.1356	3.5	21455	0.1404	46313216
0.0207	4.0	24520	0.1368	52929744
0.3164	4.5	27585	0.1374	59549072
0.1342	5.0	30650	0.1362	66152480
0.1577	5.5	33715	0.1376	72765696
0.0909	6.0	36780	0.1374	79389648
0.0616	6.5	39845	0.1406	86008784
0.0944	7.0	42910	0.1389	92621824
0.1514	7.5	45975	0.1381	99237152
0.0194	8.0	49040	0.1357	105830544
0.0666	8.5	52105	0.1389	112458064
0.0326	9.0	55170	0.1372	119047920
0.1196	9.5	58235	0.1371	125686064
0.1177	10.0	61300	0.1369	132272272

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1