train_copa_1753094178

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0424
• Num Input Tokens Seen: 281856

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0489	0.5	45	0.1557	14016
0.2106	1.0	90	0.0874	28096
0.0645	1.5	135	0.1064	42144
0.1351	2.0	180	0.0424	56128
0.0005	2.5	225	0.0604	70272
0.0005	3.0	270	0.0472	84352
0.0	3.5	315	0.0488	98464
0.0	4.0	360	0.0504	112576
0.0	4.5	405	0.0520	126624
0.0	5.0	450	0.0500	140832
0.0	5.5	495	0.0497	154976
0.0	6.0	540	0.0507	169056
0.0	6.5	585	0.0505	183200
0.0	7.0	630	0.0532	197344
0.0	7.5	675	0.0490	211392
0.0	8.0	720	0.0466	225536
0.0	8.5	765	0.0529	239680
0.0	9.0	810	0.0514	253696
0.0	9.5	855	0.0501	267840
0.0	10.0	900	0.0506	281856

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1