train_copa_101112_1757596165

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.9577
• Num Input Tokens Seen: 281312

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2006	0.5	45	0.1967	14144
0.3225	1.0	90	0.0856	28192
0.4327	1.5	135	0.0478	42208
0.0202	2.0	180	0.0775	56256
0.1742	2.5	225	0.0552	70368
0.0049	3.0	270	0.0273	84320
0.0011	3.5	315	0.0583	98400
0.0018	4.0	360	0.0332	112416
0.0013	4.5	405	0.0406	126496
0.0002	5.0	450	0.0364	140544
0.0001	5.5	495	0.0473	154592
0.0001	6.0	540	0.0446	168768
0.0001	6.5	585	0.0423	182848
0.0	7.0	630	0.0465	196896
0.0	7.5	675	0.0435	210912
0.0	8.0	720	0.0428	225024
0.0	8.5	765	0.0453	239200
0.0	9.0	810	0.0443	253152
0.0	9.5	855	0.0495	267040
0.0	10.0	900	0.0484	281312

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1