train_cola_1754652147

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.2484
• Num Input Tokens Seen: 3669168

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4911	0.5	962	0.4523	184192
0.1788	1.0	1924	0.3084	367320
0.2984	1.5	2886	0.2642	550840
0.2195	2.0	3848	0.2571	734600
0.2629	2.5	4810	0.2611	918600
0.2186	3.0	5772	0.2509	1101216
0.266	3.5	6734	0.2540	1284288
0.2514	4.0	7696	0.2523	1468552
0.2335	4.5	8658	0.2568	1651528
0.247	5.0	9620	0.2540	1834816
0.2706	5.5	10582	0.2504	2018016
0.2512	6.0	11544	0.2505	2201584
0.2484	6.5	12506	0.2499	2385200
0.2463	7.0	13468	0.2510	2568288
0.2712	7.5	14430	0.2528	2751584
0.2327	8.0	15392	0.2523	2935056
0.2754	8.5	16354	0.2484	3118000
0.2497	9.0	17316	0.2506	3301760
0.2494	9.5	18278	0.2497	3485344
0.2594	10.0	19240	0.2495	3669168

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1