train_codealpacapy_1755551519

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 4.8186
• Num Input Tokens Seen: 12472912

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
8.26	0.5	954	8.4784	616992
7.2055	1.0	1908	6.7617	1248304
5.7336	1.5	2862	6.1830	1877040
5.889	2.0	3816	5.9066	2497016
6.3024	2.5	4770	5.7254	3129368
5.6745	3.0	5724	5.5857	3742552
5.552	3.5	6678	5.4694	4361944
4.9018	4.0	7632	5.3500	4985200
5.0486	4.5	8586	5.2432	5611760
5.3254	5.0	9540	5.1530	6233920
4.9765	5.5	10494	5.0725	6849184
5.0447	6.0	11448	5.0027	7478504
4.4829	6.5	12402	4.9468	8083560
4.8278	7.0	13356	4.9016	8722744
4.4256	7.5	14310	4.8703	9345976
4.8333	8.0	15264	4.8446	9977520
4.065	8.5	16218	4.8296	10604656
5.2516	9.0	17172	4.8221	11225416
4.5529	9.5	18126	4.8194	11845704
4.9849	10.0	19080	4.8186	12472912

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1