train_record_1753094159

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3168
• Num Input Tokens Seen: 464483424

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5452	0.5	15621	0.5554	23227520
0.441	1.0	31242	0.4424	46454112
0.4276	1.5	46863	0.3949	69694624
0.2116	2.0	62484	0.3702	92908288
0.2065	2.5	78105	0.3554	116099296
0.2894	3.0	93726	0.3452	139351808
0.3357	3.5	109347	0.3372	162566976
0.322	4.0	124968	0.3318	185790304
0.2549	4.5	140589	0.3277	208997696
0.3211	5.0	156210	0.3244	232243968
0.2328	5.5	171831	0.3230	255458112
0.24	6.0	187452	0.3202	278686752
0.3243	6.5	203073	0.3204	301925344
0.2331	7.0	218694	0.3179	325137568
0.3213	7.5	234315	0.3172	348361920
0.2405	8.0	249936	0.3176	371592704
0.2168	8.5	265557	0.3172	394838368
0.2821	9.0	281178	0.3168	418033696
0.3505	9.5	296799	0.3169	441282560
0.3429	10.0	312420	0.3170	464483424

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1