train_record_1753094161

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3058
• Num Input Tokens Seen: 464483424

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4176	0.5	15621	0.4658	23227520
0.3199	1.0	31242	0.3683	46454112
0.3515	1.5	46863	0.3365	69694624
0.1657	2.0	62484	0.3196	92908288
0.2076	2.5	78105	0.3097	116099296
0.241	3.0	93726	0.3085	139351808
0.2739	3.5	109347	0.3098	162566976
0.2938	4.0	124968	0.3070	185790304
0.1931	4.5	140589	0.3065	208997696
0.2193	5.0	156210	0.3058	232243968
0.1887	5.5	171831	0.3139	255458112
0.3024	6.0	187452	0.3074	278686752
0.2492	6.5	203073	0.3190	301925344
0.157	7.0	218694	0.3129	325137568
0.2087	7.5	234315	0.3214	348361920
0.1324	8.0	249936	0.3220	371592704
0.1739	8.5	265557	0.3221	394838368
0.2043	9.0	281178	0.3231	418033696
0.2574	9.5	296799	0.3248	441282560
0.3072	10.0	312420	0.3243	464483424

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1