train_qqp_1753094139

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qqp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0913
• Num Input Tokens Seen: 250787112

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1238	0.5	40933	0.1127	12552544
0.1039	1.0	81866	0.1053	25087944
0.0958	1.5	122799	0.0913	37621672
0.0725	2.0	163732	0.0958	50164864
0.0595	2.5	204665	0.1253	62700096
0.0661	3.0	245598	0.1075	75242048
0.0009	3.5	286531	0.1315	87769248
0.059	4.0	327464	0.1418	100320328
0.0005	4.5	368397	0.1872	112855464
0.0018	5.0	409330	0.1745	125387608
0.0009	5.5	450263	0.1917	137931704
0.0001	6.0	491196	0.2095	150463800
0.0	6.5	532129	0.2719	163003576
0.0063	7.0	573062	0.2258	175543400
0.0	7.5	613995	0.2751	188096200
0.0001	8.0	654928	0.2579	200622552
0.0	8.5	695861	0.3145	213151000
0.0	9.0	736794	0.3225	225701792
0.0	9.5	777727	0.3955	238243968
0.0	10.0	818660	0.4010	250787112

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1