train_qqp_1753094138

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the qqp dataset. It achieves the following results on the evaluation set:

• Loss: 0.0969
• Num Input Tokens Seen: 250787112

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0904	0.5	40933	0.1278	12552544
0.2432	1.0	81866	0.1161	25087944
0.1091	1.5	122799	0.1090	37621672
0.0788	2.0	163732	0.1051	50164864
0.1158	2.5	204665	0.1038	62700096
0.0957	3.0	245598	0.1009	75242048
0.132	3.5	286531	0.1031	87769248
0.2164	4.0	327464	0.0996	100320328
0.0902	4.5	368397	0.0997	112855464
0.1326	5.0	409330	0.0972	125387608
0.0654	5.5	450263	0.0987	137931704
0.0854	6.0	491196	0.0973	150463800
0.0535	6.5	532129	0.0991	163003576
0.0614	7.0	573062	0.0970	175543400
0.0201	7.5	613995	0.0969	188096200
0.0454	8.0	654928	0.0975	200622552
0.0318	8.5	695861	0.0976	213151000
0.1643	9.0	736794	0.0976	225701792
0.033	9.5	777727	0.0974	238243968
0.0997	10.0	818660	0.0974	250787112

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1