train_boolq_1753094166

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1579
• Num Input Tokens Seen: 21342336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6818	0.5002	1061	0.1805	1069568
0.0215	1.0005	2122	0.1579	2133248
0.1723	1.5007	3183	0.1975	3194016
0.0653	2.0009	4244	0.1596	4266592
0.1114	2.5012	5305	0.2336	5343200
0.0032	3.0014	6366	0.1998	6407840
0.0005	3.5017	7427	0.2926	7476544
0.0006	4.0019	8488	0.2855	8540672
0.0001	4.5021	9549	0.4418	9613088
0.0004	5.0024	10610	0.4177	10682528
0.0004	5.5026	11671	0.3844	11757792
0.0001	6.0028	12732	0.4755	12820704
0.0	6.5031	13793	0.5053	13892416
0.004	7.0033	14854	0.4459	14957120
0.0	7.5035	15915	0.5498	16020704
0.0	8.0038	16976	0.5147	17090912
0.0	8.5040	18037	0.5662	18157184
0.0	9.0042	19098	0.5899	19222688
0.0	9.5045	20159	0.5972	20290176

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1