train_boolq_1753094167

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1641
• Num Input Tokens Seen: 21342336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7593	0.5002	1061	0.2250	1069568
0.1768	1.0005	2122	0.1875	2133248
0.1648	1.5007	3183	0.1822	3194016
0.1705	2.0009	4244	0.1641	4266592
0.0984	2.5012	5305	0.1677	5343200
0.1156	3.0014	6366	0.1647	6407840
0.1606	3.5017	7427	0.1721	7476544
0.0845	4.0019	8488	0.1736	8540672
0.0972	4.5021	9549	0.1735	9613088
0.0107	5.0024	10610	0.1824	10682528
0.1973	5.5026	11671	0.1916	11757792
0.0982	6.0028	12732	0.1825	12820704
0.1888	6.5031	13793	0.1930	13892416
0.2615	7.0033	14854	0.1939	14957120
0.1862	7.5035	15915	0.1987	16020704
0.0053	8.0038	16976	0.2059	17090912
0.2801	8.5040	18037	0.2077	18157184
0.0042	9.0042	19098	0.2077	19222688
0.1154	9.5045	20159	0.2080	20290176

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1