train_boolq_1753094165

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1533
• Num Input Tokens Seen: 21342336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7445	0.5002	1061	0.2372	1069568
0.2202	1.0005	2122	0.2001	2133248
0.1428	1.5007	3183	0.1845	3194016
0.1635	2.0009	4244	0.1737	4266592
0.1989	2.5012	5305	0.1689	5343200
0.1422	3.0014	6366	0.1670	6407840
0.2293	3.5017	7427	0.1613	7476544
0.1897	4.0019	8488	0.1626	8540672
0.1142	4.5021	9549	0.1573	9613088
0.0582	5.0024	10610	0.1572	10682528
0.1723	5.5026	11671	0.1558	11757792
0.0985	6.0028	12732	0.1557	12820704
0.1387	6.5031	13793	0.1552	13892416
0.2209	7.0033	14854	0.1540	14957120
0.1787	7.5035	15915	0.1533	16020704
0.0452	8.0038	16976	0.1543	17090912
0.2625	8.5040	18037	0.1536	18157184
0.0591	9.0042	19098	0.1539	19222688
0.1612	9.5045	20159	0.1538	20290176

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1