train_boolq_1755555080

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.3287
• Num Input Tokens Seen: 21342336

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.2842	0.5002	1061	1.2705	1069568
0.3203	1.0005	2122	0.3713	2133248
0.3396	1.5007	3183	0.3413	3194016
0.3162	2.0009	4244	0.3389	4266592
0.3763	2.5012	5305	0.3402	5343200
0.3416	3.0014	6366	0.3455	6407840
0.3181	3.5017	7427	0.3338	7476544
0.319	4.0019	8488	0.3427	8540672
0.3932	4.5021	9549	0.3342	9613088
0.3826	5.0024	10610	0.3316	10682528
0.3462	5.5026	11671	0.3337	11757792
0.3193	6.0028	12732	0.3320	12820704
0.3118	6.5031	13793	0.3314	13892416
0.2944	7.0033	14854	0.3305	14957120
0.3672	7.5035	15915	0.3302	16020704
0.2215	8.0038	16976	0.3287	17090912
0.3548	8.5040	18037	0.3313	18157184
0.3066	9.0042	19098	0.3300	19222688
0.3451	9.5045	20159	0.3298	20290176

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1