train_boolq_1745950280

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.3260
• Num Input Tokens Seen: 37097424

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.3
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 40000

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3571	0.0943	200	0.3353	186768
0.6655	0.1886	400	1.2637	369808
0.4242	0.2829	600	0.3287	554928
0.2999	0.3772	800	0.3277	746560
0.3501	0.4715	1000	0.3283	932848
0.3099	0.5658	1200	0.3302	1116128
0.3801	0.6601	1400	0.3414	1299664
0.2952	0.7544	1600	0.3291	1481856
0.3585	0.8487	1800	0.3288	1672160
0.3976	0.9430	2000	0.3275	1860608
0.359	1.0372	2200	0.3295	2047984
0.2945	1.1315	2400	0.3282	2230960
0.3179	1.2258	2600	0.3287	2417664
0.3535	1.3201	2800	0.3271	2600368
0.2831	1.4144	3000	0.3374	2786848
0.3071	1.5087	3200	0.3292	2972672
0.3365	1.6030	3400	0.3338	3154640
0.3532	1.6973	3600	0.3458	3339328
0.3734	1.7916	3800	0.3275	3522384
0.294	1.8859	4000	0.3281	3712352
0.3144	1.9802	4200	0.3302	3899328
0.3425	2.0745	4400	0.3273	4085888
0.4222	2.1688	4600	0.3266	4271936
0.2797	2.2631	4800	0.3314	4456320
0.3179	2.3574	5000	0.3278	4638512
0.3202	2.4517	5200	0.3283	4830688
0.304	2.5460	5400	0.3269	5016480
0.3414	2.6403	5600	0.3288	5204048
0.3502	2.7346	5800	0.3281	5383984
0.376	2.8289	6000	0.3293	5574016
0.3344	2.9231	6200	0.3377	5761616
0.3395	3.0174	6400	0.3289	5948128
0.352	3.1117	6600	0.3286	6134304
0.3654	3.2060	6800	0.3271	6319616
0.3445	3.3003	7000	0.3292	6505744
0.3611	3.3946	7200	0.3290	6692208
0.3608	3.4889	7400	0.3350	6875616
0.3623	3.5832	7600	0.3289	7059472
0.2933	3.6775	7800	0.3278	7243472
0.3393	3.7718	8000	0.3299	7428048
0.3473	3.8661	8200	0.3291	7611184
0.3928	3.9604	8400	0.3353	7796112
0.3407	4.0547	8600	0.3320	7979520
0.3312	4.1490	8800	0.3279	8167776
0.2816	4.2433	9000	0.3280	8355856
0.3064	4.3376	9200	0.3321	8543120
0.3669	4.4319	9400	0.3272	8727088
0.3667	4.5262	9600	0.3289	8914992
0.5646	4.6205	9800	0.3886	9095040
0.3517	4.7148	10000	0.3328	9283072
0.3462	4.8091	10200	0.3449	9467600
0.3709	4.9033	10400	0.3676	9653456
0.3385	4.9976	10600	0.3286	9841232
0.3236	5.0919	10800	0.3289	10025504
0.3695	5.1862	11000	0.3284	10216464
0.2978	5.2805	11200	0.3291	10402448
0.2926	5.3748	11400	0.3312	10586976
0.3201	5.4691	11600	0.3283	10770896
0.3963	5.5634	11800	0.3280	10959424
0.3661	5.6577	12000	0.3312	11146816
0.3531	5.7520	12200	0.3316	11328528
0.3431	5.8463	12400	0.3332	11515600
0.2751	5.9406	12600	0.3284	11697056
0.3721	6.0349	12800	0.3386	11884336
0.3374	6.1292	13000	0.3296	12074128
0.3791	6.2235	13200	0.3321	12258064
0.3481	6.3178	13400	0.3273	12443248
0.2563	6.4121	13600	0.3305	12626480
0.2924	6.5064	13800	0.3275	12813808
0.343	6.6007	14000	0.3342	12998256
0.4058	6.6950	14200	0.3272	13180928
0.3372	6.7893	14400	0.3324	13364368
0.3641	6.8835	14600	0.3278	13552272
0.3362	6.9778	14800	0.3312	13735904
0.3351	7.0721	15000	0.3312	13924000
0.343	7.1664	15200	0.3343	14113184
0.348	7.2607	15400	0.3388	14295568
0.3783	7.3550	15600	0.3273	14480560
0.4022	7.4493	15800	0.3331	14664736
0.3086	7.5436	16000	0.3277	14852128
0.338	7.6379	16200	0.3275	15033840
0.3371	7.7322	16400	0.3274	15219136
0.3172	7.8265	16600	0.3261	15404160
0.3308	7.9208	16800	0.3270	15589632
0.2996	8.0151	17000	0.3276	15781760
0.3615	8.1094	17200	0.3266	15967648
0.264	8.2037	17400	0.3272	16155248
0.3241	8.2980	17600	0.3269	16343648
0.3462	8.3923	17800	0.3293	16523360
0.3463	8.4866	18000	0.3384	16709008
0.3472	8.5809	18200	0.3287	16893648
0.3603	8.6752	18400	0.3282	17079824
0.3141	8.7694	18600	0.3294	17265072
0.3251	8.8637	18800	0.3276	17445904
0.3337	8.9580	19000	0.3269	17631504
0.3541	9.0523	19200	0.3276	17818512
0.3245	9.1466	19400	0.3277	18005200
0.3733	9.2409	19600	0.3266	18190416
0.3328	9.3352	19800	0.3274	18373200
0.3801	9.4295	20000	0.3274	18556672
0.3399	9.5238	20200	0.3290	18742816
0.3368	9.6181	20400	0.3276	18930224
0.3135	9.7124	20600	0.3280	19115456
0.3251	9.8067	20800	0.3301	19296016
0.3095	9.9010	21000	0.3281	19482416
0.3491	9.9953	21200	0.3279	19668640
0.324	10.0896	21400	0.3267	19860880
0.3144	10.1839	21600	0.3266	20052672
0.335	10.2782	21800	0.3275	20236224
0.3269	10.3725	22000	0.3287	20421632
0.3284	10.4668	22200	0.3282	20608320
0.3202	10.5611	22400	0.3301	20788112
0.3279	10.6554	22600	0.3290	20969744
0.3447	10.7496	22800	0.3268	21151648
0.3432	10.8439	23000	0.3269	21335600
0.353	10.9382	23200	0.3271	21522352
0.3276	11.0325	23400	0.3275	21709568
0.325	11.1268	23600	0.3270	21894592
0.3204	11.2211	23800	0.3273	22079344
0.3423	11.3154	24000	0.3278	22269152
0.3098	11.4097	24200	0.3266	22451760
0.3876	11.5040	24400	0.3276	22639312
0.3428	11.5983	24600	0.3287	22821728
0.3634	11.6926	24800	0.3275	23005696
0.296	11.7869	25000	0.3260	23192112
0.3063	11.8812	25200	0.3262	23373840
0.3055	11.9755	25400	0.3281	23559968
0.3414	12.0698	25600	0.3334	23743680
0.3125	12.1641	25800	0.3279	23931472
0.3537	12.2584	26000	0.3308	24118800
0.3198	12.3527	26200	0.3270	24308976
0.2884	12.4470	26400	0.3279	24493584
0.3503	12.5413	26600	0.3273	24679264
0.3456	12.6355	26800	0.3280	24861136
0.3058	12.7298	27000	0.3267	25046496
0.3181	12.8241	27200	0.3272	25230592
0.3211	12.9184	27400	0.3268	25411904
0.3335	13.0127	27600	0.3284	25595280
0.3438	13.1070	27800	0.3360	25777696
0.3493	13.2013	28000	0.3276	25963552
0.3277	13.2956	28200	0.3268	26150464
0.3618	13.3899	28400	0.3277	26335552
0.3095	13.4842	28600	0.3290	26524096
0.3783	13.5785	28800	0.3298	26713392
0.3054	13.6728	29000	0.3292	26900464
0.3617	13.7671	29200	0.3308	27087040
0.3589	13.8614	29400	0.3291	27270960
0.3306	13.9557	29600	0.3292	27457936
0.3116	14.0500	29800	0.3331	27639216
0.3343	14.1443	30000	0.3289	27829056
0.363	14.2386	30200	0.3274	28019840
0.3075	14.3329	30400	0.3264	28205616
0.3307	14.4272	30600	0.3274	28390464
0.3363	14.5215	30800	0.3279	28571424
0.3314	14.6157	31000	0.3272	28758128
0.3598	14.7100	31200	0.3294	28942096
0.3207	14.8043	31400	0.3290	29127440
0.3102	14.8986	31600	0.3290	29310016
0.2936	14.9929	31800	0.3286	29497520
0.3225	15.0872	32000	0.3301	29680160
0.3111	15.1815	32200	0.3300	29872080
0.2723	15.2758	32400	0.3283	30060048
0.3502	15.3701	32600	0.3284	30243024
0.3462	15.4644	32800	0.3277	30433968
0.3086	15.5587	33000	0.3275	30617936
0.3572	15.6530	33200	0.3278	30802960
0.3382	15.7473	33400	0.3282	30985296
0.3195	15.8416	33600	0.3279	31168496
0.2936	15.9359	33800	0.3295	31350688
0.316	16.0302	34000	0.3288	31530704
0.3678	16.1245	34200	0.3289	31718960
0.3434	16.2188	34400	0.3285	31901696
0.3191	16.3131	34600	0.3290	32092528
0.2926	16.4074	34800	0.3287	32279920
0.3153	16.5017	35000	0.3287	32461952
0.3194	16.5959	35200	0.3286	32647696
0.2946	16.6902	35400	0.3288	32828656
0.3516	16.7845	35600	0.3290	33016320
0.3322	16.8788	35800	0.3295	33202224
0.2848	16.9731	36000	0.3288	33385424
0.3162	17.0674	36200	0.3279	33572672
0.3277	17.1617	36400	0.3288	33759120
0.3295	17.2560	36600	0.3283	33946224
0.3543	17.3503	36800	0.3300	34137504
0.291	17.4446	37000	0.3289	34322448
0.3199	17.5389	37200	0.3286	34506880
0.3043	17.6332	37400	0.3295	34692032
0.272	17.7275	37600	0.3287	34873984
0.3184	17.8218	37800	0.3295	35058576
0.296	17.9161	38000	0.3284	35245152
0.3044	18.0104	38200	0.3294	35431232
0.2966	18.1047	38400	0.3291	35615248
0.2675	18.1990	38600	0.3291	35798688
0.3953	18.2933	38800	0.3287	35984224
0.3121	18.3876	39000	0.3288	36168064
0.284	18.4818	39200	0.3287	36351216
0.3022	18.5761	39400	0.3288	36537456
0.3618	18.6704	39600	0.3286	36723376
0.3315	18.7647	39800	0.3283	36910256
0.3403	18.8590	40000	0.3286	37097424

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1