train_multirc_1745950270

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 2.5283
• Num Input Tokens Seen: 83543088

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 40000

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.8061	0.0326	200	2.6757	418192
1.6352	0.0653	400	2.5977	836224
3.1415	0.0979	600	2.5582	1258320
2.0399	0.1305	800	2.5548	1673984
1.762	0.1631	1000	2.5650	2097344
2.7923	0.1958	1200	2.5482	2515056
1.7457	0.2284	1400	2.5522	2943280
1.7453	0.2610	1600	2.5496	3360448
1.8503	0.2937	1800	2.5513	3782768
2.2157	0.3263	2000	2.5354	4205680
2.5347	0.3589	2200	2.5436	4620944
2.6464	0.3915	2400	2.5454	5037232
3.414	0.4242	2600	2.5461	5452736
4.0924	0.4568	2800	2.5363	5872752
1.594	0.4894	3000	2.5560	6285232
2.1049	0.5221	3200	2.5613	6699264
2.4132	0.5547	3400	2.5524	7118336
2.0952	0.5873	3600	2.5529	7533408
2.8358	0.6200	3800	2.5426	7950560
2.2672	0.6526	4000	2.5474	8372672
1.7307	0.6852	4200	2.5581	8796048
2.0188	0.7178	4400	2.5475	9210416
2.0703	0.7505	4600	2.5433	9628832
3.0207	0.7831	4800	2.5367	10048144
2.1577	0.8157	5000	2.5384	10460144
1.4574	0.8484	5200	2.5494	10871296
3.7552	0.8810	5400	2.5325	11287600
1.8036	0.9136	5600	2.5376	11707328
2.9821	0.9462	5800	2.5556	12120624
1.8289	0.9789	6000	2.5492	12542416
1.8497	1.0114	6200	2.5442	12963008
1.2982	1.0440	6400	2.5316	13388336
2.5333	1.0767	6600	2.5480	13816224
2.6627	1.1093	6800	2.5380	14228240
3.0394	1.1419	7000	2.5363	14637984
2.7862	1.1746	7200	2.5418	15049216
2.3596	1.2072	7400	2.5583	15471984
2.6311	1.2398	7600	2.5384	15891152
1.9245	1.2725	7800	2.5406	16309376
1.3522	1.3051	8000	2.5443	16729632
2.9042	1.3377	8200	2.5449	17139952
2.6583	1.3703	8400	2.5401	17557136
2.7435	1.4030	8600	2.5592	17974864
2.7448	1.4356	8800	2.5489	18394320
3.6386	1.4682	9000	2.5528	18820208
1.1522	1.5009	9200	2.5401	19244192
2.7058	1.5335	9400	2.5399	19654192
1.7284	1.5661	9600	2.5552	20077520
2.7139	1.5987	9800	2.5283	20493344
2.3272	1.6314	10000	2.5429	20912896
2.7251	1.6640	10200	2.5555	21328976
3.511	1.6966	10400	2.5309	21752192
2.7897	1.7293	10600	2.5391	22164912
2.4284	1.7619	10800	2.5455	22585216
2.028	1.7945	11000	2.5416	23005600
2.9135	1.8271	11200	2.5485	23413712
1.8818	1.8598	11400	2.5370	23827536
1.3497	1.8924	11600	2.5497	24242256
2.077	1.9250	11800	2.5323	24655408
2.8321	1.9577	12000	2.5416	25074096
1.0373	1.9903	12200	2.5359	25489056
1.368	2.0228	12400	2.5435	25898496
2.2438	2.0555	12600	2.5495	26319696
2.1512	2.0881	12800	2.5462	26744896
1.7513	2.1207	13000	2.5287	27166064
0.9243	2.1534	13200	2.5427	27581264
1.8392	2.1860	13400	2.5633	27988880
2.2886	2.2186	13600	2.5420	28397472
3.2633	2.2512	13800	2.5451	28812800
2.5096	2.2839	14000	2.5461	29222656
2.9558	2.3165	14200	2.5657	29642224
3.4931	2.3491	14400	2.5559	30064704
1.6007	2.3818	14600	2.5469	30481488
2.3649	2.4144	14800	2.5445	30900976
2.3624	2.4470	15000	2.5305	31321184
1.8445	2.4796	15200	2.5533	31730928
2.4441	2.5123	15400	2.5513	32146304
1.2673	2.5449	15600	2.5355	32566096
2.9281	2.5775	15800	2.5534	32981664
3.8721	2.6102	16000	2.5421	33403328
1.8551	2.6428	16200	2.5468	33827808
2.6711	2.6754	16400	2.5521	34245456
1.1239	2.7081	16600	2.5457	34673616
1.4874	2.7407	16800	2.5342	35089872
1.152	2.7733	17000	2.5512	35508944
2.1899	2.8059	17200	2.5564	35922144
3.1566	2.8386	17400	2.5408	36345856
1.2781	2.8712	17600	2.5483	36770688
2.3983	2.9038	17800	2.5336	37194864
1.9318	2.9365	18000	2.5379	37615344
1.6621	2.9691	18200	2.5444	38030400
2.3376	3.0016	18400	2.5351	38435312
3.3812	3.0343	18600	2.5402	38869040
2.2142	3.0669	18800	2.5482	39294832
2.0647	3.0995	19000	2.5384	39706928
2.2582	3.1321	19200	2.5614	40121472
2.1523	3.1648	19400	2.5427	40537792
3.2834	3.1974	19600	2.5321	40958928
2.2189	3.2300	19800	2.5436	41378128
3.4322	3.2627	20000	2.5374	41794480
3.3739	3.2953	20200	2.5433	42208144
2.6455	3.3279	20400	2.5355	42625520
2.4638	3.3606	20600	2.5472	43054848
0.7799	3.3932	20800	2.5442	43472928
1.9196	3.4258	21000	2.5403	43892704
2.9388	3.4584	21200	2.5528	44309408
1.287	3.4911	21400	2.5454	44724144
1.2195	3.5237	21600	2.5339	45143632
1.8093	3.5563	21800	2.5370	45567152
2.1424	3.5890	22000	2.5510	45983168
3.3381	3.6216	22200	2.5418	46401184
1.9925	3.6542	22400	2.5449	46813008
1.7949	3.6868	22600	2.5409	47233968
2.2354	3.7195	22800	2.5383	47650016
2.1881	3.7521	23000	2.5354	48064160
3.1696	3.7847	23200	2.5350	48484384
2.9062	3.8174	23400	2.5572	48897744
1.5842	3.8500	23600	2.5381	49308304
3.0507	3.8826	23800	2.5330	49728368
3.2435	3.9152	24000	2.5345	50140272
2.597	3.9479	24200	2.5380	50557680
2.3862	3.9805	24400	2.5573	50978512
2.1603	4.0131	24600	2.5403	51394160
2.1706	4.0457	24800	2.5451	51821712
1.6054	4.0783	25000	2.5288	52244608
2.0526	4.1109	25200	2.5517	52659888
1.658	4.1436	25400	2.5458	53073648
1.2321	4.1762	25600	2.5428	53493696
2.4438	4.2088	25800	2.5512	53907648
1.8597	4.2415	26000	2.5443	54327568
2.9831	4.2741	26200	2.5389	54743840
2.0809	4.3067	26400	2.5431	55158912
2.8693	4.3393	26600	2.5383	55575232
1.9899	4.3720	26800	2.5557	55994160
2.8289	4.4046	27000	2.5426	56410736
2.7897	4.4372	27200	2.5423	56838864
1.8468	4.4699	27400	2.5330	57245776
2.2285	4.5025	27600	2.5301	57651824
2.8184	4.5351	27800	2.5348	58060672
3.3468	4.5677	28000	2.5454	58475488
3.1295	4.6004	28200	2.5404	58898896
1.6093	4.6330	28400	2.5398	59318976
1.9266	4.6656	28600	2.5383	59739680
2.7749	4.6983	28800	2.5486	60159984
2.3452	4.7309	29000	2.5389	60579344
3.2174	4.7635	29200	2.5337	60992736
2.856	4.7961	29400	2.5531	61414080
3.9432	4.8288	29600	2.5470	61829776
2.0286	4.8614	29800	2.5465	62250704
1.1607	4.8940	30000	2.5402	62662656
1.9432	4.9267	30200	2.5602	63088352
2.651	4.9593	30400	2.5388	63504960
4.0587	4.9919	30600	2.5485	63926432
1.5048	5.0245	30800	2.5377	64346032
2.8357	5.0571	31000	2.5535	64764608
1.7275	5.0897	31200	2.5578	65180560
0.6242	5.1224	31400	2.5458	65600032
2.2156	5.1550	31600	2.5486	66007440
3.6034	5.1876	31800	2.5378	66416480
1.1157	5.2202	32000	2.5337	66829712
2.284	5.2529	32200	2.5511	67253936
1.9202	5.2855	32400	2.5335	67674048
2.2426	5.3181	32600	2.5446	68096656
2.4398	5.3508	32800	2.5287	68521600
3.0732	5.3834	33000	2.5504	68948064
3.9775	5.4160	33200	2.5293	69357008
2.9763	5.4486	33400	2.5489	69771824
4.1438	5.4813	33600	2.5422	70189824
1.7663	5.5139	33800	2.5325	70602704
2.9164	5.5465	34000	2.5480	71032768
3.1158	5.5792	34200	2.5471	71445488
1.7717	5.6118	34400	2.5390	71858096
2.4476	5.6444	34600	2.5338	72276272
1.8602	5.6771	34800	2.5324	72694032
2.1456	5.7097	35000	2.5467	73119856
2.6911	5.7423	35200	2.5467	73537984
1.2754	5.7749	35400	2.5464	73955216
1.6987	5.8076	35600	2.5390	74371040
1.9141	5.8402	35800	2.5438	74795680
2.1558	5.8728	36000	2.5439	75209824
2.556	5.9055	36200	2.5378	75634096
1.1027	5.9381	36400	2.5389	76046144
2.3357	5.9707	36600	2.5388	76453936
1.7899	6.0033	36800	2.5388	76873152
3.1736	6.0359	37000	2.5388	77290000
2.7029	6.0685	37200	2.5388	77708416
1.6779	6.1012	37400	2.5388	78124432
3.2998	6.1338	37600	2.5388	78542400
2.5407	6.1664	37800	2.5388	78968368
1.2663	6.1990	38000	2.5388	79378528
4.8279	6.2317	38200	2.5388	79802112
3.159	6.2643	38400	2.5388	80229344
1.9346	6.2969	38600	2.5388	80643632
1.6984	6.3296	38800	2.5388	81051936
3.1322	6.3622	39000	2.5388	81475504
3.2842	6.3948	39200	2.5388	81889856
2.8503	6.4274	39400	2.5388	82305408
2.0501	6.4601	39600	2.5388	82712800
2.6112	6.4927	39800	2.5388	83128704
2.4207	6.5253	40000	2.5388	83543088

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1