clapAI/ModernBERT-base-ViHSD-ep50

This model is a fine-tuned version of answerdotai/ModernBERT-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1876
• Micro F1: 89.9371
• Micro Precision: 89.9371
• Micro Recall: 89.9371
• Macro F1: 86.0668
• Macro Precision: 88.4407
• Macro Recall: 84.2926

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 50.0

Training results

Training Loss	Epoch	Step	Validation Loss	Micro F1	Micro Precision	Micro Recall	Macro F1	Macro Precision	Macro Recall
1.541	1.0	56	0.3452	85.8491	85.8491	85.8491	78.4177	86.0731	75.0986
1.0652	2.0	112	0.3162	87.4214	87.4214	87.4214	82.5192	84.8517	80.8292
1.0885	3.0	168	0.2911	88.0503	88.0503	88.0503	83.9452	84.8136	83.1858
0.8368	4.0	224	0.2861	88.8050	88.8050	88.8050	83.7964	88.6088	80.9531
0.7777	5.0	280	0.2805	89.3082	89.3082	89.3082	85.3561	87.1360	83.9506
0.4158	6.0	336	0.3528	88.4277	88.4277	88.4277	84.3174	85.5629	83.2783
0.3086	7.0	392	0.4122	89.4340	89.4340	89.4340	85.1781	88.2062	83.0681
0.191	8.0	448	0.4914	88.8050	88.8050	88.8050	84.5839	86.5890	83.0483
0.1652	9.0	504	0.5783	88.3019	88.3019	88.3019	83.9491	85.7456	82.5491
0.1177	10.0	560	0.5563	87.3585	87.3585	87.3585	83.7779	83.0858	84.5744
0.1236	11.0	616	0.7119	85.4717	85.4717	85.4717	82.1129	80.6222	84.4346
0.058	12.0	672	0.6722	88.3019	88.3019	88.3019	84.2832	85.1600	83.5161
0.0767	13.0	728	0.6633	88.6164	88.6164	88.6164	84.3665	86.2274	82.9215
0.0607	14.0	784	0.6969	88.8050	88.8050	88.8050	85.3984	85.1782	85.6271
0.066	15.0	840	0.9945	88.6164	88.6164	88.6164	83.1966	89.3007	79.9399
0.0474	16.0	896	0.8278	88.1761	88.1761	88.1761	84.6267	84.3094	84.9626
0.0634	17.0	952	0.7015	87.2956	87.2956	87.2956	83.6848	83.0154	84.4515
0.0188	18.0	1008	0.9059	88.8679	88.8679	88.8679	85.1323	85.7851	84.5411
0.028	19.0	1064	0.9812	89.3082	89.3082	89.3082	85.3034	87.2576	83.7894
0.0704	20.0	1120	0.9311	88.3648	88.3648	88.3648	84.5892	84.9113	84.2836
0.0363	21.0	1176	0.9205	88.7421	88.7421	88.7421	85.4197	84.9700	85.9071
0.0025	22.0	1232	0.9776	89.6855	89.6855	89.6855	85.7964	87.8686	84.2042
0.0188	23.0	1288	1.1122	88.9937	88.9937	88.9937	84.6616	87.2876	82.7722
0.0282	24.0	1344	1.0915	89.3711	89.3711	89.3711	85.2695	87.6416	83.5093
0.0136	25.0	1400	1.1382	88.9308	88.9308	88.9308	84.7294	86.8333	83.1329
0.0	26.0	1456	1.1641	89.6855	89.6855	89.6855	85.8476	87.7402	84.3653
0.0	27.0	1512	1.1644	89.7484	89.7484	89.7484	85.8702	87.9931	84.2464
0.0	28.0	1568	1.1839	89.6855	89.6855	89.6855	85.7446	88.0020	84.0430
0.0	29.0	1624	1.1693	89.6226	89.6226	89.6226	85.7741	87.6184	84.3231
0.0	30.0	1680	1.1933	89.5597	89.5597	89.5597	85.4909	88.0249	83.6361
0.0063	31.0	1736	1.1838	89.4969	89.4969	89.4969	85.5497	87.5624	83.9968
0.0013	32.0	1792	1.1904	89.8113	89.8113	89.8113	85.8926	88.2556	84.1275
0.0	33.0	1848	1.1758	89.6855	89.6855	89.6855	85.8221	87.8038	84.2847
0.0009	34.0	1904	1.1772	89.4969	89.4969	89.4969	85.5758	87.4993	84.0774
0.0	35.0	1960	1.1786	89.8742	89.8742	89.8742	86.0689	88.1097	84.4921
0.0069	36.0	2016	1.1819	89.6855	89.6855	89.6855	85.8221	87.8038	84.2847
0.001	37.0	2072	1.1876	89.9371	89.9371	89.9371	86.0668	88.4407	84.2926
0.0	38.0	2128	1.1881	89.5597	89.5597	89.5597	85.5971	87.7511	83.9585
0.0	39.0	2184	1.1881	89.5597	89.5597	89.5597	85.5971	87.7511	83.9585
0.0	40.0	2240	1.1869	89.6855	89.6855	89.6855	85.7706	87.9347	84.1236
0.001	41.0	2296	1.1930	89.8742	89.8742	89.8742	85.9668	88.3835	84.1698
0.0	42.0	2352	1.1892	89.8113	89.8113	89.8113	85.9185	88.1863	84.2081
0.0	43.0	2408	1.1840	89.6855	89.6855	89.6855	85.7446	88.0020	84.0430
0.0	44.0	2464	1.1925	89.5597	89.5597	89.5597	85.6232	87.6858	84.0390
0.0	45.0	2520	1.1892	89.6226	89.6226	89.6226	85.6968	87.8099	84.0813
0.0009	46.0	2576	1.1895	89.7484	89.7484	89.7484	85.8186	88.1285	84.0852
0.0	47.0	2632	1.1887	89.8742	89.8742	89.8742	86.0182	88.2441	84.3309
0.0	48.0	2688	1.1933	89.7484	89.7484	89.7484	85.8445	88.0601	84.1658
0.0	49.0	2744	1.1901	89.7484	89.7484	89.7484	85.8186	88.1285	84.0852
0.0043	50.0	2800	1.1904	89.6226	89.6226	89.6226	85.6708	87.8762	84.0007

Framework versions

• Transformers 4.50.0
• Pytorch 2.4.0+cu121
• Datasets 2.15.0
• Tokenizers 0.21.1