clapAI/phobert-base-v2-VSMEC-ep30

This model is a fine-tuned version of vinai/phobert-base-v2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1583
• Micro F1: 62.0991
• Micro Precision: 62.0991
• Micro Recall: 62.0991
• Macro F1: 54.8464
• Macro Precision: 56.9883
• Macro Recall: 53.8477

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 30.0

Training results

Training Loss	Epoch	Step	Validation Loss	Micro F1	Micro Precision	Micro Recall	Macro F1	Macro Precision	Macro Recall
1.7449	1.0	22	1.5945	45.1895	45.1895	45.1895	21.8900	26.9959	26.4581
1.4133	2.0	44	1.3106	54.8105	54.8105	54.8105	31.9928	29.9948	34.9272
1.175	3.0	66	1.2191	58.0175	58.0175	58.0175	36.7657	55.0529	38.6372
1.0596	4.0	88	1.1273	61.0787	61.0787	61.0787	48.3338	58.4381	47.7289
0.7949	5.0	110	1.1273	61.6618	61.6618	61.6618	53.0643	58.9490	53.2467
0.7303	6.0	132	1.1650	61.8076	61.8076	61.8076	55.9052	56.1562	58.1667
0.5736	7.0	154	1.1583	62.0991	62.0991	62.0991	54.8464	56.9883	53.8477
0.5044	8.0	176	1.2125	60.4956	60.4956	60.4956	54.4159	54.5775	55.6200
0.3788	9.0	198	1.2797	60.6414	60.6414	60.6414	55.5452	55.2250	57.9434
0.3342	10.0	220	1.3261	59.9125	59.9125	59.9125	54.1145	55.1820	55.2814
0.2737	11.0	242	1.4136	60.2041	60.2041	60.2041	53.4261	53.8931	56.8025
0.2206	12.0	264	1.4722	59.7668	59.7668	59.7668	52.5744	53.0730	53.5133
0.2027	13.0	286	1.4627	61.2245	61.2245	61.2245	56.6680	55.6226	58.6076
0.1578	14.0	308	1.5438	58.6006	58.6006	58.6006	52.1842	52.1753	53.1364
0.1236	15.0	330	1.5138	61.8076	61.8076	61.8076	55.5209	55.9526	55.4750
0.1341	16.0	352	1.6036	61.0787	61.0787	61.0787	56.2561	56.2149	57.6631
0.1001	17.0	374	1.6289	60.7872	60.7872	60.7872	56.2241	55.6502	58.4228
0.1029	18.0	396	1.6524	60.9329	60.9329	60.9329	56.0318	54.9867	57.7622
0.0843	19.0	418	1.6548	61.6618	61.6618	61.6618	56.2764	55.9093	57.2296
0.062	20.0	440	1.7255	61.0787	61.0787	61.0787	56.6280	55.9351	57.8994
0.0569	21.0	462	1.7440	60.3499	60.3499	60.3499	55.8524	55.9810	56.5576
0.0701	22.0	484	1.7638	61.2245	61.2245	61.2245	56.4928	56.5271	57.1536
0.0461	23.0	506	1.7829	60.9329	60.9329	60.9329	55.7281	55.4889	56.9253
0.0405	24.0	528	1.7763	61.2245	61.2245	61.2245	55.8934	55.9735	56.7029
0.0442	25.0	550	1.7976	60.7872	60.7872	60.7872	55.5945	55.6735	56.3259
0.0583	26.0	572	1.8069	61.0787	61.0787	61.0787	56.0711	56.1338	56.7661
0.0485	27.0	594	1.8212	60.7872	60.7872	60.7872	55.5985	55.6477	56.5246
0.0466	28.0	616	1.8198	60.7872	60.7872	60.7872	55.5764	55.5871	56.4900
0.0571	28.6437	630	1.8183	60.7872	60.7872	60.7872	55.5691	55.5735	56.4253

Framework versions

• Transformers 4.50.0
• Pytorch 2.4.0+cu121
• Datasets 2.15.0
• Tokenizers 0.21.1