clapAI/phobert-base-v1-VSMEC-ep80

clapAI/phobert-base-v1-VSMEC-ep50

This model is a fine-tuned version of clapAI/phobert-base-v1-VSMEC-ep30 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2812
• Micro F1: 63.2653
• Micro Precision: 63.2653
• Micro Recall: 63.2653
• Macro F1: 59.2545
• Macro Precision: 59.4208
• Macro Recall: 60.0346

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 512
• eval_batch_size: 512
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 2
• total_train_batch_size: 1024
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Micro F1	Micro Precision	Micro Recall	Macro F1	Macro Precision	Macro Recall
0.7492	1.0	6	1.1406	60.2041	60.2041	60.2041	54.3501	56.1563	54.1594
0.6788	2.0	12	1.1406	60.2041	60.2041	60.2041	54.3501	56.1563	54.1594
0.6642	3.0	18	1.1406	60.2041	60.2041	60.2041	54.3501	56.1563	54.1594
0.6194	4.0	24	1.3242	56.5598	56.5598	56.5598	49.0766	53.3981	49.6779
0.6299	5.0	30	1.1484	61.2245	61.2245	61.2245	54.8351	57.0638	54.5338
0.6613	6.0	36	1.2061	58.7464	58.7464	58.7464	52.6458	54.2105	53.1790
0.494	7.0	42	1.2314	60.9329	60.9329	60.9329	53.2413	59.2943	52.7742
0.4942	8.0	48	1.2256	61.0787	61.0787	61.0787	55.6521	55.1524	56.5297
0.4852	9.0	54	1.2627	60.9329	60.9329	60.9329	55.5971	58.5970	56.0129
0.4238	10.0	60	1.2344	61.2245	61.2245	61.2245	56.3638	55.9710	57.4674
0.4349	11.0	66	1.2412	62.9738	62.9738	62.9738	58.8606	59.8777	59.1984
0.3519	12.0	72	1.2754	61.8076	61.8076	61.8076	57.6704	56.6056	59.7323
0.3324	13.0	78	1.2812	63.2653	63.2653	63.2653	59.2545	59.4208	60.0346
0.3159	14.0	84	1.3340	62.5364	62.5364	62.5364	59.0794	58.8172	60.7730
0.2748	15.0	90	1.3438	61.3703	61.3703	61.3703	57.2733	56.2081	59.2583
0.3046	16.0	96	1.3545	62.6822	62.6822	62.6822	58.8793	58.7409	59.8721
0.2628	16.7273	100	1.3525	61.8076	61.8076	61.8076	58.1471	57.7131	59.4325

Framework versions

• Transformers 4.50.0
• Pytorch 2.4.0+cu121
• Datasets 2.15.0
• Tokenizers 0.21.1