clapAI/phobert-base-v1-VSMEC-ep30

This model is a fine-tuned version of clapAI/phobert-base-v1-VSMEC-ep20 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1406
• Micro F1: 60.2041
• Micro Precision: 60.2041
• Micro Recall: 60.2041
• Macro F1: 54.3501
• Macro Precision: 56.1563
• Macro Recall: 54.1594

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 512
• eval_batch_size: 512
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 2
• total_train_batch_size: 1024
• optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 10.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Micro F1	Micro Precision	Micro Recall	Macro F1	Macro Precision	Macro Recall
1.0699	1.0	6	1.1797	58.0175	58.0175	58.0175	43.6555	51.3407	43.7120
0.9746	2.0	12	1.1797	58.0175	58.0175	58.0175	43.6555	51.3407	43.7120
0.9521	3.0	18	1.1797	58.0175	58.0175	58.0175	43.6555	51.3407	43.7120
0.9427	4.0	24	1.2275	55.6851	55.6851	55.6851	41.4678	47.4149	43.1301
0.8847	5.0	30	1.1758	57.7259	57.7259	57.7259	44.9068	55.8831	45.2846
0.9161	6.0	36	1.1211	59.4752	59.4752	59.4752	49.5625	53.3040	49.2513
0.7421	7.0	42	1.1455	59.1837	59.1837	59.1837	50.3827	56.6258	49.2719
0.7157	8.0	48	1.1465	59.0379	59.0379	59.0379	52.6727	53.9745	53.3124
0.7021	8.3636	50	1.1406	60.2041	60.2041	60.2041	54.3501	56.1563	54.1594

Framework versions

• Transformers 4.50.0
• Pytorch 2.4.0+cu121
• Datasets 2.15.0
• Tokenizers 0.21.1