mms-1b-all-bemgen-combined-m50f50-62-DAT-0.2

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2971
• Cer: 0.0833

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 62
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
8.1838	0.8439	100	2.8906	0.9992
2.373	1.6835	200	0.4915	0.1362
1.4267	2.5232	300	0.3532	0.1006
1.3195	3.3629	400	0.3235	0.0936
1.2038	4.2025	500	0.3179	0.0924
1.1582	5.0422	600	0.3006	0.0871
1.1903	5.8861	700	0.2988	0.0861
1.151	6.7257	800	0.2971	0.0833
1.039	7.5654	900	0.2860	0.0811
1.0605	8.4051	1000	0.2784	0.0763
1.0572	9.2447	1100	0.2755	0.0780
1.1154	10.0844	1200	0.2759	0.0789
1.0075	10.9283	1300	0.2765	0.0764
1.0782	11.7679	1400	0.2718	0.0753
1.0123	12.6076	1500	0.2688	0.0751
1.0461	13.4473	1600	0.2715	0.0756
1.068	14.2869	1700	0.2667	0.0748
0.9508	15.1266	1800	0.2650	0.0736
0.9704	15.9705	1900	0.2643	0.0732
0.9858	16.8101	2000	0.2664	0.0753
1.0525	17.6498	2100	0.2661	0.0749
0.9867	18.4895	2200	0.2635	0.0745
0.9996	19.3291	2300	0.2647	0.0741
0.89	20.1688	2400	0.2641	0.0745
0.9625	21.0084	2500	0.2650	0.0740

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0