mms-1b-all-bemgen-combined-m50f50-62-DAT-0.4

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2859
• Cer: 0.0788

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 62
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
8.2767	0.8439	100	2.8936	0.9993
2.4301	1.6835	200	0.4919	0.1363
1.4618	2.5232	300	0.3398	0.0954
1.346	3.3629	400	0.3112	0.0891
1.2349	4.2025	500	0.2968	0.0839
1.1879	5.0422	600	0.2881	0.0823
1.2288	5.8861	700	0.2875	0.0803
1.189	6.7257	800	0.2859	0.0788
1.0875	7.5654	900	0.2796	0.0779
1.1294	8.4051	1000	0.2768	0.0754
1.1259	9.2447	1100	0.2746	0.0776
1.1996	10.0844	1200	0.2766	0.0782
1.0921	10.9283	1300	0.2766	0.0750
1.1635	11.7679	1400	0.2720	0.0748
1.1144	12.6076	1500	0.2704	0.0758
1.1534	13.4473	1600	0.2731	0.0767
1.1759	14.2869	1700	0.2692	0.0757
1.0518	15.1266	1800	0.2671	0.0745
1.0768	15.9705	1900	0.2648	0.0741
1.0759	16.8101	2000	0.2661	0.0748
1.1551	17.6498	2100	0.2661	0.0745
1.0846	18.4895	2200	0.2646	0.0743
1.0965	19.3291	2300	0.2647	0.0736
0.9806	20.1688	2400	0.2646	0.0737
1.0592	21.0084	2500	0.2651	0.0739

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0