mms-1b-all-bemgen-combined-m50f50-52-DAT-0.5

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2687
• Cer: 0.0762

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 52
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
7.5249	0.8439	100	2.8759	0.9925
2.5604	1.6835	200	0.5393	0.1710
1.5275	2.5232	300	0.3629	0.1072
1.3518	3.3629	400	0.3246	0.0949
1.218	4.2025	500	0.3083	0.0895
1.2951	5.0422	600	0.2886	0.0838
1.2115	5.8861	700	0.2859	0.0827
1.2352	6.7257	800	0.2849	0.0814
1.2512	7.5654	900	0.2735	0.0782
1.1716	8.4051	1000	0.2759	0.0775
1.2454	9.2447	1100	0.2692	0.0750
1.1644	10.0844	1200	0.2687	0.0762
1.1234	10.9283	1300	0.2658	0.0745
1.1794	11.7679	1400	0.2677	0.0753
1.1293	12.6076	1500	0.2669	0.0746
1.1625	13.4473	1600	0.2686	0.0736

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0