mms-1b-all-bemgen-combined-m50f50-42-DAT-0.5

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2685
• Cer: 0.0768

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
8.7038	0.8439	100	2.9604	1.0000
2.896	1.6835	200	1.1561	0.3748
1.6872	2.5232	300	0.3652	0.1080
1.3958	3.3629	400	0.3356	0.0984
1.3331	4.2025	500	0.3044	0.0873
1.2736	5.0422	600	0.2927	0.0829
1.2319	5.8861	700	0.2865	0.0818
1.2493	6.7257	800	0.2809	0.0798
1.162	7.5654	900	0.2779	0.0789
1.2392	8.4051	1000	0.2759	0.0786
1.2305	9.2447	1100	0.2718	0.0760
1.1666	10.0844	1200	0.2686	0.0768
1.1685	10.9283	1300	0.2668	0.0738
1.158	11.7679	1400	0.2667	0.0742
1.1767	12.6076	1500	0.2628	0.0736
1.1855	13.4473	1600	0.2646	0.0749
1.1381	14.2869	1700	0.2662	0.0740
1.14	15.1266	1800	0.2656	0.0749

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0