mms-1b-all-bemgen-combined-m50f50-62-DAT-0.7

This model is a fine-tuned version of facebook/mms-1b-all on the BEMGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2684
• Cer: 0.0753

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 62
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
8.4148	0.8439	100	2.9018	0.9994
2.5347	1.6835	200	0.5233	0.1540
1.5325	2.5232	300	0.3628	0.1027
1.4104	3.3629	400	0.3293	0.0970
1.3083	4.2025	500	0.3197	0.0919
1.2851	5.0422	600	0.3059	0.0884
1.3503	5.8861	700	0.3020	0.0865
1.3769	6.7257	800	0.3012	0.0825
1.2606	7.5654	900	0.2831	0.0798
1.2803	8.4051	1000	0.2801	0.0770
1.2855	9.2447	1100	0.2793	0.0782
1.3776	10.0844	1200	0.2801	0.0786
1.2692	10.9283	1300	0.2765	0.0764
1.3227	11.7679	1400	0.2740	0.0746
1.2628	12.6076	1500	0.2730	0.0757
1.3158	13.4473	1600	0.2757	0.0760
1.3531	14.2869	1700	0.2706	0.0754
1.2122	15.1266	1800	0.2701	0.0751
1.2249	15.9705	1900	0.2691	0.0750
1.2273	16.8101	2000	0.2713	0.0757
1.304	17.6498	2100	0.2701	0.0761
1.249	18.4895	2200	0.2687	0.0753
1.2562	19.3291	2300	0.2687	0.0752
1.1351	20.1688	2400	0.2684	0.0753
1.2185	21.0084	2500	0.2698	0.0749
1.1919	21.8523	2600	0.2693	0.0745
1.1667	22.6920	2700	0.2686	0.0752

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0