mms-1b-all-swagen-female-5hrs-62

This model is a fine-tuned version of facebook/mms-1b-all on the SWAGEN - SWA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2021
• Wer: 0.1846

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 62
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.6415	0.4706	100	0.3005	0.1938
0.2987	0.9412	200	0.2424	0.1917
0.2612	1.4094	300	0.2313	0.1907
0.249	1.88	400	0.2168	0.1850
0.2322	2.3482	500	0.2133	0.1865
0.2279	2.8188	600	0.2088	0.1859
0.2276	3.2871	700	0.2101	0.1913
0.2352	3.7576	800	0.2049	0.1871
0.2228	4.2259	900	0.2063	0.1867
0.225	4.6965	1000	0.2034	0.1823
0.2092	5.1647	1100	0.2050	0.1863
0.2237	5.6353	1200	0.2021	0.1844
0.2097	6.1035	1300	0.2037	0.1850
0.217	6.5741	1400	0.2009	0.1848
0.2204	7.0424	1500	0.2066	0.1834
0.2073	7.5129	1600	0.2050	0.1905
0.2195	7.9835	1700	0.2030	0.1842

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0