mms-1b-all-swagen-female-5hrs-42

This model is a fine-tuned version of facebook/mms-1b-all on the SWAGEN - SWA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2827
• Wer: 0.2164

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
7.1563	0.4706	100	3.1660	1.0112
2.7351	0.9412	200	2.3595	0.9961
2.0633	1.4094	300	1.5806	0.8581
1.2475	1.88	400	0.7942	0.4642
0.7625	2.3482	500	0.5963	0.3666
0.623	2.8188	600	0.4957	0.3348
0.548	3.2871	700	0.4650	0.3213
0.5001	3.7576	800	0.4111	0.3003
0.4372	4.2259	900	0.3933	0.2970
0.43	4.6965	1000	0.3585	0.2756
0.41	5.1647	1100	0.3658	0.2687
0.3953	5.6353	1200	0.3489	0.2542
0.3768	6.1035	1300	0.3350	0.2449
0.3647	6.5741	1400	0.3235	0.2482
0.3617	7.0424	1500	0.3187	0.2461
0.3398	7.5129	1600	0.3177	0.2345
0.3306	7.9835	1700	0.3122	0.2355
0.3346	8.4518	1800	0.3031	0.2322
0.3221	8.9224	1900	0.3063	0.2278
0.2996	9.3906	2000	0.3066	0.2291
0.3156	9.8612	2100	0.3014	0.2247
0.306	10.3294	2200	0.2993	0.2214
0.3098	10.8	2300	0.2966	0.2226
0.3143	11.2682	2400	0.2961	0.2206
0.2902	11.7388	2500	0.2901	0.2181
0.278	12.2071	2600	0.2904	0.2222
0.2824	12.6776	2700	0.2890	0.2168
0.2775	13.1459	2800	0.2883	0.2195
0.2765	13.6165	2900	0.2872	0.2162
0.276	14.0847	3000	0.2849	0.2160
0.2713	14.5553	3100	0.2835	0.2203
0.2736	15.0235	3200	0.2827	0.2164
0.2533	15.4941	3300	0.2796	0.2177
0.2693	15.9647	3400	0.2822	0.2162
0.2533	16.4329	3500	0.2826	0.2150
0.2618	16.9035	3600	0.2808	0.2135

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0