wav2vec2-large-mms-1b-igbo

This model is a fine-tuned version of facebook/mms-1b-all on the fleurs dataset. It achieves the following results on the evaluation set:

• Loss: 0.4649
• Wer: 0.4449

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.464	0.0731	1000	0.7265	0.5768
0.4324	0.1463	2000	0.7455	0.6102
0.4307	0.2194	3000	1.1129	0.6445
0.3982	0.2925	4000	0.7999	0.5870
0.3915	0.3657	5000	0.7252	0.5210
0.3834	0.4388	6000	0.7565	0.5677
0.376	0.5120	7000	0.7596	0.6294
0.388	0.5851	8000	0.6784	0.5679
0.3687	0.6582	9000	0.7597	0.5916
0.374	0.7314	10000	0.6482	0.5023
0.3576	0.8045	11000	0.6486	0.5572
0.3621	0.8776	12000	0.5482	0.4869
0.363	0.9508	13000	0.6543	0.5082
0.3549	1.0239	14000	0.5477	0.4849
0.342	1.0971	15000	0.5505	0.5079
0.3296	1.1702	16000	0.5701	0.5211
0.3363	1.2433	17000	0.5565	0.5281
0.3265	1.3165	18000	0.6660	0.5794
0.327	1.3896	19000	0.5414	0.4854
0.3319	1.4627	20000	0.5677	0.5181
0.3273	1.5359	21000	0.5482	0.4901
0.3209	1.6090	22000	0.5475	0.5019
0.3153	1.6821	23000	0.5278	0.4723
0.3214	1.7553	24000	0.5232	0.4809
0.3227	1.8284	25000	0.5419	0.4950
0.306	1.9016	26000	0.5120	0.4653
0.2956	1.9747	27000	0.5043	0.4790
0.2875	2.0478	28000	0.5111	0.4592
0.3158	2.1210	29000	0.4959	0.4582
0.2906	2.1941	30000	0.4857	0.4577
0.2985	2.2672	31000	0.4897	0.4625
0.2877	2.3404	32000	0.4869	0.4667
0.2832	2.4135	33000	0.4877	0.4541
0.2815	2.4867	34000	0.4869	0.4598
0.28	2.5598	35000	0.4935	0.4624
0.2904	2.6329	36000	0.4859	0.4540
0.2767	2.7061	37000	0.4879	0.4550
0.2801	2.7792	38000	0.4855	0.4536
0.2711	2.8523	39000	0.5059	0.4674
0.2652	2.9255	40000	0.4715	0.4512
0.276	2.9986	41000	0.4804	0.4568
0.2556	3.0717	42000	0.4869	0.4572
0.275	3.1449	43000	0.4761	0.4536
0.2615	3.2180	44000	0.4848	0.4679
0.264	3.2912	45000	0.4722	0.4518
0.2554	3.3643	46000	0.4747	0.4551
0.2632	3.4374	47000	0.4695	0.4507
0.2565	3.5106	48000	0.4761	0.4506
0.2555	3.5837	49000	0.4802	0.4619
0.2397	3.6568	50000	0.4687	0.4497
0.2599	3.7300	51000	0.4684	0.4506
0.2451	3.8031	52000	0.4678	0.4504
0.2623	3.8763	53000	0.4642	0.4461
0.2475	3.9494	54000	0.4649	0.4449

Framework versions

• Transformers 4.54.0.dev0
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0