deepdml/wav2vec2-large-mms-1b-igbo-mix

wav2vec2-large-mms-1b-igbo

This model is a fine-tuned version of facebook/mms-1b-all on the fleurs dataset. It achieves the following results on the evaluation set:

• Loss: 0.4487
• Wer: 0.4385

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.5485	0.0662	1000	0.7264	0.5983
0.5147	0.1324	2000	0.6157	0.5242
0.4708	0.1986	3000	0.6952	0.5986
0.4622	0.2648	4000	0.5811	0.4961
0.4792	0.3310	5000	0.6102	0.5537
0.443	0.3972	6000	0.5850	0.5035
0.4471	0.4635	7000	0.5633	0.5028
0.4528	0.5297	8000	0.5434	0.4816
0.4588	0.5959	9000	0.5881	0.5055
0.4215	0.6621	10000	0.5508	0.5089
0.4328	0.7283	11000	0.6404	0.5783
0.4341	0.7945	12000	0.5919	0.5443
0.4009	0.8607	13000	0.5558	0.5276
0.3945	0.9269	14000	0.5928	0.5400
0.4231	0.9931	15000	0.5397	0.4766
0.4001	1.0593	16000	0.5406	0.5065
0.3996	1.1255	17000	0.5299	0.4816
0.4275	1.1917	18000	0.5446	0.4962
0.3637	1.2579	19000	0.5525	0.4804
0.4127	1.3242	20000	0.5115	0.4789
0.3877	1.3904	21000	0.5398	0.4735
0.3756	1.4566	22000	0.4917	0.4624
0.3913	1.5228	23000	0.5166	0.4737
0.3838	1.5890	24000	0.5150	0.5000
0.3691	1.6552	25000	0.4945	0.4643
0.3905	1.7214	26000	0.5175	0.5236
0.3578	1.7876	27000	0.4977	0.4734
0.3598	1.8538	28000	0.5344	0.4816
0.3398	1.9200	29000	0.4834	0.4665
0.3785	1.9862	30000	0.4863	0.4604
0.3444	2.0524	31000	0.4886	0.4624
0.386	2.1186	32000	0.4933	0.4582
0.3398	2.1849	33000	0.4845	0.4588
0.3559	2.2511	34000	0.4964	0.4951
0.3615	2.3173	35000	0.4954	0.4740
0.346	2.3835	36000	0.4782	0.4622
0.3293	2.4497	37000	0.4775	0.4697
0.3329	2.5159	38000	0.4774	0.4597
0.3486	2.5821	39000	0.4916	0.4860
0.3356	2.6483	40000	0.4810	0.4551
0.3305	2.7145	41000	0.4788	0.4668
0.3226	2.7807	42000	0.4687	0.4465
0.3199	2.8469	43000	0.4748	0.4592
0.3222	2.9131	44000	0.4618	0.4441
0.3328	2.9793	45000	0.4627	0.4548
0.3326	3.0456	46000	0.4624	0.4456
0.2993	3.1118	47000	0.4640	0.4430
0.3116	3.1780	48000	0.4675	0.4550
0.3447	3.2442	49000	0.4586	0.4426
0.3079	3.3104	50000	0.4586	0.4433
0.3155	3.3766	51000	0.4577	0.4409
0.3021	3.4428	52000	0.4617	0.4453
0.3147	3.5090	53000	0.4560	0.4418
0.3368	3.5752	54000	0.4555	0.4424
0.2948	3.6414	55000	0.4536	0.4412
0.3169	3.7076	56000	0.4520	0.4402
0.3176	3.7738	57000	0.4533	0.4444
0.3164	3.8400	58000	0.4505	0.4382
0.3009	3.9062	59000	0.4497	0.4373
0.3114	3.9725	60000	0.4487	0.4385

Framework versions

• Transformers 4.54.0.dev0
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0