wav2vec2-xls-r-1b-E4-faroese-100h-30-epochs_20250208_v2

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1025
• Wer: 18.6544
• Cer: 3.9782

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.0062	0.4877	1000	0.7503	73.8424	22.8910
0.5172	0.9754	2000	0.2760	37.2164	9.8233
0.3955	1.4628	3000	0.2290	32.4580	8.5024
0.3725	1.9505	4000	0.2194	31.7972	8.2350
0.3008	2.4379	5000	0.2091	30.6340	7.8097
0.3032	2.9256	6000	0.1989	29.9599	7.6787
0.2796	3.4131	7000	0.1679	27.7614	6.8013
0.2535	3.9008	8000	0.1830	27.7570	6.8842
0.2304	4.3882	9000	0.1762	26.8273	6.8029
0.2275	4.8759	10000	0.1585	25.9153	6.3398
0.2017	5.3633	11000	0.1543	25.6554	6.3042
0.2087	5.8510	12000	0.1536	25.4968	6.2214
0.1782	6.3385	13000	0.1549	25.4042	6.1796
0.1721	6.8261	14000	0.1481	25.2236	6.1551
0.1755	7.3136	15000	0.1509	25.2721	6.1883
0.1436	7.8013	16000	0.1495	24.2323	5.8687
0.1362	8.2887	17000	0.1483	24.2851	5.8016
0.1355	8.7764	18000	0.1406	23.7476	5.6328
0.1082	9.2638	19000	0.1375	23.7168	5.6210
0.1211	9.7515	20000	0.1370	23.3952	5.5894
0.1149	10.2390	21000	0.1367	23.2982	5.4931
0.1026	10.7267	22000	0.1322	22.9458	5.3929
0.1182	11.2141	23000	0.1304	22.9458	5.3693
0.1008	11.7018	24000	0.1296	22.4435	5.2241
0.0989	12.1892	25000	0.1281	22.5536	5.2233
0.0891	12.6769	26000	0.1312	22.4347	5.2241
0.0816	13.1644	27000	0.1231	21.9148	5.0189
0.0851	13.6520	28000	0.1191	21.3949	4.8588
0.0736	14.1395	29000	0.1247	21.8707	4.9834
0.0745	14.6272	30000	0.1114	21.3420	4.8075
0.0702	15.1146	31000	0.1172	21.4786	4.8375
0.0654	15.6023	32000	0.1145	21.1041	4.8130
0.065	16.0897	33000	0.1133	20.7296	4.6875
0.0578	16.5774	34000	0.1181	20.7472	4.6662
0.0565	17.0649	35000	0.1156	20.6591	4.6473
0.0609	17.5525	36000	0.1078	20.4256	4.5558
0.051	18.0400	37000	0.1099	20.2362	4.5116
0.0487	18.5277	38000	0.1102	20.0731	4.4856
0.0438	19.0151	39000	0.1109	20.1657	4.4903
0.0464	19.5028	40000	0.1102	19.9277	4.4469
0.0418	19.9905	41000	0.1076	20.0731	4.4224
0.0451	20.4779	42000	0.1056	19.5753	4.3120
0.0409	20.9656	43000	0.1045	19.4960	4.2773
0.0352	21.4531	44000	0.1082	19.3902	4.2591
0.0416	21.9407	45000	0.1043	19.2008	4.1944
0.0318	22.4282	46000	0.1059	19.4255	4.2370
0.0371	22.9159	47000	0.1055	19.1523	4.1794
0.0351	23.4033	48000	0.1032	19.0466	4.1408
0.0347	23.8910	49000	0.1000	18.9673	4.1163
0.0357	24.3784	50000	0.1046	19.0378	4.1439
0.0278	24.8661	51000	0.1016	18.9012	4.0776
0.0326	25.3536	52000	0.1093	18.7514	4.0697
0.0294	25.8413	53000	0.1030	18.8527	4.0516
0.0294	26.3287	54000	0.1026	18.7073	4.0114
0.023	26.8164	55000	0.1029	18.6897	4.0137
0.0236	27.3038	56000	0.1041	18.7249	4.0058
0.03	27.7915	57000	0.1029	18.6544	3.9979
0.0246	28.2790	58000	0.1026	18.6280	3.9822
0.0277	28.7666	59000	0.1027	18.6500	3.9806
0.0355	29.2541	60000	0.1026	18.6412	3.9782
0.0325	29.7418	61000	0.1025	18.6544	3.9782

Framework versions

• Transformers 4.48.2
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0