wav2vec2-xls-r-1b-E4-faroese-100h-30-epochs_20250208_v4

This model is a fine-tuned version of davidilag/wav2vec2-xls-r-1b-scandinavian-E4-100h-30-epochs-20250208_v3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1043
• Wer: 18.7161
• Cer: 4.0366

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.7469	0.4877	1000	0.4760	52.5224	15.1744
0.4523	0.9754	2000	0.2306	32.5858	8.5932
0.3754	1.4628	3000	0.2148	30.9865	7.8720
0.3589	1.9505	4000	0.1997	30.3961	7.7332
0.3001	2.4379	5000	0.2112	29.9203	7.7316
0.3195	2.9256	6000	0.1985	29.2814	7.5272
0.271	3.4131	7000	0.1791	27.8847	6.9938
0.2524	3.9008	8000	0.1839	27.8098	6.9355
0.2384	4.3882	9000	0.1701	27.0388	6.7532
0.2615	4.8759	10000	0.1603	26.6863	6.6262
0.2282	5.3633	11000	0.1659	26.2458	6.5189
0.2191	5.8510	12000	0.1575	26.2766	6.4636
0.1803	6.3385	13000	0.1524	26.0519	6.3476
0.1836	6.8261	14000	0.1498	25.1399	6.1638
0.1755	7.3136	15000	0.1499	24.8843	6.0178
0.1607	7.8013	16000	0.1546	24.6552	6.0155
0.146	8.2887	17000	0.1450	24.3865	5.8727
0.1478	8.7764	18000	0.1426	24.0428	5.7795
0.1173	9.2638	19000	0.1327	23.7520	5.6793
0.1326	9.7515	20000	0.1368	23.8269	5.6573
0.121	10.2390	21000	0.1339	23.5714	5.5421
0.115	10.7267	22000	0.1314	23.2101	5.5184
0.1169	11.2141	23000	0.1260	22.8312	5.3945
0.1115	11.7018	24000	0.1255	22.6814	5.3243
0.1017	12.1892	25000	0.1294	22.5625	5.3109
0.0993	12.6769	26000	0.1250	21.9456	5.1531
0.0918	13.1644	27000	0.1292	22.1615	5.1775
0.0943	13.6520	28000	0.1292	21.8135	5.0955
0.0719	14.1395	29000	0.1278	21.6769	4.9834
0.0792	14.6272	30000	0.1157	21.6328	4.9219
0.0761	15.1146	31000	0.1158	21.5403	4.9172
0.0762	15.6023	32000	0.1105	21.0512	4.7380
0.0641	16.0897	33000	0.1164	20.8750	4.7278
0.0628	16.5774	34000	0.1125	21.0689	4.7467
0.0612	17.0649	35000	0.1089	20.8221	4.6923
0.0671	17.5525	36000	0.1108	20.3463	4.5968
0.0499	18.0400	37000	0.1130	20.5930	4.6536
0.0475	18.5277	38000	0.1090	20.1613	4.5313
0.0457	19.0151	39000	0.1116	20.0511	4.5037
0.0491	19.5028	40000	0.1041	19.8793	4.4264
0.0421	19.9905	41000	0.1053	19.8925	4.3909
0.0521	20.4779	42000	0.1104	19.5841	4.3443
0.0413	20.9656	43000	0.1120	19.6370	4.3617
0.0388	21.4531	44000	0.1088	19.5841	4.3538
0.0451	21.9407	45000	0.1058	19.3506	4.2607
0.0301	22.4282	46000	0.1091	19.1611	4.2189
0.0333	22.9159	47000	0.1102	19.2713	4.2291
0.0343	23.4033	48000	0.1074	19.2404	4.2165
0.0347	23.8910	49000	0.1075	19.1391	4.1826
0.0343	24.3784	50000	0.1066	19.0422	4.1494
0.0267	24.8661	51000	0.1044	19.0862	4.1518
0.0315	25.3536	52000	0.1066	18.9012	4.0966
0.0303	25.8413	53000	0.1034	18.9232	4.1013
0.0297	26.3287	54000	0.1075	18.9188	4.0926
0.0226	26.8164	55000	0.1044	18.9100	4.0879
0.0225	27.3038	56000	0.1064	18.8263	4.0674
0.0274	27.7915	57000	0.1050	18.7514	4.0595
0.0259	28.2790	58000	0.1053	18.6985	4.0453
0.0286	28.7666	59000	0.1047	18.6985	4.0358
0.0336	29.2541	60000	0.1043	18.6809	4.0311
0.0376	29.7418	61000	0.1043	18.7161	4.0366

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0