miosipof/asr2_aug_v2

Whisper Medium IT

This model is a fine-tuned version of openai/whisper-medium on the b-brave-balanced-augmented dataset. It achieves the following results on the evaluation set:

• Loss: 0.5097
• Wer: 37.4718
• Cer: 25.0621
• Lr: 0.0000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.3
• num_epochs: 12
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer	Lr
0.9163	1.0	413	0.8402	59.8194	36.7854	0.0001
0.5141	2.0	826	0.5758	116.7043	100.3728	0.0002
0.4624	3.0	1239	0.5542	181.9413	220.1740	0.0002
0.2622	4.0	1652	0.5540	36.7946	24.9379	0.0003
0.1687	5.0	2065	0.5141	93.4537	81.3173	0.0002
0.0797	6.0	2478	0.5233	34.9887	24.7307	0.0002
0.0367	7.0	2891	0.5124	36.3431	25.2693	0.0002
0.0264	8.0	3304	0.5188	35.2144	23.4880	0.0001
0.0108	9.0	3717	0.4938	33.6343	23.1152	0.0001
0.0084	10.0	4130	0.5044	37.9233	25.1450	0.0001
0.0014	11.0	4543	0.5071	37.2460	25.0621	0.0000
0.0017	11.9721	4944	0.5097	37.4718	25.0621	0.0000

Framework versions

• PEFT 0.14.0
• Transformers 4.49.0
• Pytorch 2.2.0
• Datasets 3.3.2
• Tokenizers 0.21.1