whisper-large-v3-turbo-urdu

This model is a fine-tuned version of openai/whisper-large-v3-turbo on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4630
• Wer: 0.3826

Usage

from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
from datasets import load_dataset
import torch, warnings, os

device      = "cuda:0"
torch_dtype = torch.float16
model_id    = "kingabzpro/whisper-large-v3-turbo-urdu"

model      = AutoModelForSpeechSeq2Seq.from_pretrained(
                model_id, torch_dtype=torch_dtype, use_safetensors=True
            ).to(device)
model.generation_config.forced_decoder_ids = None
processor  = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch_dtype,
    device=device,
)

ds     = load_dataset("mozilla-foundation/common_voice_17_0", "ur", split="test")
audio  = ds[100]["audio"]

result = pipe(audio)
print("Original   :", ds[100]['sentence'])
print("Predicted  :", result["text"])

Original   : اگر عمران خان ٹھیک کر رہے ہیں۔
Predicted  : اگر یمران خان ٹھیک کر رہے ہیں

It requires 1.8GiB VRAM.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• training_steps: 2000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.5561	0.1863	500	1.0707	0.6629
0.6472	0.3726	1000	0.7007	0.508
0.5162	0.5589	1500	0.5460	0.4263
0.3663	0.7452	2000	0.4630	0.3826

Framework versions

• Transformers 4.53.0
• Pytorch 2.8.0.dev20250319+cu128
• Datasets 3.6.0
• Tokenizers 0.21.2