---
library_name: transformers
license: mit
metrics:
- name: wer
  type: wer
  value: 17.26
base_model:
- mesolitica/wav2vec2-xls-r-300m-mixed
pipeline_tag: automatic-speech-recognition
tags:
- wav2vec2
- asr
- automatic-speech-recognition
- malay
- english
- speech
---

# Model Card for Malay-English Fine-Tuned ASR Model

This model was fine-tuned on approximately 50 hours of manually curated Malay-English code-switched audio data for 10 epochs. It achieves a Word Error Rate (WER) of 17.26% on a held-out evaluation set vs 34.29% with base model.

## Model Details

### Model Description

This is a fine-tuned version of the `mesolitica/wav2vec2-xls-r-300m-mixed` model on a custom Malay-English dataset. It is designed to transcribe speech that includes both Malay and English, especially in informal or conversational contexts where code-switching is common.

- **Developed by:** mysterio
- **Model type:** CTC-based automatic speech recognition
- **Languages:** Malay, English
- **License:** MIT
- **Fine-tuned from:** https://huggingface.co/mesolitica/wav2vec2-xls-r-300m-mixed

### Model Sources

- **Base Model:** [mesolitica/wav2vec2-xls-r-300m-mixed](https://huggingface.co/mesolitica/wav2vec2-xls-r-300m-mixed)

## Uses

### Direct Use

This model can be used to transcribe conversational Malay-English audio recordings, especially in domains such as:
- Broadcast interviews
- YouTube vlogs
- Podcasts
- Community recordings

### Downstream Use

The model can be fine-tuned further or used as part of downstream applications such as:
- Real-time transcription services
- Voice assistants tailored for Malaysian users
- Speech-driven translation systems

### Out-of-Scope Use

- High-stakes transcription scenarios (e.g., legal or medical contexts) where exact word accuracy is critical
- Non-Malay, non-English languages
- Noisy or far-field audio environments (unless fine-tuned further)

## Bias, Risks, and Limitations

### Known Limitations

- May underperform on accents or dialects not well-represented in training data
- Inconsistent casing or punctuation handling (model is CTC-based)
- Limited robustness to background noise or overlapping speakers

### Recommendations

- Always verify outputs for critical tasks
- Pair with punctuation restoration or diarization for production-grade use
- Retrain with domain-specific data for higher accuracy

## How to Get Started with the Model

```python
from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="langminer/wav2vec2-custom-asr")
transcription = asr("your_audio_file.wav")
print(transcription)