---
language:
- ms
- en
datasets:
- mesolitica/Malaysian-Reasoning
base_model:
- mesolitica/Malaysian-Qwen2.5-7B-Instruct
---

# Malaysian Qwen 2.5 7B Instruct Reasoning SFT

Continue finetuning https://huggingface.co/mesolitica/Malaysian-Qwen2.5-7B-Instruct on highly curated Malaysian Reasoning dataset.

## Improvement

1. Reasoning on Math, Science, Translation, Dialects, Multiple choices, coding and Maktabah Al Bakri.
2. Warmup reasoning.

## Training session

Finetune on [mesolitica/Malaysian-Reasoning](https://huggingface.co/datasets/mesolitica/Malaysian-Reasoning) to make the model better reasoning on Malaysian context.
  
## How we train

1. Full parameters on 12k context length.
5. WanDB at https://wandb.ai/huseinzol05/fpf-qwen2.5-7b-malaysian-12k-reasoning

Source code at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5

## Benchmark

### Dialect Translation

All the benchmarks generate using vLLM, evaluation based on sacrebleu CHRF max@5.

Source code for evaluation at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5/evaluate-dialect

Dialect to standard Malay,

```
From: johor To: malay, score: 53.80150476295704
From: kedah To: malay, score: 59.656783756383994
From: pahang To: malay, score: 54.613757432364295
From: negeri sembilan To: malay, score: 51.792391481041555
From: kelantan To: malay, score: 51.31613831154433
From: penang To: malay, score: 55.92313437424021
From: melaka To: malay, score: 49.89166863202441
average: 53.85648267865083
```

Standard Malay to dialect,

```
From: malay To: johor, score: 52.16692422736562
From: malay To: kedah, score: 53.25312749019
From: malay To: pahang, score: 53.94390758910295
From: malay To: negeri sembilan, score: 36.27480628354344
From: malay To: kelantan, score: 37.559315524669195
From: malay To: penang, score: 53.41504444742441
From: malay To: melaka, score: 65.06211074823777
average: 50.239319472933346
```

### MalayMMLU

Source code for evaluation at https://github.com/mesolitica/malaya/tree/master/session/qwen2.5/evaluate-malaymmlu

Evaluation based on Accuracy@1,

```

```

Evaluation based on Accuracy@5,

```

```

## Special thanks

Special thanks to https://www.sns.com.my and Nvidia for 8x H100 node!