ChindaMT-4B

ChindaMT-4B is an open-weight Thai-English machine translation model fine-tuned from Qwen/Qwen3.5-4B on Grounded, a 1.97M-record dataset built by Reference-Grounded Data Curation (RGDC). It translates in both directions and follows auxiliary rules given in the prompt, such as terminology, register, length, and output format. It is one of three sizes in the ChindaMT family (4B, 2B, 0.8B).

Task: Thai-English machine translation with instruction following
Base model: Qwen3.5-4B
Parameters: 4B
License: Apache-2.0 (inherits the base-model license)

Prompting

Plain translation. Same template for both directions; swap the language line and the source tag:

Translate English to Thai.

EN: The weather is nice today.

Translate Thai to English.

TH: วันนี้อากาศดีมาก

With rules. Add a Rules: block between the language line and the source line. Rules are free-form text:

Translate English to Thai.
Rules:
- Return only the translated text
- Use a clear, professional tone in Thai
- Keep all numerals in Arabic digits

EN: <source text>

Inference

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "iapp/ChindaMT-4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

prompt = "Translate English to Thai.\n\nEN: The weather is nice today."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False, enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(
    **inputs, max_new_tokens=1024, temperature=0.01, top_p=0.7, top_k=20,
    repetition_penalty=1.05, do_sample=True,
)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Thinking mode is disabled (enable_thinking=False); all reported results use this setting.

Training

Method: full-parameter supervised fine-tuning with LLaMA-Factory, DeepSpeed ZeRO-2
Data: Grounded, 1,973,529 records (the released file omits 171 of them: 133 WMT24-derived rows and 38 rows overlapping the evaluation suites)
Epochs: 1
Learning rate: 2e-5, inverse-square-root schedule, 1% warmup
Optimizer: AdamW (0.9, 0.999), weight decay 0.01
Effective batch size: 64, on two H100 GPUs
Cutoff length: 1024 tokens
Seed: 42
The same recipe trains all three sizes.

Decoding settings

All reported results, and the inference snippet above, use these settings:

temperature 0.01, top-p 0.7, top-k 20, repetition penalty 1.05
max new tokens 1024
thinking mode off (enable_thinking=False)
input length: trained with a 1024-token cutoff. Longer inputs worked in our tests but are used at your own risk; recommended: translate long documents paragraph by paragraph, and raise max_new_tokens if an output is cut short.

Evaluation

How to read the tables

Metric: length-controlled pairwise win rate (LC%), AlpacaEval-v2 protocol, 400 items per split. Each cell is the LC% of ChindaMT-4B against the model named in that row, so values above 50 mean ChindaMT-4B wins and 50 is a tie.
Judge: Qwen3.6-35B, with two cross-judges from other model families as a check (below).
Plain (shared): both systems get the same prompt scaffold, translation only.
Plain (own template): the baseline uses its own recommended prompt, the hardest comparison for ChindaMT-4B.
Constrained: the prompt adds a rules block of one to four constraints; wins here reflect rule-following as well as translation quality.
Dashes: comparisons not run because the comparator produced majority non-target-language output under the shared prompt.

Against its own base (Qwen3.5-4B): Plain 55.7, Constrained 62.2 on CoreEval.

CoreEval (five deployment domains):

ChindaMT-4B vs	Plain (shared)	Plain (own template)	Constrained
MiLMMT-46-4B	86.3	53.3	89.5
TranslateGemma-4B	81.0	66.7	87.2
Typhoon-Translate-1.5-4B	61.8	57.8	68.4
HY-MT-1.5-7B	74.0	62.9	97.9
GemmaX2-28-9B	93.6	56.6	94.1

BroadEval (ten broader domains, cross-domain generalization):

ChindaMT-4B vs	Plain (shared)	Plain (own template)	Constrained
MiLMMT-46-4B	-	56.2	-
TranslateGemma-4B	81.8	67.0	76.1
Typhoon-Translate-1.5-4B	58.7	53.1	63.2
HY-MT-1.5-7B	70.1	62.0	98.2
GemmaX2-28-9B	-	67.4	-

What the numbers mean

Each number is a win rate out of 100: how often the judge preferred the translation from ChindaMT-4B over the other model's, on the same 400 sentences, after correcting for output length. 50 is a tie. For example, 61.8 against Typhoon-Translate-1.5-4B means the judge preferred ChindaMT-4B in about 62 of 100 head-to-head comparisons.
Under Constrained, a win means the translation was judged better and respected the rules, so those numbers measure rule-following as well as translation quality.
In short: ChindaMT-4B is preferred over its base and over every same-size baseline on both splits, with the widest margins under rules.

External metrics (direction-averaged; quality mean averages CometKiwi, GEMBA-DA, and GEMBA-MQM on a 0-100 scale, higher is better; MetricX-24 is an error score, lower is better):

Benchmark	Quality mean	MetricX-24 error
FLORES-200 (Wikipedia)	90.7	2.06
WMT24++ en-th (news)	87.8	3.20

Human evaluation. Three native Thai raters preferred ChindaMT-4B over Typhoon-Translate-1.5-4B on 64% of Plain items (Plain with the baseline template) and 69% of Constrained items, inter-rater kappa 0.73.

Cross-judges. GPT-OSS-120B and Llama-3.3-70B-Instruct agree with the primary judge in direction on every Constrained and Plain (shared) comparison. Full protocol, standard errors, and all baselines are in the paper.

Evaluation suites

iapp/ChindaMT-CoreEval: 800 items, five deployment domains
iapp/ChindaMT-BroadEval: 800 items, ten broader domains

Limitations

Thai-English only, in both directions.
Rule types are those a reference translation can demonstrate: terminology, length, register, and output format. Mandated glossaries and placeholder tokens are outside the training signal by design.
The primary judge shares the Qwen family with the model; cross-judges from two other families and non-LLM metrics are reported in the paper as checks.
Input length. Trained with a 1024-token cutoff. Inputs above 1k tokens still translated cleanly in our tests, but use them at your own risk. Recommended: translate long documents paragraph by paragraph.