Hunminai-1.0-12b

Hunminai-1.0 is a Korean-aligned language model based on Google's Gemma-3 architecture. To improve performance on Korean natural language tasks, the model was fine-tuned on a corpus of 100k instruction examples using Supervised Fine-Tuning (SFT) followed by Direct Preference Optimization (DPO). This approach enables the model to better align with user intents in Korean and enhances its applicability to downstream tasks such as dialogue generation, question answering, and long-form text generation.

Model Details

• Base Model: google/gemma-3-12b-it
• Base Model Release Date: March 12, 2025
• Context Length: 128k
• License: gemma
• Model Type: Text Generation
• Fine-Tuning Techniques: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO)

Usage

Gemma 3 is supported starting from version 4.50.0 of the Transformers library.

To update to the latest version, run the following command:

$ pip install -U transformers

Install the required package and run the example code below to load the Hunminai-3-12B model and perform a simple Korean-language chat completion.

# pip install accelerate

from transformers import AutoProcessor, Gemma3ForConditionalGeneration
import torch

model_id = "davidkim205/Hunminai-1.0-12b"

model = Gemma3ForConditionalGeneration.from_pretrained(
    model_id, device_map="auto"
).eval()

processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "system",
        "content": [{"type": "text", "text": "당신은 유용한 AI 비서입니다."}]
    },
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "대한민국의 수도는 어디인가요?"}
        ]
    }
]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt"
).to(model.device, dtype=torch.bfloat16)

input_len = inputs["input_ids"].shape[-1]

with torch.inference_mode():
    generation = model.generate(**inputs, max_new_tokens=128, do_sample=False)
    generation = generation[0][input_len:]

decoded = processor.decode(generation, skip_special_tokens=True)
print(decoded)

Training Dataset

The model was trained on approximately 100k high-quality Korean instruction examples. The dataset was curated to cover a wide range of Korean language contexts and tasks, with a focus on aligning model outputs with user intent and natural language generation, and is currently undisclosed.

Evaluation

Benchmarks Datasets

The table below contains a description of the Korean LLM evaluation benchmark dataset used for the model evaluation. More information on the benchmarks is available at Blog.

Benchmark	Description	Abbreviation
ko-bench	Korean-translated dataset of MT-Bench questions	bench
ko-ged	Korean GED (elementary, middle, high school) open-ended question dataset Subjects: Korean, English, Mathematics, Science, Social Studies	ged
ko-ifeval	Instruction-following evaluation dataset translated from IFEval, adapted for Korean language and culture	ifeval
ko-ged-mc-elementary	Korean elementary school GED multiple-choice question dataset	ged:E
ko-ged-mc-middle	Korean middle school GED multiple-choice question dataset	ged:M
ko-ged-mc-high	Korean high school GED multiple-choice question dataset	ged:H
ko-gpqa	Korean version of GPQA containing challenging physics questions designed to test deep understanding and logical reasoning	gpqa
ko-math-500	Korean-translated subset of 500 high school-level math problems from the MATH dataset, including detailed solutions with LaTeX notation	math500

Benchmark Results

	davidkim205 Hunminai -1.0-12b	google gemma-3 -12b-it	unsloth gemma-3 -12b-it	K-intelligence Midm-2.0 -Base-Instruct	LGAI-EXAONE EXAONE-3.5 -7.8B-Instruct
Avg.	7.80	7.75	7.71	7.54	7.31
bench	7.96	8.00	7.83	8.01	7.70
ged	8.65	8.61	8.73	8.10	8.25
ged:E	9.72	9.72	9.51	9.72	9.65
ged:M	9.63	9.55	9.39	9.31	9.10
ged:H	9.32	9.36	9.24	9.48	9.00
gpqa	3.18	2.88	2.98	2.68	3.13
math500	5.60	5.58	5.70	4.80	4.88
ifeval	8.37	8.30	8.33	8.24	6.76