wesjos/Qwen3-4B-math-GGUF

Model information

• These models are GGUF format of my finetuned models wesjos/Qwen3-4b-math.

Model Performence

Q4_K_M

• +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  | Model               | Dataset   | Metric          | Subset        |   Num |   Score | Cat.0   |
  +=====================+===========+=================+===============+=======+=========+=========+
  |  qwen3-4b-q4.gguf   | aime24    | AveragePass@1   | default       |    30 |  0.0333 | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q4.gguf   | arc       | AverageAccuracy | ARC-Easy      |   100 |  0.9    | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q4.gguf   | arc       | AverageAccuracy | ARC-Challenge |   100 |  0.8    | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q4.gguf   | arc       | AverageAccuracy | OVERALL       |   200 |  0.85   | -       |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q4.gguf   | gsm8k     | AverageAccuracy | main          |   100 |  0.81   | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+ 
  

Q5_K_M

• +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  | Model               | Dataset   | Metric          | Subset        |   Num |   Score | Cat.0   |
  +=====================+===========+=================+===============+=======+=========+=========+
  |  qwen3-4b-q5.gguf   | aime24    | AveragePass@1   | default       |    30 |  0.0667 | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q5.gguf   | arc       | AverageAccuracy | ARC-Easy      |   100 |  0.92   | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q5.gguf   | arc       | AverageAccuracy | ARC-Challenge |   100 |  0.84   | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q5.gguf   | arc       | AverageAccuracy | OVERALL       |   200 |  0.88   | -       |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q5.gguf   | gsm8k     | AverageAccuracy | main          |   100 |  0.84   | default |
  +---------------------+-----------+-----------------+---------------+-------+---------+---------+ 
  

Q8_0

• +-------------------+-----------+-----------------+---------------+-------+---------+---------+
  | Model             | Dataset   | Metric          | Subset        |   Num |   Score | Cat.0   |
  +===================+===========+=================+===============+=======+=========+=========+
  |  qwen3-4b-q8.gguf | aime24    | AveragePass@1   | default       |    30 |  0.0667 | default |
  +-------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q8.gguf | arc       | AverageAccuracy | ARC-Easy      |   100 |  0.92   | default |
  +-------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q8.gguf | arc       | AverageAccuracy | ARC-Challenge |   100 |  0.84   | default |
  +-------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q8.gguf | arc       | AverageAccuracy | OVERALL       |   200 |  0.88   | -       |
  +-------------------+-----------+-----------------+---------------+-------+---------+---------+
  |  qwen3-4b-q8.gguf | gsm8k     | AverageAccuracy | main          |   100 |  0.82   | default |
  +-------------------+-----------+-----------------+---------------+-------+---------+---------+ 
  

Use this model

with llama-cli

• llama-cli -m qwen3-4b-q4.gguf

with ollama

• edit a makefile named(qwen3-4b-q4.txt) like:

• FROM ./qwen3-4b-q4.gguf
  TEMPLATE """<|im_start|>system
  You are a helpful assistant<|im_end|>
  <|im_start|>user
  {{ .Prompt }}<|im_end|>
  <|im_start|>assistant
  """
  

• then create a model using ollama
• ollama create qwen3-4b-q4 -f qwen3-4b-q4.txt
• then run it
• ollama run qwen3-4b-q4