README: gold6 numbers (mean retention 95.1% -> 96.3%)
Browse files
README.md
CHANGED
|
@@ -22,7 +22,7 @@ tags:
|
|
| 22 |
|
| 23 |
| | Mean retention, 12 benchmarks |
|
| 24 |
| --- | ---: |
|
| 25 |
-
| **Mach-1 Small** | **
|
| 26 |
| Ternary Bonsai 27B (PrismML) | 93.6% |
|
| 27 |
| Gemma 4 Q2_K_XL (Unsloth) | 85.6% |
|
| 28 |
|
|
@@ -30,36 +30,36 @@ tags:
|
|
| 30 |
|
| 31 |
| Benchmark | Mach-1 Small | Ternary Bonsai 27B | Gemma 4 Q2_K_XL |
|
| 32 |
| --- | ---: | ---: | ---: |
|
| 33 |
-
| AIME26 | **
|
| 34 |
-
| MATH-500 | **
|
| 35 |
-
| AIME25 | **
|
| 36 |
-
| GSM8K | 99.
|
| 37 |
-
| MBPP+ |
|
| 38 |
-
| HumanEval+ |
|
| 39 |
-
| MMLU-Redux |
|
| 40 |
-
| IFEval |
|
| 41 |
-
| MuSR | **
|
| 42 |
| BFCL-v3 | 96.3% | **98.9%** | 95.7% |
|
| 43 |
-
| τ²-bench | **
|
| 44 |
-
| IFBench | **
|
| 45 |
-
| **Mean** | **
|
| 46 |
|
| 47 |
Mach-1 Small's own scores and teacher scores:
|
| 48 |
|
| 49 |
| Benchmark | Score | Teacher (Qwen3.6-35B-A3B BF16) | Retention |
|
| 50 |
| --- | ---: | ---: | ---: |
|
| 51 |
-
| AIME26 |
|
| 52 |
-
| MATH-500 |
|
| 53 |
-
| AIME25 |
|
| 54 |
-
| GSM8K | 95.
|
| 55 |
-
| MBPP+ |
|
| 56 |
-
| HumanEval+ |
|
| 57 |
-
| MMLU-Redux |
|
| 58 |
-
| IFEval |
|
| 59 |
-
| MuSR |
|
| 60 |
-
| BFCL-v3 | 72.
|
| 61 |
-
| τ²-bench |
|
| 62 |
-
| IFBench |
|
| 63 |
|
| 64 |
### Speed
|
| 65 |
|
|
|
|
| 22 |
|
| 23 |
| | Mean retention, 12 benchmarks |
|
| 24 |
| --- | ---: |
|
| 25 |
+
| **Mach-1 Small** | **96.3%** |
|
| 26 |
| Ternary Bonsai 27B (PrismML) | 93.6% |
|
| 27 |
| Gemma 4 Q2_K_XL (Unsloth) | 85.6% |
|
| 28 |
|
|
|
|
| 30 |
|
| 31 |
| Benchmark | Mach-1 Small | Ternary Bonsai 27B | Gemma 4 Q2_K_XL |
|
| 32 |
| --- | ---: | ---: | ---: |
|
| 33 |
+
| AIME26 | **95.8%** | 92.7% | 67.7% |
|
| 34 |
+
| MATH-500 | **99.2%** | 98.2% | 95.6% |
|
| 35 |
+
| AIME25 | **93.9%** | 91.7% | 67.2% |
|
| 36 |
+
| GSM8K | 99.8% | **100.2%** | 97.3% |
|
| 37 |
+
| MBPP+ | 98.1% | **98.4%** | 92.2% |
|
| 38 |
+
| HumanEval+ | 96.5% | **98.7%** | 94.1% |
|
| 39 |
+
| MMLU-Redux | **97.0%** | 94.0% | 96.9% |
|
| 40 |
+
| IFEval | 94.8% | 89.8% | **95.5%** |
|
| 41 |
+
| MuSR | **100.4%** | 91.6% | 91.1% |
|
| 42 |
| BFCL-v3 | 96.3% | **98.9%** | 95.7% |
|
| 43 |
+
| τ²-bench | **94.1%** | 91.2% | 73.1% |
|
| 44 |
+
| IFBench | **89.8%** | 77.7% | 61.3% |
|
| 45 |
+
| **Mean** | **96.3%** | 93.6% | 85.6% |
|
| 46 |
|
| 47 |
Mach-1 Small's own scores and teacher scores:
|
| 48 |
|
| 49 |
| Benchmark | Score | Teacher (Qwen3.6-35B-A3B BF16) | Retention |
|
| 50 |
| --- | ---: | ---: | ---: |
|
| 51 |
+
| AIME26 | 86.25 | 90.00 | 95.8% |
|
| 52 |
+
| MATH-500 | 97.80 | 98.60 | 99.2% |
|
| 53 |
+
| AIME25 | 82.92 | 88.33 | 93.9% |
|
| 54 |
+
| GSM8K | 95.98 | 96.21 | 99.8% |
|
| 55 |
+
| MBPP+ | 94.18 | 96.03 | 98.1% |
|
| 56 |
+
| HumanEval+ | 91.77 | 95.12 | 96.5% |
|
| 57 |
+
| MMLU-Redux | 89.88 | 92.68 | 97.0% |
|
| 58 |
+
| IFEval | 84.42 | 89.05 | 94.8% |
|
| 59 |
+
| MuSR | 66.93 | 66.66 | 100.4% |
|
| 60 |
+
| BFCL-v3 | 72.19 | 74.98 | 96.3% |
|
| 61 |
+
| τ²-bench | 74.82 | 79.51 | 94.1% |
|
| 62 |
+
| IFBench | 58.33 | 64.97 | 89.8% |
|
| 63 |
|
| 64 |
### Speed
|
| 65 |
|