maniac-11111 commited on
Commit
3b59cf5
·
verified ·
1 Parent(s): 2719b8e

README: gold6 numbers (mean retention 95.1% -> 96.3%)

Browse files
Files changed (1) hide show
  1. README.md +25 -25
README.md CHANGED
@@ -22,7 +22,7 @@ tags:
22
 
23
  | | Mean retention, 12 benchmarks |
24
  | --- | ---: |
25
- | **Mach-1 Small** | **95.1%** |
26
  | Ternary Bonsai 27B (PrismML) | 93.6% |
27
  | Gemma 4 Q2_K_XL (Unsloth) | 85.6% |
28
 
@@ -30,36 +30,36 @@ tags:
30
 
31
  | Benchmark | Mach-1 Small | Ternary Bonsai 27B | Gemma 4 Q2_K_XL |
32
  | --- | ---: | ---: | ---: |
33
- | AIME26 | **93.5%** | 92.7% | 67.7% |
34
- | MATH-500 | **98.2%** | **98.2%** | 95.6% |
35
- | AIME25 | **92.5%** | 91.7% | 67.2% |
36
- | GSM8K | 99.0% | **100.2%** | 97.3% |
37
- | MBPP+ | 93.9% | **98.4%** | 92.2% |
38
- | HumanEval+ | 91.7% | **98.7%** | 94.1% |
39
- | MMLU-Redux | 96.0% | 94.0% | **96.9%** |
40
- | IFEval | **96.5%** | 89.8% | 95.5% |
41
- | MuSR | **97.6%** | 91.6% | 91.1% |
42
  | BFCL-v3 | 96.3% | **98.9%** | 95.7% |
43
- | τ²-bench | **95.5%** | 91.2% | 73.1% |
44
- | IFBench | **91.1%** | 77.7% | 61.3% |
45
- | **Mean** | **95.1%** | **93.6%** | **85.6%** |
46
 
47
  Mach-1 Small's own scores and teacher scores:
48
 
49
  | Benchmark | Score | Teacher (Qwen3.6-35B-A3B BF16) | Retention |
50
  | --- | ---: | ---: | ---: |
51
- | AIME26 | 84.17 | 90.00 | 93.5% |
52
- | MATH-500 | 96.80 | 98.60 | 98.2% |
53
- | AIME25 | 81.67 | 88.33 | 92.5% |
54
- | GSM8K | 95.22 | 96.21 | 99.0% |
55
- | MBPP+ | 90.21 | 96.03 | 93.9% |
56
- | HumanEval+ | 87.20 | 95.12 | 91.7% |
57
- | MMLU-Redux | 88.93 | 92.68 | 96.0% |
58
- | IFEval | 85.90 | 89.05 | 96.5% |
59
- | MuSR | 65.08 | 66.66 | 97.6% |
60
- | BFCL-v3 | 72.21 | 74.98 | 96.3% |
61
- | τ²-bench | 75.90 | 79.51 | 95.5% |
62
- | IFBench | 59.18 | 64.97 | 91.1% |
63
 
64
  ### Speed
65
 
 
22
 
23
  | | Mean retention, 12 benchmarks |
24
  | --- | ---: |
25
+ | **Mach-1 Small** | **96.3%** |
26
  | Ternary Bonsai 27B (PrismML) | 93.6% |
27
  | Gemma 4 Q2_K_XL (Unsloth) | 85.6% |
28
 
 
30
 
31
  | Benchmark | Mach-1 Small | Ternary Bonsai 27B | Gemma 4 Q2_K_XL |
32
  | --- | ---: | ---: | ---: |
33
+ | AIME26 | **95.8%** | 92.7% | 67.7% |
34
+ | MATH-500 | **99.2%** | 98.2% | 95.6% |
35
+ | AIME25 | **93.9%** | 91.7% | 67.2% |
36
+ | GSM8K | 99.8% | **100.2%** | 97.3% |
37
+ | MBPP+ | 98.1% | **98.4%** | 92.2% |
38
+ | HumanEval+ | 96.5% | **98.7%** | 94.1% |
39
+ | MMLU-Redux | **97.0%** | 94.0% | 96.9% |
40
+ | IFEval | 94.8% | 89.8% | **95.5%** |
41
+ | MuSR | **100.4%** | 91.6% | 91.1% |
42
  | BFCL-v3 | 96.3% | **98.9%** | 95.7% |
43
+ | τ²-bench | **94.1%** | 91.2% | 73.1% |
44
+ | IFBench | **89.8%** | 77.7% | 61.3% |
45
+ | **Mean** | **96.3%** | 93.6% | 85.6% |
46
 
47
  Mach-1 Small's own scores and teacher scores:
48
 
49
  | Benchmark | Score | Teacher (Qwen3.6-35B-A3B BF16) | Retention |
50
  | --- | ---: | ---: | ---: |
51
+ | AIME26 | 86.25 | 90.00 | 95.8% |
52
+ | MATH-500 | 97.80 | 98.60 | 99.2% |
53
+ | AIME25 | 82.92 | 88.33 | 93.9% |
54
+ | GSM8K | 95.98 | 96.21 | 99.8% |
55
+ | MBPP+ | 94.18 | 96.03 | 98.1% |
56
+ | HumanEval+ | 91.77 | 95.12 | 96.5% |
57
+ | MMLU-Redux | 89.88 | 92.68 | 97.0% |
58
+ | IFEval | 84.42 | 89.05 | 94.8% |
59
+ | MuSR | 66.93 | 66.66 | 100.4% |
60
+ | BFCL-v3 | 72.19 | 74.98 | 96.3% |
61
+ | τ²-bench | 74.82 | 79.51 | 94.1% |
62
+ | IFBench | 58.33 | 64.97 | 89.8% |
63
 
64
  ### Speed
65