Update README.md
Browse files
README.md
CHANGED
|
@@ -46,3 +46,16 @@ TODO:
|
|
| 46 |
| Reka-Flash 21B | 19,648 |
|
| 47 |
| Mistral 2503 | 32,768 |
|
| 48 |
| Codestral 22B | 16,384 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
| Reka-Flash 21B | 19,648 |
|
| 47 |
| Mistral 2503 | 32,768 |
|
| 48 |
| Codestral 22B | 16,384 |
|
| 49 |
+
|
| 50 |
+
------------
|
| 51 |
+
|
| 52 |
+
## T-MAC (larger groupsize 128?)
|
| 53 |
+
|
| 54 |
+
| Model | Size | Params | Backend | Threads | Test | t/s (tokens/sec) |
|
| 55 |
+
|-------------------------|---------|--------|---------|---------|--------|----------------------|
|
| 56 |
+
| qwen2 3B Q4_K - Medium | 1.95 GiB| 3.40 B | CPU | 4 | pp512 | 67.33 ± 0.10 |
|
| 57 |
+
| qwen2 3B Q4_K - Medium | 1.95 GiB| 3.40 B | CPU | 4 | tg128 | 22.72 ± 0.04 |
|
| 58 |
+
| qwen2 ?B INT_N Q4_K | 1.70 GiB| 3.40 B | CPU | 4 | pp512 | 59.66 ± 0.10 |
|
| 59 |
+
| qwen2 ?B INT_N Q4_K | 1.70 GiB| 3.40 B | CPU | 4 | tg128 | 26.43 ± 0.14 |
|
| 60 |
+
|
| 61 |
+
[Test Issue Link](https://github.com/microsoft/T-MAC/issues/79)
|