Add quant table
Browse files
README.md
CHANGED
|
@@ -23,4 +23,21 @@ tags:
|
|
| 23 |
- speech
|
| 24 |
- voice
|
| 25 |
- distil
|
| 26 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
- speech
|
| 24 |
- voice
|
| 25 |
- distil
|
| 26 |
+
---
|
| 27 |
+
|
| 28 |
+
## List of Quants
|
| 29 |
+
Clicking on a link will download the corresponding quant instantly.
|
| 30 |
+
| Link | Quant | Size |
|
| 31 |
+
|:-----|:-----|--------:|
|
| 32 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-f32.bin) | F32 (likely overkill) | 3.03 GB |
|
| 33 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-f16.bin) | F16 | 1.52 GB |
|
| 34 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q8_0.bin) | Q8_0 (sweet spot) | 818 MB |
|
| 35 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q6_k.bin) | Q6_K | 637 MB |
|
| 36 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q5_k.bin) | Q5_K | 538 MB |
|
| 37 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q5_1.bin) | Q5_1 | 585 MB |
|
| 38 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q5_0.bin) | Q5_0 | 538 MB |
|
| 39 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q4_k.bin) | Q4_K | 444 MB |
|
| 40 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q4_1.bin) | Q4_1 | 491 MB |
|
| 41 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q4_0.bin) | Q4_0 | 444 MB |
|
| 42 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q3_k.bin) | Q3_K | 345 MB |
|
| 43 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q2_k.bin) | Q2_K (nonsensical output) | 269 MB |
|