Add info to readme
Browse files
README.md
CHANGED
@@ -24,7 +24,8 @@ tags:
|
|
24 |
- voice
|
25 |
- distil
|
26 |
---
|
27 |
-
|
|
|
28 |
## List of Quants
|
29 |
Clicking on a link will download the corresponding quant instantly.
|
30 |
| Link | Quant | Size |
|
@@ -40,4 +41,11 @@ Clicking on a link will download the corresponding quant instantly.
|
|
40 |
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q4_1.bin) | Q4_1 | 491 MB |
|
41 |
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q4_0.bin) | Q4_0 | 444 MB |
|
42 |
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q3_k.bin) | Q3_K | 345 MB |
|
43 |
-
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q2_k.bin) | Q2_K (nonsensical output) | 269 MB |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
- voice
|
25 |
- distil
|
26 |
---
|
27 |
+
# Distil-Large-v3.5 quants
|
28 |
+
This is a repository of **GGML quants for [distil-large-v3.5](https://huggingface.co/distil-whisper/distil-large-v3.5)** (a Whisper-based transcription model) for use with [whisper.cpp](https://github.com/ggml-org/whisper.cpp). If you are looking for a program to run this model with, then I would recommend [EasyWhisper UI](https://github.com/mehtabmahir/easy-whisper-ui), as it is user-friendly, has a GUI, and automates a lot of the hard stuff for you.
|
29 |
## List of Quants
|
30 |
Clicking on a link will download the corresponding quant instantly.
|
31 |
| Link | Quant | Size |
|
|
|
41 |
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q4_1.bin) | Q4_1 | 491 MB |
|
42 |
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q4_0.bin) | Q4_0 | 444 MB |
|
43 |
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q3_k.bin) | Q3_K | 345 MB |
|
44 |
+
| [GGML](https://huggingface.co/Pomni/distil-large-v3.5-ggml-allquants/resolve/main/ggml-distil-large-v3.5-q2_k.bin) | Q2_K (nonsensical output) | 269 MB |
|
45 |
+
## Questions you may have
|
46 |
+
### Why do the "K-quants" not work for me?
|
47 |
+
My guess is that your GPU might be too old to recognize them, considering that I have gotten the same error on my GTX 1080. If you would like to run them regardless, you can try switching to CPU inference.
|
48 |
+
### Are the K-quants "S", "M", or "L"?
|
49 |
+
The quantizer I was using was not specific about this, so I do not know about this either.
|
50 |
+
### What program did you use to make these quants?
|
51 |
+
I used [whisper.cpp v1.7.6](https://github.com/ggml-org/whisper.cpp/releases/tag/v1.7.6) on Windows x64, leveraging CUDA 12.4.0. For the F32 quant, I converted the original Hugging Face (H5) format model to a GGML using the `models/convert-h5-to-ggml.py` script.
|