Update README.md
Browse files
README.md
CHANGED
|
@@ -12,16 +12,20 @@ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a
|
|
| 12 |
|
| 13 |
Original model: https://huggingface.co/deepseek-ai/DeepSeek-R1-0528
|
| 14 |
|
|
|
|
|
|
|
| 15 |
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
|
| 16 |
|
| 17 |
Run them in [LM Studio](https://lmstudio.ai/)
|
| 18 |
|
| 19 |
Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
|
| 20 |
|
|
|
|
|
|
|
| 21 |
## Prompt format
|
| 22 |
|
| 23 |
```
|
| 24 |
-
<｜begin▁of▁sentence｜>{system_prompt}<｜User｜>{prompt}<｜Assistant
|
| 25 |
```
|
| 26 |
|
| 27 |
## Download a file (not the whole branch) from below:
|
|
|
|
| 12 |
|
| 13 |
Original model: https://huggingface.co/deepseek-ai/DeepSeek-R1-0528
|
| 14 |
|
| 15 |
+
Huge shoutout to [Artus](https://huggingface.co/ArtusDev) who helped with both conversion and imatrix calculation!
|
| 16 |
+
|
| 17 |
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
|
| 18 |
|
| 19 |
Run them in [LM Studio](https://lmstudio.ai/)
|
| 20 |
|
| 21 |
Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
|
| 22 |
|
| 23 |
+
These quants were made with the changes from this PR for improved performance: https://github.com/ggml-org/llama.cpp/pull/12727
|
| 24 |
+
|
| 25 |
## Prompt format
|
| 26 |
|
| 27 |
```
|
| 28 |
+
<｜begin▁of▁sentence｜>{system_prompt}<｜User｜>{prompt}<｜Assistant｜>
|
| 29 |
```
|
| 30 |
|
| 31 |
## Download a file (not the whole branch) from below:
|