bartowski commited on
Commit
96165fe
·
1 Parent(s): bd6ade8

Fix naming

Browse files
Files changed (29) hide show
  1. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf +0 -0
  2. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf +0 -0
  3. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf +0 -0
  4. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf +0 -0
  5. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf +0 -0
  6. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf +0 -0
  7. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf +0 -0
  8. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf +0 -0
  9. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf +0 -0
  10. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf +0 -0
  11. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf +0 -0
  12. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf +0 -0
  13. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf +0 -0
  14. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf +0 -0
  15. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf +0 -0
  16. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf +0 -0
  17. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf +0 -0
  18. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf +0 -0
  19. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf +0 -0
  20. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf +0 -0
  21. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf +0 -0
  22. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf +0 -0
  23. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf +0 -0
  24. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf +0 -0
  25. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf +0 -0
  26. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16-00001-of-00002.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16-00001-of-00002.gguf +0 -0
  27. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16-00002-of-00002.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16-00002-of-00002.gguf +0 -0
  28. FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview.imatrix → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview.imatrix +0 -0
  29. README.md +32 -32
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16-00001-of-00002.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16-00001-of-00002.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16-00002-of-00002.gguf → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16-00002-of-00002.gguf RENAMED
File without changes
FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview.imatrix → FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview.imatrix RENAMED
File without changes
README.md CHANGED
@@ -1,14 +1,14 @@
1
  ---
2
  quantized_by: bartowski
3
  pipeline_tag: text-generation
4
- base_model: FuseAI/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview
5
  ---
6
 
7
- ## Llamacpp imatrix Quantizations of FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview
8
 
9
  Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4514">b4514</a> for quantization.
10
 
11
- Original model: https://huggingface.co/FuseAI/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview
12
 
13
  All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
14
 
@@ -24,32 +24,32 @@ Run them in [LM Studio](https://lmstudio.ai/)
24
 
25
  | Filename | Quant type | File Size | Split | Description |
26
  | -------- | ---------- | --------- | ----- | ----------- |
27
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/tree/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-bf16) | bf16 | 65.54GB | true | Full BF16 weights. |
28
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf) | Q8_0 | 34.82GB | false | Extremely high quality, generally unneeded but max available quant. |
29
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf) | Q6_K_L | 27.26GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
30
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf) | Q6_K | 26.89GB | false | Very high quality, near perfect, *recommended*. |
31
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf) | Q5_K_L | 23.74GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
32
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf) | Q5_K_M | 23.26GB | false | High quality, *recommended*. |
33
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf) | Q5_K_S | 22.64GB | false | High quality, *recommended*. |
34
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf) | Q4_1 | 20.64GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
35
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf) | Q4_K_L | 20.43GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
36
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf) | Q4_K_M | 19.85GB | false | Good quality, default size for most use cases, *recommended*. |
37
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf) | Q4_K_S | 18.78GB | false | Slightly lower quality with more space savings, *recommended*. |
38
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf) | Q4_0 | 18.71GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
39
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf) | IQ4_NL | 18.68GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
40
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf) | Q3_K_XL | 17.93GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
41
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf) | IQ4_XS | 17.69GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
42
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf) | Q3_K_L | 17.25GB | false | Lower quality but usable, good for low RAM availability. |
43
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf) | Q3_K_M | 15.94GB | false | Low quality. |
44
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf) | IQ3_M | 14.81GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
45
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf) | Q3_K_S | 14.39GB | false | Low quality, not recommended. |
46
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf) | IQ3_XS | 13.71GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
47
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf) | Q2_K_L | 13.07GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
48
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf) | Q2_K | 12.31GB | false | Very low quality but surprisingly usable. |
49
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf) | IQ2_M | 11.26GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
50
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf) | IQ2_S | 10.39GB | false | Low quality, uses SOTA techniques to be usable. |
51
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf) | IQ2_XS | 9.96GB | false | Low quality, uses SOTA techniques to be usable. |
52
- | [FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf](https://huggingface.co/bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf) | IQ2_XXS | 9.03GB | false | Very low quality, uses SOTA techniques to be usable. |
53
 
54
  ## Embed/output weights
55
 
@@ -69,16 +69,16 @@ pip install -U "huggingface_hub[cli]"
69
  Then, you can target the specific file you want:
70
 
71
  ```
72
- huggingface-cli download bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF --include "FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf" --local-dir ./
73
  ```
74
 
75
  If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
76
 
77
  ```
78
- huggingface-cli download bartowski/FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-GGUF --include "FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q8_0/*" --local-dir ./
79
  ```
80
 
81
- You can either specify a new local-dir (FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q8_0) or download them all in place (./)
82
 
83
  </details>
84
 
 
1
  ---
2
  quantized_by: bartowski
3
  pipeline_tag: text-generation
4
+ base_model: FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview
5
  ---
6
 
7
+ ## Llamacpp imatrix Quantizations of FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview
8
 
9
  Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4514">b4514</a> for quantization.
10
 
11
+ Original model: https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview
12
 
13
  All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
14
 
 
24
 
25
  | Filename | Quant type | File Size | Split | Description |
26
  | -------- | ---------- | --------- | ----- | ----------- |
27
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/tree/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-bf16) | bf16 | 65.54GB | true | Full BF16 weights. |
28
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q8_0.gguf) | Q8_0 | 34.82GB | false | Extremely high quality, generally unneeded but max available quant. |
29
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K_L.gguf) | Q6_K_L | 27.26GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
30
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q6_K.gguf) | Q6_K | 26.89GB | false | Very high quality, near perfect, *recommended*. |
31
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_L.gguf) | Q5_K_L | 23.74GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
32
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_M.gguf) | Q5_K_M | 23.26GB | false | High quality, *recommended*. |
33
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q5_K_S.gguf) | Q5_K_S | 22.64GB | false | High quality, *recommended*. |
34
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_1.gguf) | Q4_1 | 20.64GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
35
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_L.gguf) | Q4_K_L | 20.43GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
36
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf) | Q4_K_M | 19.85GB | false | Good quality, default size for most use cases, *recommended*. |
37
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_S.gguf) | Q4_K_S | 18.78GB | false | Slightly lower quality with more space savings, *recommended*. |
38
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_0.gguf) | Q4_0 | 18.71GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
39
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_NL.gguf) | IQ4_NL | 18.68GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
40
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_XL.gguf) | Q3_K_XL | 17.93GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
41
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ4_XS.gguf) | IQ4_XS | 17.69GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
42
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_L.gguf) | Q3_K_L | 17.25GB | false | Lower quality but usable, good for low RAM availability. |
43
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_M.gguf) | Q3_K_M | 15.94GB | false | Low quality. |
44
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_M.gguf) | IQ3_M | 14.81GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
45
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q3_K_S.gguf) | Q3_K_S | 14.39GB | false | Low quality, not recommended. |
46
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ3_XS.gguf) | IQ3_XS | 13.71GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
47
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K_L.gguf) | Q2_K_L | 13.07GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
48
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q2_K.gguf) | Q2_K | 12.31GB | false | Very low quality but surprisingly usable. |
49
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_M.gguf) | IQ2_M | 11.26GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
50
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_S.gguf) | IQ2_S | 10.39GB | false | Low quality, uses SOTA techniques to be usable. |
51
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XS.gguf) | IQ2_XS | 9.96GB | false | Low quality, uses SOTA techniques to be usable. |
52
+ | [FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf](https://huggingface.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF/blob/main/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-IQ2_XXS.gguf) | IQ2_XXS | 9.03GB | false | Very low quality, uses SOTA techniques to be usable. |
53
 
54
  ## Embed/output weights
55
 
 
69
  Then, you can target the specific file you want:
70
 
71
  ```
72
+ huggingface-cli download bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF --include "FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf" --local-dir ./
73
  ```
74
 
75
  If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
76
 
77
  ```
78
+ huggingface-cli download bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF --include "FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q8_0/*" --local-dir ./
79
  ```
80
 
81
+ You can either specify a new local-dir (FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-Q8_0) or download them all in place (./)
82
 
83
  </details>
84