Hugging Face's logo

ThomasBaruzier
/
DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF

Text Generation
GGUF
conversational
Model card Files Community
ThomasBaruzier commited on
Commit
796d845
·
verified ·
1 Parent(s): 2e1cbce

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -30
README.md CHANGED
@@ -19,36 +19,36 @@ All quants were made using the imatrix option and Bartowski's [calibration file]
19
 
20
  # Perplexity table (the lower the better)
21
 
22
- | Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
23
- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | --------- | ------- | -------- | ------------ | -------------- |
24
- | [IQ1_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ1_S.gguf) | 489 | 88.4250 | 14.40 | 23.35 | 1.76 |
25
- | [IQ1_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ1_M.gguf) | 516 | 53.8278 | 15.19 | 38.35 | 1.03 |
26
- | [IQ2_XXS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_XXS.gguf) | 560 | 45.5693 | 16.49 | 45.31 | 0.93 |
27
- | [IQ2_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_XS.gguf) | 598 | 32.6813 | 17.61 | 63.17 | 0.62 |
28
- | [IQ2_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_S.gguf) | 633 | 28.5477 | 18.64 | 72.32 | 0.54 |
29
- | [IQ2_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_M.gguf) | 669 | 31.8272 | 19.70 | 64.87 | 0.63 |
30
- | [Q2_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q2_K_S.gguf) | 683 | 28.7707 | 20.11 | 71.76 | 0.54 |
31
- | [Q2_K](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q2_K.gguf) | 718 | 27.6342 | 21.14 | 74.71 | 0.51 |
32
- | [IQ3_XXS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_XXS.gguf) | 733 | 23.5511 | 21.58 | 87.66 | 0.44 |
33
- | [IQ3_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_XS.gguf) | 793 | 22.9887 | 23.35 | 89.81 | 0.42 |
34
- | [Q3_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q3_K_S.gguf) | 821 | 28.0462 | 24.17 | 73.61 | 0.53 |
35
- | [IQ3_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_S.gguf) | 822 | 22.9268 | 24.20 | 90.05 | 0.42 |
36
- | [IQ3_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_M.gguf) | 836 | 22.3167 | 24.62 | 92.51 | 0.41 |
37
- | [Q3_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q3_K_M.gguf) | 881 | 22.5727 | 25.94 | 91.46 | 0.41 |
38
- | [Q3_K_L](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q3_K_L.gguf) | 935 | 22.3758 | 27.53 | 92.27 | 0.41 |
39
- | [IQ4_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ4_XS.gguf) | 972 | 21.3273 | 28.62 | 96.80 | 0.38 |
40
- | [IQ4_NL](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ4_NL.gguf) | 1018 | 21.3234 | 29.98 | 96.82 | 0.38 |
41
- | [Q4_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_0.gguf) | 1019 | 22.5210 | 30.00 | 91.67 | 0.41 |
42
- | [Q4_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_K_S.gguf) | 1022 | 21.1717 | 30.09 | 97.51 | 0.38 |
43
- | [Q4_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_K_M.gguf) | 1065 | 21.0532 | 31.36 | 98.06 | 0.38 |
44
- | [Q4_1](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_1.gguf) | 1109 | 21.1492 | 32.66 | 97.62 | 0.38 |
45
- | [Q5_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_K_S.gguf) | 1201 | 20.7883 | 35.37 | 99.31 | 0.37 |
46
- | [Q5_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_0.gguf) | 1203 | 20.8643 | 35.42 | 98.95 | 0.37 |
47
- | [Q5_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_K_M.gguf) | 1226 | 20.7488 | 36.10 | 99.50 | 0.37 |
48
- | [Q5_1](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_1.gguf) | 1293 | 20.7773 | 38.07 | 99.37 | 0.37 |
49
- | [Q6_K](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q6_K.gguf) | 1396 | 20.6994 | 41.11 | 99.74 | 0.37 |
50
- | [Q8_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q8_0.gguf) | 1807 | 20.6659 | 53.21 | 99.90 | 0.37 |
51
- | [F16](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-F16.gguf) | 3396 | 20.6457 | 100 | 100 | 0.37 |
52
 
53
  <hr>
54
 
 
19
 
20
  # Perplexity table (the lower the better)
21
 
22
+ | Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
23
+ | -------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | ------- | -------- | ------------ | -------------- |
24
+ | [IQ1_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ1_S.gguf) | 1815 | 29.3739 | 12.49 | 49.92 | 0.53 |
25
+ | [IQ1_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ1_M.gguf) | 1947 | 23.4611 | 13.40 | 62.50 | 0.42 |
26
+ | [IQ2_XXS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_XXS.gguf) | 2167 | 23.8257 | 14.91 | 61.54 | 0.46 |
27
+ | [IQ2_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_XS.gguf) | 2354 | 20.5413 | 16.20 | 71.38 | 0.39 |
28
+ | [IQ2_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_S.gguf) | 2475 | 19.3763 | 17.03 | 75.67 | 0.36 |
29
+ | [IQ2_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ2_M.gguf) | 2651 | 22.3007 | 18.24 | 65.75 | 0.44 |
30
+ | [Q2_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q2_K_S.gguf) | 2702 | 17.5446 | 18.59 | 83.57 | 0.31 |
31
+ | [Q2_K](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q2_K.gguf) | 2876 | 16.9426 | 19.79 | 86.54 | 0.29 |
32
+ | [IQ3_XXS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_XXS.gguf) | 2970 | 16.2668 | 20.44 | 90.14 | 0.29 |
33
+ | [IQ3_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_XS.gguf) | 3191 | 16.1443 | 21.96 | 90.82 | 0.29 |
34
+ | [Q3_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q3_K_S.gguf) | 3330 | 17.0364 | 22.92 | 86.07 | 0.29 |
35
+ | [IQ3_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_S.gguf) | 3337 | 16.1048 | 22.96 | 91.04 | 0.29 |
36
+ | [IQ3_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ3_M.gguf) | 3408 | 15.8128 | 23.45 | 92.72 | 0.28 |
37
+ | [Q3_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q3_K_M.gguf) | 3631 | 15.2580 | 24.99 | 96.10 | 0.26 |
38
+ | [Q3_K_L](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q3_K_L.gguf) | 3899 | 15.1997 | 26.83 | 96.46 | 0.26 |
39
+ | [IQ4_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ4_XS.gguf) | 4023 | 14.9385 | 27.68 | 98.15 | 0.25 |
40
+ | [IQ4_NL](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-IQ4_NL.gguf) | 4232 | 14.9257 | 29.12 | 98.24 | 0.25 |
41
+ | [Q4_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_0.gguf) | 4238 | 15.2621 | 29.17 | 96.07 | 0.26 |
42
+ | [Q4_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_K_S.gguf) | 4251 | 14.8852 | 29.25 | 98.50 | 0.26 |
43
+ | [Q4_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_K_M.gguf) | 4466 | 14.8666 | 30.73 | 98.63 | 0.26 |
44
+ | [Q4_1](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q4_1.gguf) | 4647 | 14.8789 | 31.98 | 98.54 | 0.26 |
45
+ | [Q5_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_K_S.gguf) | 5068 | 14.7449 | 34.88 | 99.44 | 0.25 |
46
+ | [Q5_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_0.gguf) | 5081 | 14.7425 | 34.97 | 99.46 | 0.25 |
47
+ | [Q5_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_K_M.gguf) | 5192 | 14.7327 | 35.73 | 99.52 | 0.25 |
48
+ | [Q5_1](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q5_1.gguf) | 5490 | 14.7293 | 37.78 | 99.55 | 0.25 |
49
+ | [Q6_K](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q6_K.gguf) | 5964 | 14.6907 | 41.04 | 99.81 | 0.25 |
50
+ | [Q8_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-Q8_0.gguf) | 7723 | 14.6686 | 53.15 | 99.96 | 0.25 |
51
+ | [F16](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-7B-v1.1-F16.gguf) | 14531 | 14.6625 | 100 | 100 | 0.25 |
52
 
53
  <hr>
54