Hugging Face's logo

mradermacher
/
L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF

Transformers
GGUF
English
creative
creative writing
fiction writing
plot generation
sub-plot generation
llama 3.1
llama-3
llama3
llama-3.1
story generation
scene continue
storytelling
fiction story
science fiction
romance
all genres
story
writing
vivid prosing
vivid writing
fiction
roleplaying
bfloat16
swearing
role play
sillytavern
backyard
horror
context 128k
mergekit
Merge
Not-For-All-Audiences
imatrix
conversational
Model card Files
xet
Community
mradermacher commited on
Commit
3eddcc4
·
verified ·
1 Parent(s): 0a779c7

auto-patch README.md

Browse files
Files changed (1) hide show
  1. README.md +101 -0
README.md CHANGED
@@ -1,6 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <!-- ### quantize_version: 2 -->
2
  <!-- ### output_tensor_quantised: 1 -->
3
  <!-- ### convert_type: hf -->
4
  <!-- ### vocab_type: -->
5
  <!-- ### tags: nicoboss -->
6
  weighted/imatrix quants of https://huggingface.co/DavidAU/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: DavidAU/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ license: apache-2.0
7
+ quantized_by: mradermacher
8
+ tags:
9
+ - creative
10
+ - creative writing
11
+ - fiction writing
12
+ - plot generation
13
+ - sub-plot generation
14
+ - fiction writing
15
+ - story generation
16
+ - scene continue
17
+ - storytelling
18
+ - fiction story
19
+ - science fiction
20
+ - romance
21
+ - all genres
22
+ - story
23
+ - writing
24
+ - vivid prosing
25
+ - vivid writing
26
+ - fiction
27
+ - roleplaying
28
+ - bfloat16
29
+ - swearing
30
+ - role play
31
+ - sillytavern
32
+ - backyard
33
+ - horror
34
+ - llama 3.1
35
+ - context 128k
36
+ - mergekit
37
+ - merge
38
+ - not-for-all-audiences
39
+ ---
40
+ ## About
41
+
42
  <!-- ### quantize_version: 2 -->
43
  <!-- ### output_tensor_quantised: 1 -->
44
  <!-- ### convert_type: hf -->
45
  <!-- ### vocab_type: -->
46
  <!-- ### tags: nicoboss -->
47
  weighted/imatrix quants of https://huggingface.co/DavidAU/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B
48
+
49
+ <!-- provided-files -->
50
+ static quants are available at https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-GGUF
51
+ ## Usage
52
+
53
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
54
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
55
+ more details, including on how to concatenate multi-part files.
56
+
57
+ ## Provided Quants
58
+
59
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
60
+
61
+ | Link | Type | Size/GB | Notes |
62
+ |:-----|:-----|--------:|:------|
63
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
64
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
65
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
66
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
67
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
68
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
69
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.1 | very low quality |
70
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
71
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
72
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
73
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
74
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
75
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
76
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
77
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
78
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
79
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
80
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.8 | prefer IQ4_XS |
81
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
82
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
83
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q4_1.gguf) | i1-Q4_1 | 5.2 | |
84
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
85
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
86
+ | [GGUF](https://huggingface.co/mradermacher/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B-i1-GGUF/resolve/main/L3.1-Dark-Reasoning-Dark-Planet-Hermes-R1-Uncensored-8B.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
87
+
88
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
89
+ types (lower is better):
90
+
91
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
92
+
93
+ And here are Artefact2's thoughts on the matter:
94
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
95
+
96
+ ## FAQ / Model Request
97
+
98
+ See https://huggingface.co/mradermacher/model_requests for some answers to
99
+ questions you might have and/or if you want some other model quantized.
100
+
101
+ ## Thanks
102
+
103
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
104
+ me use its servers and providing upgrades to my workstation to enable
105
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
106
+
107
+ <!-- end -->