Hugging Face's logo

mradermacher
/
Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF

Transformers
GGUF
English
llama-3
llama3
llama
bfloat16
uncensored
all use cases
creative use cases
creative
creative writing
fiction writing
plot generation
sub-plot generation
story generation
scene continue
storytelling
fiction story
science fiction
romance
all genres
story
writing
vivid prose
vivid writing
fiction
roleplaying
swearing
rp
GGUFs
Merge
imatrix
conversational
Model card Files
xet
Community
mradermacher commited on
Commit
7278292
·
verified ·
1 Parent(s): 30332a2

auto-patch README.md

Browse files
Files changed (1) hide show
  1. README.md +108 -0
README.md CHANGED
@@ -1,3 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <!-- ### quantize_version: 2 -->
2
  <!-- ### output_tensor_quantised: 1 -->
3
  <!-- ### convert_type: hf -->
@@ -7,3 +51,67 @@
7
  <!-- ### quants_skip: -->
8
  <!-- ### skip_mmproj: -->
9
  weighted/imatrix quants of https://huggingface.co/DavidAU/Llama-3.2-9B-Uncensored-Brainstorm-Alpha
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: DavidAU/Llama-3.2-9B-Uncensored-Brainstorm-Alpha
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ license: apache-2.0
7
+ mradermacher:
8
+ readme_rev: 1
9
+ quantized_by: mradermacher
10
+ tags:
11
+ - llama-3
12
+ - llama3
13
+ - llama
14
+ - bfloat16
15
+ - uncensored
16
+ - all use cases
17
+ - creative use cases
18
+ - creative
19
+ - creative writing
20
+ - fiction writing
21
+ - plot generation
22
+ - sub-plot generation
23
+ - fiction writing
24
+ - story generation
25
+ - scene continue
26
+ - storytelling
27
+ - fiction story
28
+ - science fiction
29
+ - romance
30
+ - all genres
31
+ - story
32
+ - writing
33
+ - vivid prose
34
+ - vivid writing
35
+ - fiction
36
+ - roleplaying
37
+ - bfloat16
38
+ - swearing
39
+ - rp
40
+ - GGUFs
41
+ - merge
42
+ ---
43
+ ## About
44
+
45
  <!-- ### quantize_version: 2 -->
46
  <!-- ### output_tensor_quantised: 1 -->
47
  <!-- ### convert_type: hf -->
 
51
  <!-- ### quants_skip: -->
52
  <!-- ### skip_mmproj: -->
53
  weighted/imatrix quants of https://huggingface.co/DavidAU/Llama-3.2-9B-Uncensored-Brainstorm-Alpha
54
+
55
+ <!-- provided-files -->
56
+
57
+ ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF).***
58
+
59
+ static quants are available at https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-GGUF
60
+ ## Usage
61
+
62
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
63
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
64
+ more details, including on how to concatenate multi-part files.
65
+
66
+ ## Provided Quants
67
+
68
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
69
+
70
+ | Link | Type | Size/GB | Notes |
71
+ |:-----|:-----|--------:|:------|
72
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own qwuants) |
73
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ1_S.gguf) | i1-IQ1_S | 2.2 | for the desperate |
74
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ1_M.gguf) | i1-IQ1_M | 2.4 | mostly desperate |
75
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.6 | |
76
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.9 | |
77
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ2_S.gguf) | i1-IQ2_S | 3.1 | |
78
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.3 | very low quality |
79
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ2_M.gguf) | i1-IQ2_M | 3.3 | |
80
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q2_K.gguf) | i1-Q2_K | 3.5 | IQ3_XXS probably better |
81
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.6 | lower quality |
82
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.9 | |
83
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ3_S.gguf) | i1-IQ3_S | 4.1 | beats Q3_K* |
84
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.1 | IQ3_XS probably better |
85
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ3_M.gguf) | i1-IQ3_M | 4.3 | |
86
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.5 | IQ3_S probably better |
87
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.9 | IQ3_M probably better |
88
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ4_XS.gguf) | i1-IQ4_XS | 5.0 | |
89
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-IQ4_NL.gguf) | i1-IQ4_NL | 5.2 | prefer IQ4_XS |
90
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q4_0.gguf) | i1-Q4_0 | 5.3 | fast, low quality |
91
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q4_K_S.gguf) | i1-Q4_K_S | 5.3 | optimal size/speed/quality |
92
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.5 | fast, recommended |
93
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q4_1.gguf) | i1-Q4_1 | 5.8 | |
94
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q5_K_S.gguf) | i1-Q5_K_S | 6.3 | |
95
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q5_K_M.gguf) | i1-Q5_K_M | 6.5 | |
96
+ | [GGUF](https://huggingface.co/mradermacher/Llama-3.2-9B-Uncensored-Brainstorm-Alpha-i1-GGUF/resolve/main/Llama-3.2-9B-Uncensored-Brainstorm-Alpha.i1-Q6_K.gguf) | i1-Q6_K | 7.4 | practically like static Q6_K |
97
+
98
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
99
+ types (lower is better):
100
+
101
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
102
+
103
+ And here are Artefact2's thoughts on the matter:
104
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
105
+
106
+ ## FAQ / Model Request
107
+
108
+ See https://huggingface.co/mradermacher/model_requests for some answers to
109
+ questions you might have and/or if you want some other model quantized.
110
+
111
+ ## Thanks
112
+
113
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
114
+ me use its servers and providing upgrades to my workstation to enable
115
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
116
+
117
+ <!-- end -->