Hugging Face's logo

Loom-Labs
/
Apollo-1-4B-GGUF

Transformers
GGUF
text-generation-inference
unsloth
qwen3
conversational
Model card Files
xet
Community
Spestly mradermacher commited on
Commit
9b11188
·
verified ·
0 Parent(s):

Duplicate from mradermacher/Nous-V1-4B-GGUF

Browse files

Co-authored-by: team mradermacher <[email protected]>

Files changed (14) hide show
  1. .gitattributes +47 -0
  2. Nous-V1-4B.IQ4_XS.gguf +3 -0
  3. Nous-V1-4B.Q2_K.gguf +3 -0
  4. Nous-V1-4B.Q3_K_L.gguf +3 -0
  5. Nous-V1-4B.Q3_K_M.gguf +3 -0
  6. Nous-V1-4B.Q3_K_S.gguf +3 -0
  7. Nous-V1-4B.Q4_K_M.gguf +3 -0
  8. Nous-V1-4B.Q4_K_S.gguf +3 -0
  9. Nous-V1-4B.Q5_K_M.gguf +3 -0
  10. Nous-V1-4B.Q5_K_S.gguf +3 -0
  11. Nous-V1-4B.Q6_K.gguf +3 -0
  12. Nous-V1-4B.Q8_0.gguf +3 -0
  13. Nous-V1-4B.f16.gguf +3 -0
  14. README.md +185 -0
.gitattributes ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Nous-V1-4B.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Nous-V1-4B.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Nous-V1-4B.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Nous-V1-4B.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Nous-V1-4B.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Nous-V1-4B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Nous-V1-4B.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Nous-V1-4B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Nous-V1-4B.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Nous-V1-4B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Nous-V1-4B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Nous-V1-4B.f16.gguf filter=lfs diff=lfs merge=lfs -text
Nous-V1-4B.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7287fa9c698265cf165250c687dcd4b60287a30453a0fa444ff14ec33809a8d3
3
+ size 2286317216
Nous-V1-4B.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da88d095ef6276268889bf36ce13a32594cf510bbc591c6b20daf8cfe2743565
3
+ size 1669500576
Nous-V1-4B.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ead50152db80acbcc46d59f1cf4dcf772bbae425ff928eca6a8cf11c932f58bb
3
+ size 2239786656
Nous-V1-4B.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:483cb842c5637c3250c22684120748039c6642d8f710f9fbac0d312dc734f29b
3
+ size 2075618976
Nous-V1-4B.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d39724dbaabf957b6af48c618cd336ddde0a3764da97b218e735383c6c01030
3
+ size 1886998176
Nous-V1-4B.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:857cd05d57be578586bb840c4f9ad47d3f9d0d968bf09e3009288d44b521f944
3
+ size 2497281696
Nous-V1-4B.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa2dd008c2da40a2b8c3009e4ae6d325fb1e6a503c0c1b29883c5e0899ab1fd1
3
+ size 2383310496
Nous-V1-4B.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8d16d07ec6aed14743f351362cb6d6c75a6598ea1b2c0cbb56109aa210f20b0
3
+ size 2889514656
Nous-V1-4B.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:df9b601aae5b1187fe9126e38b75210ccdd060e28a9e77ba8b1533f4a329eeb6
3
+ size 2823712416
Nous-V1-4B.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2dd281334ee965394e619f6b4653322cb98a4a18c6fd75f46a96e5eaf88fb215
3
+ size 3306262176
Nous-V1-4B.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f5422d81c6b9bf32449c9d7cac8a24f6cc7bf933e1d4c0e23f2954b0981c400
3
+ size 4280406176
Nous-V1-4B.f16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:daaf2ef4b9d2d6f3c27fbf28883c58114182e3f29f1fc2cc53482f618fbe4e79
3
+ size 8051286176
README.md ADDED
@@ -0,0 +1,185 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: apexion-ai/Nous-V1-4B
3
+ language:
4
+ - en
5
+ - fr
6
+ - pt
7
+ - de
8
+ - ro
9
+ - sv
10
+ - da
11
+ - bg
12
+ - ru
13
+ - cs
14
+ - el
15
+ - uk
16
+ - es
17
+ - nl
18
+ - sk
19
+ - hr
20
+ - pl
21
+ - lt
22
+ - nb
23
+ - nn
24
+ - fa
25
+ - sl
26
+ - gu
27
+ - lv
28
+ - it
29
+ - oc
30
+ - ne
31
+ - mr
32
+ - be
33
+ - sr
34
+ - lb
35
+ - vec
36
+ - as
37
+ - cy
38
+ - szl
39
+ - ast
40
+ - hne
41
+ - awa
42
+ - mai
43
+ - bho
44
+ - sd
45
+ - ga
46
+ - fo
47
+ - hi
48
+ - pa
49
+ - bn
50
+ - or
51
+ - tg
52
+ - yi
53
+ - lmo
54
+ - lij
55
+ - scn
56
+ - fur
57
+ - sc
58
+ - gl
59
+ - ca
60
+ - is
61
+ - sq
62
+ - li
63
+ - prs
64
+ - af
65
+ - mk
66
+ - si
67
+ - ur
68
+ - mag
69
+ - bs
70
+ - hy
71
+ - zh
72
+ - yue
73
+ - my
74
+ - ar
75
+ - he
76
+ - mt
77
+ - id
78
+ - ms
79
+ - tl
80
+ - ceb
81
+ - jv
82
+ - su
83
+ - min
84
+ - ban
85
+ - pag
86
+ - ilo
87
+ - war
88
+ - ta
89
+ - te
90
+ - kn
91
+ - ml
92
+ - tr
93
+ - az
94
+ - uz
95
+ - kk
96
+ - ba
97
+ - tt
98
+ - th
99
+ - lo
100
+ - fi
101
+ - et
102
+ - hu
103
+ - vi
104
+ - km
105
+ - ja
106
+ - ko
107
+ - ka
108
+ - eu
109
+ - ht
110
+ - pap
111
+ - kea
112
+ - tpi
113
+ - sw
114
+ library_name: transformers
115
+ license: other
116
+ license_link: https://huggingface.co/apexion-ai/Nous-V1-8B/blob/main/LICENSE.md
117
+ license_name: anvdl-1.0
118
+ mradermacher:
119
+ readme_rev: 1
120
+ quantized_by: mradermacher
121
+ tags:
122
+ - text-generation-inference
123
+ - transformers
124
+ - unsloth
125
+ - qwen3
126
+ ---
127
+ ## About
128
+
129
+ <!-- ### quantize_version: 2 -->
130
+ <!-- ### output_tensor_quantised: 1 -->
131
+ <!-- ### convert_type: hf -->
132
+ <!-- ### vocab_type: -->
133
+ <!-- ### tags: -->
134
+ static quants of https://huggingface.co/apexion-ai/Nous-V1-4B
135
+
136
+ <!-- provided-files -->
137
+
138
+ ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Nous-V1-4B-GGUF).***
139
+
140
+ weighted/imatrix quants are available at https://huggingface.co/mradermacher/Nous-V1-4B-i1-GGUF
141
+ ## Usage
142
+
143
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
144
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
145
+ more details, including on how to concatenate multi-part files.
146
+
147
+ ## Provided Quants
148
+
149
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
150
+
151
+ | Link | Type | Size/GB | Notes |
152
+ |:-----|:-----|--------:|:------|
153
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q2_K.gguf) | Q2_K | 1.8 | |
154
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q3_K_S.gguf) | Q3_K_S | 2.0 | |
155
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q3_K_M.gguf) | Q3_K_M | 2.2 | lower quality |
156
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q3_K_L.gguf) | Q3_K_L | 2.3 | |
157
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.IQ4_XS.gguf) | IQ4_XS | 2.4 | |
158
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q4_K_S.gguf) | Q4_K_S | 2.5 | fast, recommended |
159
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q4_K_M.gguf) | Q4_K_M | 2.6 | fast, recommended |
160
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q5_K_S.gguf) | Q5_K_S | 2.9 | |
161
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q5_K_M.gguf) | Q5_K_M | 3.0 | |
162
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q6_K.gguf) | Q6_K | 3.4 | very good quality |
163
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.Q8_0.gguf) | Q8_0 | 4.4 | fast, best quality |
164
+ | [GGUF](https://huggingface.co/mradermacher/Nous-V1-4B-GGUF/resolve/main/Nous-V1-4B.f16.gguf) | f16 | 8.2 | 16 bpw, overkill |
165
+
166
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
167
+ types (lower is better):
168
+
169
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
170
+
171
+ And here are Artefact2's thoughts on the matter:
172
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
173
+
174
+ ## FAQ / Model Request
175
+
176
+ See https://huggingface.co/mradermacher/model_requests for some answers to
177
+ questions you might have and/or if you want some other model quantized.
178
+
179
+ ## Thanks
180
+
181
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
182
+ me use its servers and providing upgrades to my workstation to enable
183
+ this work in my free time.
184
+
185
+ <!-- end -->