Hugging Face's logo

ubergarm
/
DeepSeek-V3-0324-GGUF

Text Generation
GGUF
mla
imatrix
deepseek_v3
conversational
ik_llama.cpp
Model card Files Community
3
ubergarm commited on
Commit
253ffa1
·
1 Parent(s): 52485c7

Add mradermacher gguf data Thanks crew!

Browse files
Files changed (1) hide show
  1. README.md +43 -43
README.md CHANGED
@@ -125,49 +125,49 @@ compare exact recipe. Working with them to get info on their split GGUFs.
125
 
126
  | | [ubergarm/DeepSeek-V3-0324-IQ2_K_R4](https://huggingface.co/ubergarm/DeepSeek-V3-0324-GGUF?show_file_info=DeepSeek-V3-0324-IQ2_K_R4%2FDeepSeek-V3-0324-IQ2_K_R4-00001-of-00005.gguf) | [bartowski/DeepSeek-V3-0324-Q2_K_L](https://huggingface.co/bartowski/deepseek-ai_DeepSeek-V3-0324-GGUF?show_file_info=deepseek-ai_DeepSeek-V3-0324-Q2_K_L%2Fdeepseek-ai_DeepSeek-V3-0324-Q2_K_L-00001-of-00007.gguf) | [unsloth/DeepSeek-V3-0324-UD-Q2_K_XL](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF?show_file_info=UD-Q2_K_XL%2FDeepSeek-V3-0324-UD-Q2_K_XL-00001-of-00006.gguf) | [mradermacher/DeepSeek-V3-0324-i1-GGUF-Q2_K](https://huggingface.co/mradermacher/DeepSeek-V3-0324-i1-GGUF) |
127
  | --- | --- | --- | --- | --- |
128
- | **Overview** | | | | |
129
- | `split.tensors.count` | 1147 | 1025 | 1025 | |
130
- | `token_embd.weight` | `Q8_0` | `Q8_0` | `Q4_K` | |
131
- | File Size (GiB) | 227 | 228 | 231 | |
132
- | **Multi-Head Latent Attention** | | | | |
133
- | `blk.*.attn_kv_b.weight` | `Q8_0` | n/a | n/a | n/a |
134
- | `blk.*.attn_k_b.weight` | `Q8_0` | n/a | n/a | n/a |
135
- | `blk.*.attn_v_b.weight` | `Q8_0` | n/a | n/a | n/a |
136
- | **Dense Layers** | | | | |
137
- | `blk.[0-2].attn_kv_a_mqa.weight` | `Q8_0` | `Q2_K` | `Q6_K` | |
138
- | `blk.[0-2].attn_kv_a_norm.weight` | `F32` | `F32` | `F32` | |
139
- | `blk.[0-2].attn_kv_b.weight` | `Q8_0` | `Q2_K` | `Q6_K` | |
140
- | `blk.[0-2].attn_norm.weight` | `F32` | `F32` | `F32` | |
141
- | `blk.[0-2].attn_q_a.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
142
- | `blk.[0-2].attn_q_a_norm.weight` | `F32` | `F32` | `F32` | |
143
- | `blk.[0-2].attn_q_b.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
144
- | `blk.[0-2].ffn_down.weight` | `Q8_0` | `Q3_K` | `Q6_K` | |
145
- | `blk.[0-2].ffn_gate.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
146
- | `blk.[0-2].ffn_norm.weight` | `F32` | `F32` | `F32` | |
147
- | `blk.[0-2].ffn_up.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
148
- | `blk.[0-2].attn_output.weight` | `Q8_0` | `Q3_K` | `Q4_K` | |
149
- | **Shared & Routed MoE Layers** | | | | |
150
- | `blk.[3-60].attn_kv_a_mqa.weight` | `Q8_0` | `Q2_K` | `Q6_K` | |
151
- | `blk.[3-60].attn_kv_a_norm.weight` | `F32` | `F32` | `F32` | |
152
- | `blk.[3-60].attn_kv_b.weight` | `Q8_0` | `Q2_K` | `Q6_K` | |
153
- | `blk.[3-60].attn_norm.weight` | `F32` | `F32` | `F32` | |
154
- | `blk.[3-60].attn_q_a.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
155
- | `blk.[3-60].attn_q_a_norm.weight` | `F32` | `F32` | `F32` | |
156
- | `blk.[3-60].attn_q_b.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
157
- | `blk.[3-60].exp_probs_b.bias` | `F32` | `F32` | `F32` | |
158
- | `blk.[3-60].ffn_down_exps.weight` | `IQ3_K_R4` | `Q3_K` | `Q3_K` | |
159
- | `blk.[3-60].ffn_down_shexp.weight` | `Q8_0` | `Q3_K` | `Q6_K` | |
160
- | `blk.[3-60].ffn_gate_exps.weight` | `IQ2_K_R4` | `Q2_K` | `Q2_K` | |
161
- | `blk.[3-60].ffn_gate_inp.weight` | `F32` | `F32` | `F32` | |
162
- | `blk.[3-60].ffn_gate_shexp.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
163
- | `blk.[3-60].ffn_norm.weight` | `F32` | `F32` | `F32` | |
164
- | `blk.[3-60].ffn_up_exps.weight` | `IQ2_K_R4` | `Q2_K` | `Q2_K` | |
165
- | `blk.[3-60].ffn_up_shexp.weight` | `Q8_0` | `Q2_K` | `Q4_K` | |
166
- | `blk.[3-60].attn_output.weight` | `Q8_0` | `Q3_K` | `Q4_K` | |
167
- | **Important Matrix & Perplexity** | | | | |
168
- | `imatrix.dataset` | `calibration_data_v5_rc.txt`| `calibration_datav3.txt` | n/a | ? |
169
- | Final PPL (wiki.test.raw) | 3.5614 +/- 0.02001 | ? | ? | ? |
170
-
171
 
172
  </details>
173
 
 
125
 
126
  | | [ubergarm/DeepSeek-V3-0324-IQ2_K_R4](https://huggingface.co/ubergarm/DeepSeek-V3-0324-GGUF?show_file_info=DeepSeek-V3-0324-IQ2_K_R4%2FDeepSeek-V3-0324-IQ2_K_R4-00001-of-00005.gguf) | [bartowski/DeepSeek-V3-0324-Q2_K_L](https://huggingface.co/bartowski/deepseek-ai_DeepSeek-V3-0324-GGUF?show_file_info=deepseek-ai_DeepSeek-V3-0324-Q2_K_L%2Fdeepseek-ai_DeepSeek-V3-0324-Q2_K_L-00001-of-00007.gguf) | [unsloth/DeepSeek-V3-0324-UD-Q2_K_XL](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF?show_file_info=UD-Q2_K_XL%2FDeepSeek-V3-0324-UD-Q2_K_XL-00001-of-00006.gguf) | [mradermacher/DeepSeek-V3-0324-i1-GGUF-Q2_K](https://huggingface.co/mradermacher/DeepSeek-V3-0324-i1-GGUF) |
127
  | --- | --- | --- | --- | --- |
128
+ | **Overview** | | | | |
129
+ | `split.tensors.count` | 1147 | 1025 | 1025 | |
130
+ | `token_embd.weight` | `Q8_0` | `Q8_0` | `Q4_K` | `IQ3_S` |
131
+ | `output.weight` | | | | `Q5_K` |
132
+ | File Size (GiB) | 227 | 228 | 231 | |
133
+ | **Multi-Head Latent Attention** | | | | |
134
+ | `blk.*.attn_kv_b.weight` | `Q8_0` | n/a | n/a | n/a |
135
+ | `blk.*.attn_k_b.weight` | `Q8_0` | n/a | n/a | n/a |
136
+ | `blk.*.attn_v_b.weight` | `Q8_0` | n/a | n/a | n/a |
137
+ | **Dense Layers** | | | | |
138
+ | `blk.[0-2].attn_kv_a_mqa.weight` | `Q8_0` | `Q2_K` | `Q6_K` | `IQ2_XS`|
139
+ | `blk.[0-2].attn_kv_a_norm.weight` | `F32` | `F32` | `F32` | `F32` |
140
+ | `blk.[0-2].attn_kv_b.weight` | `Q8_0` | `Q2_K` | `Q6_K` | `IQ2_XS`|
141
+ | `blk.[0-2].attn_norm.weight` | `F32` | `F32` | `F32` | `F32` |
142
+ | `blk.[0-2].attn_q_a.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
143
+ | `blk.[0-2].attn_q_a_norm.weight` | `F32` | `F32` | `F32` | `F32` |
144
+ | `blk.[0-2].attn_q_b.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
145
+ | `blk.[0-2].ffn_down.weight` | `Q8_0` | `Q3_K` | `Q6_K` | `IQ3_S` |
146
+ | `blk.[0-2].ffn_gate.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
147
+ | `blk.[0-2].ffn_norm.weight` | `F32` | `F32` | `F32` | `F32` |
148
+ | `blk.[0-2].ffn_up.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
149
+ | `blk.[0-2].attn_output.weight` | `Q8_0` | `Q3_K` | `Q4_K` | `IQ3_S` |
150
+ | **Shared & Routed MoE Layers** | | | | |
151
+ | `blk.[3-60].attn_kv_a_mqa.weight` | `Q8_0` | `Q2_K` | `Q6_K` | `IQ2_XS`|
152
+ | `blk.[3-60].attn_kv_a_norm.weight` | `F32` | `F32` | `F32` | `F32` |
153
+ | `blk.[3-60].attn_kv_b.weight` | `Q8_0` | `Q2_K` | `Q6_K` | `IQ2_XS`|
154
+ | `blk.[3-60].attn_norm.weight` | `F32` | `F32` | `F32` | `F32` |
155
+ | `blk.[3-60].attn_q_a.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
156
+ | `blk.[3-60].attn_q_a_norm.weight` | `F32` | `F32` | `F32` | `F32` |
157
+ | `blk.[3-60].attn_q_b.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
158
+ | `blk.[3-60].exp_probs_b.bias` | `F32` | `F32` | `F32` | `F32` |
159
+ | `blk.[3-60].ffn_down_exps.weight` | `IQ3_K_R4` | `Q3_K` | `Q3_K` | `IQ3_S` |
160
+ | `blk.[3-60].ffn_down_shexp.weight` | `Q8_0` | `Q3_K` | `Q6_K` | `IQ3_S` |
161
+ | `blk.[3-60].ffn_gate_exps.weight` | `IQ2_K_R4` | `Q2_K` | `Q2_K` | `IQ2_XS`|
162
+ | `blk.[3-60].ffn_gate_inp.weight` | `F32` | `F32` | `F32` | `F32` |
163
+ | `blk.[3-60].ffn_gate_shexp.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
164
+ | `blk.[3-60].ffn_norm.weight` | `F32` | `F32` | `F32` | `F32` |
165
+ | `blk.[3-60].ffn_up_exps.weight` | `IQ2_K_R4` | `Q2_K` | `Q2_K` | `IQ2_XS`|
166
+ | `blk.[3-60].ffn_up_shexp.weight` | `Q8_0` | `Q2_K` | `Q4_K` | `IQ2_XS`|
167
+ | `blk.[3-60].attn_output.weight` | `Q8_0` | `Q3_K` | `Q4_K` | `IQ3_S` |
168
+ | **Important Matrix & Perplexity** | | | | |
169
+ | `imatrix.dataset` | `calibration_data_v5_rc.txt`| `calibration_datav3.txt` | `imatrix-training-full-3` | ? |
170
+ | Final PPL (wiki.test.raw) | 3.5614 +/- 0.02001 | ? | ? | ? |
171
 
172
  </details>
173