Hugging Face's logo

eaddario
/
DeepSeek-R1-Distill-Llama-8B-GGUF

Text Generation
GGUF
English
quant
experimental
conversational
Model card Files
xet
Community
DeepSeek-R1-Distill-Llama-8B-GGUF / scores /DeepSeek-R1-Distill-Llama-8B-Q5_K_S.md
eaddario's picture
eaddario
Add GGUF internal file structure
66279ab verified
preview code
|
raw
history blame contribute delete
83.1 kB

DeepSeek-R1-Distill-Llama-8B-Q5_K_S.gguf - GGUF Internal File Dump

  • Endian: LITTLE endian

Key Value Metadata Store

There are 36 key-value pairs in this file

POS TYPE Count Key Value
1 UINT32 1 GGUF.version 3
2 UINT64 1 GGUF.tensor_count 292
3 UINT64 1 GGUF.kv_count 33
4 STRING 1 general.architecture llama
5 STRING 1 general.type model
6 STRING 1 general.name DeepSeek R1 Distill Llama 8B
7 STRING 1 general.basename DeepSeek-R1-Distill-Llama
8 STRING 1 general.size_label 8B
9 STRING 1 general.license mit
10 UINT32 1 llama.block_count 32
11 UINT32 1 llama.context_length 131072
12 UINT32 1 llama.embedding_length 4096
13 UINT32 1 llama.feed_forward_length 14336
14 UINT32 1 llama.attention.head_count 32
15 UINT32 1 llama.attention.head_count_kv 8
16 FLOAT32 1 llama.rope.freq_base 500000.0
17 FLOAT32 1 llama.attention.layer_norm_rms_epsilon 1e-05
18 UINT32 1 llama.vocab_size 128256
19 UINT32 1 llama.rope.dimension_count 128
20 STRING 1 tokenizer.ggml.model gpt2
21 STRING 1 tokenizer.ggml.pre llama-bpe
22 [STRING] 128256 tokenizer.ggml.tokens [ !, ", #, $, %, ... ]
23 [INT32] 128256 tokenizer.ggml.token_type [ 1, 1, 1, 1, 1, 1, 1, ... ]
24 [STRING] 280147 tokenizer.ggml.merges [ Ġ Ġ, Ġ ĠĠĠ, ĠĠ ĠĠ, ĠĠĠ Ġ, i n, ... ]
25 UINT32 1 tokenizer.ggml.bos_token_id 128000
26 UINT32 1 tokenizer.ggml.eos_token_id 128001
27 UINT32 1 tokenizer.ggml.padding_token_id 128001
28 BOOL 1 tokenizer.ggml.add_bos_token True
29 BOOL 1 tokenizer.ggml.add_eos_token False
30 STRING 1 tokenizer.chat_template {% if not add_generation_promp...{{'<｜Assistant｜>'}}{% endif %}
31 UINT32 1 general.quantization_version 2
32 UINT32 1 general.file_type 16
33 STRING 1 quantize.imatrix.file ./imatrix/imatrix-DeepSeek-R1-Distill-Llama-8B-small.dat
34 STRING 1 quantize.imatrix.dataset ../../datasets/imatrix/calibration_all_small.txt
35 INT32 1 quantize.imatrix.entries_count 225
36 INT32 1 quantize.imatrix.chunks_count 1130

Tensors Overview ~8B Elements

Total number of elements in all tensors: 8030261312 Elements

Tensor Data Offset

This table contains the offset and data segment relative to start of file

T_ID Tensor Layer Name Data Offset (B) Data Size (B)
0 output.weight 0x779a80 0x15870000
1 output_norm.weight 0x15fe9a80 0x4000
2 rope_freqs.weight 0x15feda80 0x100
3 token_embd.weight 0x15fedb80 0xd746000
4 blk.0.attn_k.weight 0x23733b80 0x2c0000
5 blk.0.attn_norm.weight 0x239f3b80 0x4000
6 blk.0.attn_output.weight 0x239f7b80 0xb00000
7 blk.0.attn_q.weight 0x244f7b80 0xb00000
8 blk.0.attn_v.weight 0x24ff7b80 0x2c0000
9 blk.0.ffn_down.weight 0x252b7b80 0x2df0000
10 blk.0.ffn_gate.weight 0x280a7b80 0x2680000
11 blk.0.ffn_norm.weight 0x2a727b80 0x4000
12 blk.0.ffn_up.weight 0x2a72bb80 0x2680000
13 blk.1.attn_k.weight 0x2cdabb80 0x2c0000
14 blk.1.attn_norm.weight 0x2d06bb80 0x4000
15 blk.1.attn_output.weight 0x2d06fb80 0xb00000
16 blk.1.attn_q.weight 0x2db6fb80 0xb00000
17 blk.1.attn_v.weight 0x2e66fb80 0x2c0000
18 blk.1.ffn_down.weight 0x2e92fb80 0x2df0000
19 blk.1.ffn_gate.weight 0x3171fb80 0x2680000
20 blk.1.ffn_norm.weight 0x33d9fb80 0x4000
21 blk.1.ffn_up.weight 0x33da3b80 0x2680000
22 blk.2.attn_k.weight 0x36423b80 0x2c0000
23 blk.2.attn_norm.weight 0x366e3b80 0x4000
24 blk.2.attn_output.weight 0x366e7b80 0xb00000
25 blk.2.attn_q.weight 0x371e7b80 0xb00000
26 blk.2.attn_v.weight 0x37ce7b80 0x2c0000
27 blk.2.ffn_down.weight 0x37fa7b80 0x2df0000
28 blk.2.ffn_gate.weight 0x3ad97b80 0x2680000
29 blk.2.ffn_norm.weight 0x3d417b80 0x4000
30 blk.2.ffn_up.weight 0x3d41bb80 0x2680000
31 blk.3.attn_k.weight 0x3fa9bb80 0x2c0000
32 blk.3.attn_norm.weight 0x3fd5bb80 0x4000
33 blk.3.attn_output.weight 0x3fd5fb80 0xb00000
34 blk.3.attn_q.weight 0x4085fb80 0xb00000
35 blk.3.attn_v.weight 0x4135fb80 0x2c0000
36 blk.3.ffn_down.weight 0x4161fb80 0x2df0000
37 blk.3.ffn_gate.weight 0x4440fb80 0x2680000
38 blk.3.ffn_norm.weight 0x46a8fb80 0x4000
39 blk.3.ffn_up.weight 0x46a93b80 0x2680000
40 blk.4.attn_k.weight 0x49113b80 0x2c0000
41 blk.4.attn_norm.weight 0x493d3b80 0x4000
42 blk.4.attn_output.weight 0x493d7b80 0xb00000
43 blk.4.attn_q.weight 0x49ed7b80 0xb00000
44 blk.4.attn_v.weight 0x4a9d7b80 0x2c0000
45 blk.4.ffn_down.weight 0x4ac97b80 0x2df0000
46 blk.4.ffn_gate.weight 0x4da87b80 0x2680000
47 blk.4.ffn_norm.weight 0x50107b80 0x4000
48 blk.4.ffn_up.weight 0x5010bb80 0x2680000
49 blk.5.attn_k.weight 0x5278bb80 0x2c0000
50 blk.5.attn_norm.weight 0x52a4bb80 0x4000
51 blk.5.attn_output.weight 0x52a4fb80 0xb00000
52 blk.5.attn_q.weight 0x5354fb80 0xb00000
53 blk.5.attn_v.weight 0x5404fb80 0x2c0000
54 blk.5.ffn_down.weight 0x5430fb80 0x2df0000
55 blk.5.ffn_gate.weight 0x570ffb80 0x2680000
56 blk.5.ffn_norm.weight 0x5977fb80 0x4000
57 blk.5.ffn_up.weight 0x59783b80 0x2680000
58 blk.6.attn_k.weight 0x5be03b80 0x2c0000
59 blk.6.attn_norm.weight 0x5c0c3b80 0x4000
60 blk.6.attn_output.weight 0x5c0c7b80 0xb00000
61 blk.6.attn_q.weight 0x5cbc7b80 0xb00000
62 blk.6.attn_v.weight 0x5d6c7b80 0x2c0000
63 blk.6.ffn_down.weight 0x5d987b80 0x2df0000
64 blk.6.ffn_gate.weight 0x60777b80 0x2680000
65 blk.6.ffn_norm.weight 0x62df7b80 0x4000
66 blk.6.ffn_up.weight 0x62dfbb80 0x2680000
67 blk.7.attn_k.weight 0x6547bb80 0x2c0000
68 blk.7.attn_norm.weight 0x6573bb80 0x4000
69 blk.7.attn_output.weight 0x6573fb80 0xb00000
70 blk.7.attn_q.weight 0x6623fb80 0xb00000
71 blk.7.attn_v.weight 0x66d3fb80 0x2c0000
72 blk.7.ffn_down.weight 0x66fffb80 0x2df0000
73 blk.7.ffn_gate.weight 0x69defb80 0x2680000
74 blk.7.ffn_norm.weight 0x6c46fb80 0x4000
75 blk.7.ffn_up.weight 0x6c473b80 0x2680000
76 blk.8.attn_k.weight 0x6eaf3b80 0x2c0000
77 blk.8.attn_norm.weight 0x6edb3b80 0x4000
78 blk.8.attn_output.weight 0x6edb7b80 0xb00000
79 blk.8.attn_q.weight 0x6f8b7b80 0xb00000
80 blk.8.attn_v.weight 0x703b7b80 0x2c0000
81 blk.8.ffn_down.weight 0x70677b80 0x2df0000
82 blk.8.ffn_gate.weight 0x73467b80 0x2680000
83 blk.8.ffn_norm.weight 0x75ae7b80 0x4000
84 blk.8.ffn_up.weight 0x75aebb80 0x2680000
85 blk.9.attn_k.weight 0x7816bb80 0x2c0000
86 blk.9.attn_norm.weight 0x7842bb80 0x4000
87 blk.9.attn_output.weight 0x7842fb80 0xb00000
88 blk.9.attn_q.weight 0x78f2fb80 0xb00000
89 blk.9.attn_v.weight 0x79a2fb80 0x2c0000
90 blk.9.ffn_down.weight 0x79cefb80 0x2df0000
91 blk.9.ffn_gate.weight 0x7cadfb80 0x2680000
92 blk.9.ffn_norm.weight 0x7f15fb80 0x4000
93 blk.9.ffn_up.weight 0x7f163b80 0x2680000
94 blk.10.attn_k.weight 0x817e3b80 0x2c0000
95 blk.10.attn_norm.weight 0x81aa3b80 0x4000
96 blk.10.attn_output.weight 0x81aa7b80 0xb00000
97 blk.10.attn_q.weight 0x825a7b80 0xb00000
98 blk.10.attn_v.weight 0x830a7b80 0x2c0000
99 blk.10.ffn_down.weight 0x83367b80 0x2df0000
100 blk.10.ffn_gate.weight 0x86157b80 0x2680000
101 blk.10.ffn_norm.weight 0x887d7b80 0x4000
102 blk.10.ffn_up.weight 0x887dbb80 0x2680000
103 blk.11.attn_k.weight 0x8ae5bb80 0x2c0000
104 blk.11.attn_norm.weight 0x8b11bb80 0x4000
105 blk.11.attn_output.weight 0x8b11fb80 0xb00000
106 blk.11.attn_q.weight 0x8bc1fb80 0xb00000
107 blk.11.attn_v.weight 0x8c71fb80 0x2c0000
108 blk.11.ffn_down.weight 0x8c9dfb80 0x2df0000
109 blk.11.ffn_gate.weight 0x8f7cfb80 0x2680000
110 blk.11.ffn_norm.weight 0x91e4fb80 0x4000
111 blk.11.ffn_up.weight 0x91e53b80 0x2680000
112 blk.12.attn_k.weight 0x944d3b80 0x2c0000
113 blk.12.attn_norm.weight 0x94793b80 0x4000
114 blk.12.attn_output.weight 0x94797b80 0xb00000
115 blk.12.attn_q.weight 0x95297b80 0xb00000
116 blk.12.attn_v.weight 0x95d97b80 0x2c0000
117 blk.12.ffn_down.weight 0x96057b80 0x2df0000
118 blk.12.ffn_gate.weight 0x98e47b80 0x2680000
119 blk.12.ffn_norm.weight 0x9b4c7b80 0x4000
120 blk.12.ffn_up.weight 0x9b4cbb80 0x2680000
121 blk.13.attn_k.weight 0x9db4bb80 0x2c0000
122 blk.13.attn_norm.weight 0x9de0bb80 0x4000
123 blk.13.attn_output.weight 0x9de0fb80 0xb00000
124 blk.13.attn_q.weight 0x9e90fb80 0xb00000
125 blk.13.attn_v.weight 0x9f40fb80 0x2c0000
126 blk.13.ffn_down.weight 0x9f6cfb80 0x2df0000
127 blk.13.ffn_gate.weight 0xa24bfb80 0x2680000
128 blk.13.ffn_norm.weight 0xa4b3fb80 0x4000
129 blk.13.ffn_up.weight 0xa4b43b80 0x2680000
130 blk.14.attn_k.weight 0xa71c3b80 0x2c0000
131 blk.14.attn_norm.weight 0xa7483b80 0x4000
132 blk.14.attn_output.weight 0xa7487b80 0xb00000
133 blk.14.attn_q.weight 0xa7f87b80 0xb00000
134 blk.14.attn_v.weight 0xa8a87b80 0x2c0000
135 blk.14.ffn_down.weight 0xa8d47b80 0x2df0000
136 blk.14.ffn_gate.weight 0xabb37b80 0x2680000
137 blk.14.ffn_norm.weight 0xae1b7b80 0x4000
138 blk.14.ffn_up.weight 0xae1bbb80 0x2680000
139 blk.15.attn_k.weight 0xb083bb80 0x2c0000
140 blk.15.attn_norm.weight 0xb0afbb80 0x4000
141 blk.15.attn_output.weight 0xb0affb80 0xb00000
142 blk.15.attn_q.weight 0xb15ffb80 0xb00000
143 blk.15.attn_v.weight 0xb20ffb80 0x2c0000
144 blk.15.ffn_down.weight 0xb23bfb80 0x2df0000
145 blk.15.ffn_gate.weight 0xb51afb80 0x2680000
146 blk.15.ffn_norm.weight 0xb782fb80 0x4000
147 blk.15.ffn_up.weight 0xb7833b80 0x2680000
148 blk.16.attn_k.weight 0xb9eb3b80 0x240000
149 blk.16.attn_norm.weight 0xba0f3b80 0x4000
150 blk.16.attn_output.weight 0xba0f7b80 0xb00000
151 blk.16.attn_q.weight 0xbabf7b80 0x900000
152 blk.16.attn_v.weight 0xbb4f7b80 0x2c0000
153 blk.16.ffn_down.weight 0xbb7b7b80 0x2680000
154 blk.16.ffn_gate.weight 0xbde37b80 0x1f80000
155 blk.16.ffn_norm.weight 0xbfdb7b80 0x4000
156 blk.16.ffn_up.weight 0xbfdbbb80 0x1f80000
157 blk.17.attn_k.weight 0xc1d3bb80 0x240000
158 blk.17.attn_norm.weight 0xc1f7bb80 0x4000
159 blk.17.attn_output.weight 0xc1f7fb80 0xb00000
160 blk.17.attn_q.weight 0xc2a7fb80 0x900000
161 blk.17.attn_v.weight 0xc337fb80 0x2c0000
162 blk.17.ffn_down.weight 0xc363fb80 0x2680000
163 blk.17.ffn_gate.weight 0xc5cbfb80 0x1f80000
164 blk.17.ffn_norm.weight 0xc7c3fb80 0x4000
165 blk.17.ffn_up.weight 0xc7c43b80 0x1f80000
166 blk.18.attn_k.weight 0xc9bc3b80 0x240000
167 blk.18.attn_norm.weight 0xc9e03b80 0x4000
168 blk.18.attn_output.weight 0xc9e07b80 0xb00000
169 blk.18.attn_q.weight 0xca907b80 0x900000
170 blk.18.attn_v.weight 0xcb207b80 0x2c0000
171 blk.18.ffn_down.weight 0xcb4c7b80 0x2680000
172 blk.18.ffn_gate.weight 0xcdb47b80 0x1f80000
173 blk.18.ffn_norm.weight 0xcfac7b80 0x4000
174 blk.18.ffn_up.weight 0xcfacbb80 0x1f80000
175 blk.19.attn_k.weight 0xd1a4bb80 0x240000
176 blk.19.attn_norm.weight 0xd1c8bb80 0x4000
177 blk.19.attn_output.weight 0xd1c8fb80 0xb00000
178 blk.19.attn_q.weight 0xd278fb80 0x900000
179 blk.19.attn_v.weight 0xd308fb80 0x2c0000
180 blk.19.ffn_down.weight 0xd334fb80 0x2680000
181 blk.19.ffn_gate.weight 0xd59cfb80 0x1f80000
182 blk.19.ffn_norm.weight 0xd794fb80 0x4000
183 blk.19.ffn_up.weight 0xd7953b80 0x1f80000
184 blk.20.attn_k.weight 0xd98d3b80 0x240000
185 blk.20.attn_norm.weight 0xd9b13b80 0x4000
186 blk.20.attn_output.weight 0xd9b17b80 0xb00000
187 blk.20.attn_q.weight 0xda617b80 0x900000
188 blk.20.attn_v.weight 0xdaf17b80 0x2c0000
189 blk.20.ffn_down.weight 0xdb1d7b80 0x2680000
190 blk.20.ffn_gate.weight 0xdd857b80 0x1f80000
191 blk.20.ffn_norm.weight 0xdf7d7b80 0x4000
192 blk.20.ffn_up.weight 0xdf7dbb80 0x1f80000
193 blk.21.attn_k.weight 0xe175bb80 0x240000
194 blk.21.attn_norm.weight 0xe199bb80 0x4000
195 blk.21.attn_output.weight 0xe199fb80 0xb00000
196 blk.21.attn_q.weight 0xe249fb80 0x900000
197 blk.21.attn_v.weight 0xe2d9fb80 0x2c0000
198 blk.21.ffn_down.weight 0xe305fb80 0x2680000
199 blk.21.ffn_gate.weight 0xe56dfb80 0x1f80000
200 blk.21.ffn_norm.weight 0xe765fb80 0x4000
201 blk.21.ffn_up.weight 0xe7663b80 0x1f80000
202 blk.22.attn_k.weight 0xe95e3b80 0x240000
203 blk.22.attn_norm.weight 0xe9823b80 0x4000
204 blk.22.attn_output.weight 0xe9827b80 0xb00000
205 blk.22.attn_q.weight 0xea327b80 0x900000
206 blk.22.attn_v.weight 0xeac27b80 0x2c0000
207 blk.22.ffn_down.weight 0xeaee7b80 0x2680000
208 blk.22.ffn_gate.weight 0xed567b80 0x1f80000
209 blk.22.ffn_norm.weight 0xef4e7b80 0x4000
210 blk.22.ffn_up.weight 0xef4ebb80 0x1f80000
211 blk.23.attn_k.weight 0xf146bb80 0x240000
212 blk.23.attn_norm.weight 0xf16abb80 0x4000
213 blk.23.attn_output.weight 0xf16afb80 0xb00000
214 blk.23.attn_q.weight 0xf21afb80 0x900000
215 blk.23.attn_v.weight 0xf2aafb80 0x2c0000
216 blk.23.ffn_down.weight 0xf2d6fb80 0x2680000
217 blk.23.ffn_gate.weight 0xf53efb80 0x1f80000
218 blk.23.ffn_norm.weight 0xf736fb80 0x4000
219 blk.23.ffn_up.weight 0xf7373b80 0x1f80000
220 blk.24.attn_k.weight 0xf92f3b80 0x240000
221 blk.24.attn_norm.weight 0xf9533b80 0x4000
222 blk.24.attn_output.weight 0xf9537b80 0xb00000
223 blk.24.attn_q.weight 0xfa037b80 0x900000
224 blk.24.attn_v.weight 0xfa937b80 0x2c0000
225 blk.24.ffn_down.weight 0xfabf7b80 0x2680000
226 blk.24.ffn_gate.weight 0xfd277b80 0x1f80000
227 blk.24.ffn_norm.weight 0xff1f7b80 0x4000
228 blk.24.ffn_up.weight 0xff1fbb80 0x1f80000
229 blk.25.attn_k.weight 0x10117bb80 0x240000
230 blk.25.attn_norm.weight 0x1013bbb80 0x4000
231 blk.25.attn_output.weight 0x1013bfb80 0xb00000
232 blk.25.attn_q.weight 0x101ebfb80 0x900000
233 blk.25.attn_v.weight 0x1027bfb80 0x2c0000
234 blk.25.ffn_down.weight 0x102a7fb80 0x2680000
235 blk.25.ffn_gate.weight 0x1050ffb80 0x1f80000
236 blk.25.ffn_norm.weight 0x10707fb80 0x4000
237 blk.25.ffn_up.weight 0x107083b80 0x1f80000
238 blk.26.attn_k.weight 0x109003b80 0x240000
239 blk.26.attn_norm.weight 0x109243b80 0x4000
240 blk.26.attn_output.weight 0x109247b80 0xb00000
241 blk.26.attn_q.weight 0x109d47b80 0x900000
242 blk.26.attn_v.weight 0x10a647b80 0x2c0000
243 blk.26.ffn_down.weight 0x10a907b80 0x2680000
244 blk.26.ffn_gate.weight 0x10cf87b80 0x1f80000
245 blk.26.ffn_norm.weight 0x10ef07b80 0x4000
246 blk.26.ffn_up.weight 0x10ef0bb80 0x1f80000
247 blk.27.attn_k.weight 0x110e8bb80 0x240000
248 blk.27.attn_norm.weight 0x1110cbb80 0x4000
249 blk.27.attn_output.weight 0x1110cfb80 0xb00000
250 blk.27.attn_q.weight 0x111bcfb80 0x900000
251 blk.27.attn_v.weight 0x1124cfb80 0x2c0000
252 blk.27.ffn_down.weight 0x11278fb80 0x2680000
253 blk.27.ffn_gate.weight 0x114e0fb80 0x1f80000
254 blk.27.ffn_norm.weight 0x116d8fb80 0x4000
255 blk.27.ffn_up.weight 0x116d93b80 0x1f80000
256 blk.28.attn_k.weight 0x118d13b80 0x240000
257 blk.28.attn_norm.weight 0x118f53b80 0x4000
258 blk.28.attn_output.weight 0x118f57b80 0xb00000
259 blk.28.attn_q.weight 0x119a57b80 0x900000
260 blk.28.attn_v.weight 0x11a357b80 0x2c0000
261 blk.28.ffn_down.weight 0x11a617b80 0x2680000
262 blk.28.ffn_gate.weight 0x11cc97b80 0x1f80000
263 blk.28.ffn_norm.weight 0x11ec17b80 0x4000
264 blk.28.ffn_up.weight 0x11ec1bb80 0x1f80000
265 blk.29.attn_k.weight 0x120b9bb80 0x240000
266 blk.29.attn_norm.weight 0x120ddbb80 0x4000
267 blk.29.attn_output.weight 0x120ddfb80 0xb00000
268 blk.29.attn_q.weight 0x1218dfb80 0x900000
269 blk.29.attn_v.weight 0x1221dfb80 0x2c0000
270 blk.29.ffn_down.weight 0x12249fb80 0x2680000
271 blk.29.ffn_gate.weight 0x124b1fb80 0x1f80000
272 blk.29.ffn_norm.weight 0x126a9fb80 0x4000
273 blk.29.ffn_up.weight 0x126aa3b80 0x1f80000
274 blk.30.attn_k.weight 0x128a23b80 0x240000
275 blk.30.attn_norm.weight 0x128c63b80 0x4000
276 blk.30.attn_output.weight 0x128c67b80 0xb00000
277 blk.30.attn_q.weight 0x129767b80 0x900000
278 blk.30.attn_v.weight 0x12a067b80 0x2c0000
279 blk.30.ffn_down.weight 0x12a327b80 0x2680000
280 blk.30.ffn_gate.weight 0x12c9a7b80 0x1f80000
281 blk.30.ffn_norm.weight 0x12e927b80 0x4000
282 blk.30.ffn_up.weight 0x12e92bb80 0x1f80000
283 blk.31.attn_k.weight 0x1308abb80 0x240000
284 blk.31.attn_norm.weight 0x130aebb80 0x4000
285 blk.31.attn_output.weight 0x130aefb80 0xb00000
286 blk.31.attn_q.weight 0x1315efb80 0x900000
287 blk.31.attn_v.weight 0x131eefb80 0x2c0000
288 blk.31.ffn_down.weight 0x1321afb80 0x2680000
289 blk.31.ffn_gate.weight 0x13482fb80 0x1f80000
290 blk.31.ffn_norm.weight 0x1367afb80 0x4000
291 blk.31.ffn_up.weight 0x1367b3b80 0x1f80000

Base Tensor Group : ~1B Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
0 output.weight Output (W) (~525M) 525336576 4096 x 128256 x 1 x 1 Q5_K
1 output_norm.weight Output Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
2 rope_freqs.weight Rope_Freqs (W) ( 64) 64 64 x 1 x 1 x 1 F32
3 token_embd.weight Token Embedding (W) (~525M) 525336576 4096 x 128256 x 1 x 1 Q3_K
  • Total elements in base: ( ~1B) 1050677312
  • Percentage of total elements: 13.08%

Block 0 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
4 blk.0.attn_k.weight Block 0 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
5 blk.0.attn_norm.weight Block 0 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
6 blk.0.attn_output.weight Block 0 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
7 blk.0.attn_q.weight Block 0 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
8 blk.0.attn_v.weight Block 0 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
9 blk.0.ffn_down.weight Block 0 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
10 blk.0.ffn_gate.weight Block 0 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
11 blk.0.ffn_norm.weight Block 0 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
12 blk.0.ffn_up.weight Block 0 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.0: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 1 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
13 blk.1.attn_k.weight Block 1 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
14 blk.1.attn_norm.weight Block 1 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
15 blk.1.attn_output.weight Block 1 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
16 blk.1.attn_q.weight Block 1 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
17 blk.1.attn_v.weight Block 1 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
18 blk.1.ffn_down.weight Block 1 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
19 blk.1.ffn_gate.weight Block 1 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
20 blk.1.ffn_norm.weight Block 1 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
21 blk.1.ffn_up.weight Block 1 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.1: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 2 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
22 blk.2.attn_k.weight Block 2 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
23 blk.2.attn_norm.weight Block 2 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
24 blk.2.attn_output.weight Block 2 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
25 blk.2.attn_q.weight Block 2 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
26 blk.2.attn_v.weight Block 2 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
27 blk.2.ffn_down.weight Block 2 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
28 blk.2.ffn_gate.weight Block 2 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
29 blk.2.ffn_norm.weight Block 2 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
30 blk.2.ffn_up.weight Block 2 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.2: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 3 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
31 blk.3.attn_k.weight Block 3 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
32 blk.3.attn_norm.weight Block 3 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
33 blk.3.attn_output.weight Block 3 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
34 blk.3.attn_q.weight Block 3 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
35 blk.3.attn_v.weight Block 3 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
36 blk.3.ffn_down.weight Block 3 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
37 blk.3.ffn_gate.weight Block 3 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
38 blk.3.ffn_norm.weight Block 3 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
39 blk.3.ffn_up.weight Block 3 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.3: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 4 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
40 blk.4.attn_k.weight Block 4 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
41 blk.4.attn_norm.weight Block 4 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
42 blk.4.attn_output.weight Block 4 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
43 blk.4.attn_q.weight Block 4 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
44 blk.4.attn_v.weight Block 4 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
45 blk.4.ffn_down.weight Block 4 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
46 blk.4.ffn_gate.weight Block 4 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
47 blk.4.ffn_norm.weight Block 4 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
48 blk.4.ffn_up.weight Block 4 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.4: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 5 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
49 blk.5.attn_k.weight Block 5 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
50 blk.5.attn_norm.weight Block 5 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
51 blk.5.attn_output.weight Block 5 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
52 blk.5.attn_q.weight Block 5 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
53 blk.5.attn_v.weight Block 5 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
54 blk.5.ffn_down.weight Block 5 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
55 blk.5.ffn_gate.weight Block 5 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
56 blk.5.ffn_norm.weight Block 5 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
57 blk.5.ffn_up.weight Block 5 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.5: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 6 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
58 blk.6.attn_k.weight Block 6 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
59 blk.6.attn_norm.weight Block 6 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
60 blk.6.attn_output.weight Block 6 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
61 blk.6.attn_q.weight Block 6 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
62 blk.6.attn_v.weight Block 6 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
63 blk.6.ffn_down.weight Block 6 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
64 blk.6.ffn_gate.weight Block 6 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
65 blk.6.ffn_norm.weight Block 6 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
66 blk.6.ffn_up.weight Block 6 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.6: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 7 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
67 blk.7.attn_k.weight Block 7 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
68 blk.7.attn_norm.weight Block 7 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
69 blk.7.attn_output.weight Block 7 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
70 blk.7.attn_q.weight Block 7 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
71 blk.7.attn_v.weight Block 7 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
72 blk.7.ffn_down.weight Block 7 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
73 blk.7.ffn_gate.weight Block 7 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
74 blk.7.ffn_norm.weight Block 7 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
75 blk.7.ffn_up.weight Block 7 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.7: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 8 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
76 blk.8.attn_k.weight Block 8 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
77 blk.8.attn_norm.weight Block 8 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
78 blk.8.attn_output.weight Block 8 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
79 blk.8.attn_q.weight Block 8 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
80 blk.8.attn_v.weight Block 8 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
81 blk.8.ffn_down.weight Block 8 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
82 blk.8.ffn_gate.weight Block 8 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
83 blk.8.ffn_norm.weight Block 8 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
84 blk.8.ffn_up.weight Block 8 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.8: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 9 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
85 blk.9.attn_k.weight Block 9 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
86 blk.9.attn_norm.weight Block 9 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
87 blk.9.attn_output.weight Block 9 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
88 blk.9.attn_q.weight Block 9 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
89 blk.9.attn_v.weight Block 9 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
90 blk.9.ffn_down.weight Block 9 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
91 blk.9.ffn_gate.weight Block 9 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
92 blk.9.ffn_norm.weight Block 9 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
93 blk.9.ffn_up.weight Block 9 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.9: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 10 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
94 blk.10.attn_k.weight Block 10 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
95 blk.10.attn_norm.weight Block 10 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
96 blk.10.attn_output.weight Block 10 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
97 blk.10.attn_q.weight Block 10 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
98 blk.10.attn_v.weight Block 10 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
99 blk.10.ffn_down.weight Block 10 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
100 blk.10.ffn_gate.weight Block 10 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
101 blk.10.ffn_norm.weight Block 10 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
102 blk.10.ffn_up.weight Block 10 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.10: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 11 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
103 blk.11.attn_k.weight Block 11 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
104 blk.11.attn_norm.weight Block 11 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
105 blk.11.attn_output.weight Block 11 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
106 blk.11.attn_q.weight Block 11 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
107 blk.11.attn_v.weight Block 11 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
108 blk.11.ffn_down.weight Block 11 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
109 blk.11.ffn_gate.weight Block 11 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
110 blk.11.ffn_norm.weight Block 11 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
111 blk.11.ffn_up.weight Block 11 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.11: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 12 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
112 blk.12.attn_k.weight Block 12 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
113 blk.12.attn_norm.weight Block 12 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
114 blk.12.attn_output.weight Block 12 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
115 blk.12.attn_q.weight Block 12 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
116 blk.12.attn_v.weight Block 12 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
117 blk.12.ffn_down.weight Block 12 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
118 blk.12.ffn_gate.weight Block 12 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
119 blk.12.ffn_norm.weight Block 12 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
120 blk.12.ffn_up.weight Block 12 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.12: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 13 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
121 blk.13.attn_k.weight Block 13 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
122 blk.13.attn_norm.weight Block 13 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
123 blk.13.attn_output.weight Block 13 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
124 blk.13.attn_q.weight Block 13 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
125 blk.13.attn_v.weight Block 13 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
126 blk.13.ffn_down.weight Block 13 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
127 blk.13.ffn_gate.weight Block 13 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
128 blk.13.ffn_norm.weight Block 13 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
129 blk.13.ffn_up.weight Block 13 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.13: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 14 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
130 blk.14.attn_k.weight Block 14 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
131 blk.14.attn_norm.weight Block 14 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
132 blk.14.attn_output.weight Block 14 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
133 blk.14.attn_q.weight Block 14 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
134 blk.14.attn_v.weight Block 14 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
135 blk.14.ffn_down.weight Block 14 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
136 blk.14.ffn_gate.weight Block 14 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
137 blk.14.ffn_norm.weight Block 14 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
138 blk.14.ffn_up.weight Block 14 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.14: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 15 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
139 blk.15.attn_k.weight Block 15 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
140 blk.15.attn_norm.weight Block 15 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
141 blk.15.attn_output.weight Block 15 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
142 blk.15.attn_q.weight Block 15 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
143 blk.15.attn_v.weight Block 15 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
144 blk.15.ffn_down.weight Block 15 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q6_K
145 blk.15.ffn_gate.weight Block 15 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
146 blk.15.ffn_norm.weight Block 15 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
147 blk.15.ffn_up.weight Block 15 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q5_K
  • Total elements in blk.15: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 16 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
148 blk.16.attn_k.weight Block 16 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
149 blk.16.attn_norm.weight Block 16 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
150 blk.16.attn_output.weight Block 16 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
151 blk.16.attn_q.weight Block 16 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
152 blk.16.attn_v.weight Block 16 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
153 blk.16.ffn_down.weight Block 16 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
154 blk.16.ffn_gate.weight Block 16 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
155 blk.16.ffn_norm.weight Block 16 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
156 blk.16.ffn_up.weight Block 16 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.16: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 17 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
157 blk.17.attn_k.weight Block 17 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
158 blk.17.attn_norm.weight Block 17 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
159 blk.17.attn_output.weight Block 17 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
160 blk.17.attn_q.weight Block 17 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
161 blk.17.attn_v.weight Block 17 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
162 blk.17.ffn_down.weight Block 17 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
163 blk.17.ffn_gate.weight Block 17 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
164 blk.17.ffn_norm.weight Block 17 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
165 blk.17.ffn_up.weight Block 17 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.17: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 18 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
166 blk.18.attn_k.weight Block 18 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
167 blk.18.attn_norm.weight Block 18 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
168 blk.18.attn_output.weight Block 18 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
169 blk.18.attn_q.weight Block 18 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
170 blk.18.attn_v.weight Block 18 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
171 blk.18.ffn_down.weight Block 18 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
172 blk.18.ffn_gate.weight Block 18 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
173 blk.18.ffn_norm.weight Block 18 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
174 blk.18.ffn_up.weight Block 18 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.18: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 19 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
175 blk.19.attn_k.weight Block 19 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
176 blk.19.attn_norm.weight Block 19 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
177 blk.19.attn_output.weight Block 19 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
178 blk.19.attn_q.weight Block 19 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
179 blk.19.attn_v.weight Block 19 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
180 blk.19.ffn_down.weight Block 19 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
181 blk.19.ffn_gate.weight Block 19 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
182 blk.19.ffn_norm.weight Block 19 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
183 blk.19.ffn_up.weight Block 19 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.19: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 20 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
184 blk.20.attn_k.weight Block 20 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
185 blk.20.attn_norm.weight Block 20 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
186 blk.20.attn_output.weight Block 20 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
187 blk.20.attn_q.weight Block 20 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
188 blk.20.attn_v.weight Block 20 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
189 blk.20.ffn_down.weight Block 20 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
190 blk.20.ffn_gate.weight Block 20 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
191 blk.20.ffn_norm.weight Block 20 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
192 blk.20.ffn_up.weight Block 20 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.20: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 21 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
193 blk.21.attn_k.weight Block 21 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
194 blk.21.attn_norm.weight Block 21 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
195 blk.21.attn_output.weight Block 21 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
196 blk.21.attn_q.weight Block 21 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
197 blk.21.attn_v.weight Block 21 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
198 blk.21.ffn_down.weight Block 21 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
199 blk.21.ffn_gate.weight Block 21 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
200 blk.21.ffn_norm.weight Block 21 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
201 blk.21.ffn_up.weight Block 21 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.21: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 22 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
202 blk.22.attn_k.weight Block 22 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
203 blk.22.attn_norm.weight Block 22 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
204 blk.22.attn_output.weight Block 22 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
205 blk.22.attn_q.weight Block 22 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
206 blk.22.attn_v.weight Block 22 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
207 blk.22.ffn_down.weight Block 22 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
208 blk.22.ffn_gate.weight Block 22 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
209 blk.22.ffn_norm.weight Block 22 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
210 blk.22.ffn_up.weight Block 22 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.22: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 23 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
211 blk.23.attn_k.weight Block 23 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
212 blk.23.attn_norm.weight Block 23 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
213 blk.23.attn_output.weight Block 23 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
214 blk.23.attn_q.weight Block 23 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
215 blk.23.attn_v.weight Block 23 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
216 blk.23.ffn_down.weight Block 23 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
217 blk.23.ffn_gate.weight Block 23 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
218 blk.23.ffn_norm.weight Block 23 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
219 blk.23.ffn_up.weight Block 23 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.23: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 24 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
220 blk.24.attn_k.weight Block 24 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
221 blk.24.attn_norm.weight Block 24 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
222 blk.24.attn_output.weight Block 24 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
223 blk.24.attn_q.weight Block 24 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
224 blk.24.attn_v.weight Block 24 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
225 blk.24.ffn_down.weight Block 24 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
226 blk.24.ffn_gate.weight Block 24 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
227 blk.24.ffn_norm.weight Block 24 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
228 blk.24.ffn_up.weight Block 24 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.24: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 25 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
229 blk.25.attn_k.weight Block 25 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
230 blk.25.attn_norm.weight Block 25 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
231 blk.25.attn_output.weight Block 25 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
232 blk.25.attn_q.weight Block 25 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
233 blk.25.attn_v.weight Block 25 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
234 blk.25.ffn_down.weight Block 25 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
235 blk.25.ffn_gate.weight Block 25 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
236 blk.25.ffn_norm.weight Block 25 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
237 blk.25.ffn_up.weight Block 25 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.25: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 26 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
238 blk.26.attn_k.weight Block 26 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
239 blk.26.attn_norm.weight Block 26 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
240 blk.26.attn_output.weight Block 26 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
241 blk.26.attn_q.weight Block 26 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
242 blk.26.attn_v.weight Block 26 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
243 blk.26.ffn_down.weight Block 26 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
244 blk.26.ffn_gate.weight Block 26 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
245 blk.26.ffn_norm.weight Block 26 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
246 blk.26.ffn_up.weight Block 26 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.26: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 27 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
247 blk.27.attn_k.weight Block 27 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
248 blk.27.attn_norm.weight Block 27 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
249 blk.27.attn_output.weight Block 27 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
250 blk.27.attn_q.weight Block 27 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
251 blk.27.attn_v.weight Block 27 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
252 blk.27.ffn_down.weight Block 27 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
253 blk.27.ffn_gate.weight Block 27 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
254 blk.27.ffn_norm.weight Block 27 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
255 blk.27.ffn_up.weight Block 27 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.27: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 28 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
256 blk.28.attn_k.weight Block 28 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
257 blk.28.attn_norm.weight Block 28 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
258 blk.28.attn_output.weight Block 28 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
259 blk.28.attn_q.weight Block 28 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
260 blk.28.attn_v.weight Block 28 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
261 blk.28.ffn_down.weight Block 28 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
262 blk.28.ffn_gate.weight Block 28 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
263 blk.28.ffn_norm.weight Block 28 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
264 blk.28.ffn_up.weight Block 28 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.28: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 29 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
265 blk.29.attn_k.weight Block 29 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
266 blk.29.attn_norm.weight Block 29 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
267 blk.29.attn_output.weight Block 29 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
268 blk.29.attn_q.weight Block 29 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
269 blk.29.attn_v.weight Block 29 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
270 blk.29.ffn_down.weight Block 29 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
271 blk.29.ffn_gate.weight Block 29 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
272 blk.29.ffn_norm.weight Block 29 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
273 blk.29.ffn_up.weight Block 29 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.29: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 30 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
274 blk.30.attn_k.weight Block 30 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
275 blk.30.attn_norm.weight Block 30 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
276 blk.30.attn_output.weight Block 30 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
277 blk.30.attn_q.weight Block 30 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
278 blk.30.attn_v.weight Block 30 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
279 blk.30.ffn_down.weight Block 30 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
280 blk.30.ffn_gate.weight Block 30 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
281 blk.30.ffn_norm.weight Block 30 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
282 blk.30.ffn_up.weight Block 30 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.30: (~218M) 218112000
  • Percentage of total elements: 2.72%

Block 31 Tensor Group : ~218M Elements

T_ID Tensor Layer Name Human Friendly Tensor Layer Name Elements Shape Type
283 blk.31.attn_k.weight Block 31 Attention Key (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q4_K
284 blk.31.attn_norm.weight Block 31 Attention Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
285 blk.31.attn_output.weight Block 31 Attention Output (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q5_K
286 blk.31.attn_q.weight Block 31 Attention Query (W) (~17M) 16777216 4096 x 4096 x 1 x 1 Q4_K
287 blk.31.attn_v.weight Block 31 Attention Value (W) ( ~4M) 4194304 4096 x 1024 x 1 x 1 Q5_K
288 blk.31.ffn_down.weight Block 31 Feed-Forward Network "Down" (W) (~59M) 58720256 14336 x 4096 x 1 x 1 Q5_K
289 blk.31.ffn_gate.weight Block 31 Feed-Forward Network "Gate" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
290 blk.31.ffn_norm.weight Block 31 Feed-Forward Network Normalization (W) ( ~4K) 4096 4096 x 1 x 1 x 1 F32
291 blk.31.ffn_up.weight Block 31 Feed-Forward Network "Up" (W) (~59M) 58720256 4096 x 14336 x 1 x 1 Q4_K
  • Total elements in blk.31: (~218M) 218112000
  • Percentage of total elements: 2.72%