| # DeepSeek-R1-Distill-Llama-8B-Q3_K_M.gguf - GGUF Internal File Dump | |
| - Endian: LITTLE endian | |
| ## Key Value Metadata Store | |
| There are 36 key-value pairs in this file | |
| | POS | TYPE | Count | Key | Value | | |
| | ---: | :------- | -----: | :------------------------------------- | :-------------------------------------------------------------------- | | |
| | 1 | UINT32 | 1 | GGUF.version | 3 | | |
| | 2 | UINT64 | 1 | GGUF.tensor_count | 292 | | |
| | 3 | UINT64 | 1 | GGUF.kv_count | 33 | | |
| | 4 | STRING | 1 | general.architecture | `llama` | | |
| | 5 | STRING | 1 | general.type | `model` | | |
| | 6 | STRING | 1 | general.name | `DeepSeek R1 Distill Llama 8B` | | |
| | 7 | STRING | 1 | general.basename | `DeepSeek-R1-Distill-Llama` | | |
| | 8 | STRING | 1 | general.size_label | `8B` | | |
| | 9 | STRING | 1 | general.license | `mit` | | |
| | 10 | UINT32 | 1 | llama.block_count | 32 | | |
| | 11 | UINT32 | 1 | llama.context_length | 131072 | | |
| | 12 | UINT32 | 1 | llama.embedding_length | 4096 | | |
| | 13 | UINT32 | 1 | llama.feed_forward_length | 14336 | | |
| | 14 | UINT32 | 1 | llama.attention.head_count | 32 | | |
| | 15 | UINT32 | 1 | llama.attention.head_count_kv | 8 | | |
| | 16 | FLOAT32 | 1 | llama.rope.freq_base | 500000.0 | | |
| | 17 | FLOAT32 | 1 | llama.attention.layer_norm_rms_epsilon | 1e-05 | | |
| | 18 | UINT32 | 1 | llama.vocab_size | 128256 | | |
| | 19 | UINT32 | 1 | llama.rope.dimension_count | 128 | | |
| | 20 | STRING | 1 | tokenizer.ggml.model | `gpt2` | | |
| | 21 | STRING | 1 | tokenizer.ggml.pre | `llama-bpe` | | |
| | 22 | [STRING] | 128256 | tokenizer.ggml.tokens | [ `!`, `"`, `#`, `$`, `%`, ... ] | | |
| | 23 | [INT32] | 128256 | tokenizer.ggml.token_type | [ 1, 1, 1, 1, 1, 1, 1, ... ] | | |
| | 24 | [STRING] | 280147 | tokenizer.ggml.merges | [ `Ġ Ġ`, `Ġ ĠĠĠ`, `ĠĠ ĠĠ`, `ĠĠĠ Ġ`, `i n`, ... ] | | |
| | 25 | UINT32 | 1 | tokenizer.ggml.bos_token_id | 128000 | | |
| | 26 | UINT32 | 1 | tokenizer.ggml.eos_token_id | 128001 | | |
| | 27 | UINT32 | 1 | tokenizer.ggml.padding_token_id | 128001 | | |
| | 28 | BOOL | 1 | tokenizer.ggml.add_bos_token | True | | |
| | 29 | BOOL | 1 | tokenizer.ggml.add_eos_token | False | | |
| | 30 | STRING | 1 | tokenizer.chat_template | `{% if not add_generation_promp`...`{{'<｜Assistant｜>'}}{% endif %}` | | |
| | 31 | UINT32 | 1 | general.quantization_version | 2 | | |
| | 32 | UINT32 | 1 | general.file_type | 12 | | |
| | 33 | STRING | 1 | quantize.imatrix.file | `./imatrix/imatrix-DeepSeek-R1-Distill-Llama-8B-small.dat` | | |
| | 34 | STRING | 1 | quantize.imatrix.dataset | `../../datasets/imatrix/calibration_all_small.txt` | | |
| | 35 | INT32 | 1 | quantize.imatrix.entries_count | 225 | | |
| | 36 | INT32 | 1 | quantize.imatrix.chunks_count | 1130 | | |
| ## Tensors Overview ~8B Elements | |
| Total number of elements in all tensors: 8030261312 Elements | |
| - [DeepSeek-R1-Distill-Llama-8B-Q3\_K\_M.gguf - GGUF Internal File Dump](#deepseek-r1-distill-llama-8b-q3_k_mgguf---gguf-internal-file-dump) | |
| - [Key Value Metadata Store](#key-value-metadata-store) | |
| - [Tensors Overview ~8B Elements](#tensors-overview-8b-elements) | |
| - [Tensor Data Offset](#tensor-data-offset) | |
| - [Base Tensor Group : ~1B Elements](#base-tensor-group--1b-elements) | |
| - [Block 0 Tensor Group : ~218M Elements](#block-0-tensor-group--218m-elements) | |
| - [Block 1 Tensor Group : ~218M Elements](#block-1-tensor-group--218m-elements) | |
| - [Block 2 Tensor Group : ~218M Elements](#block-2-tensor-group--218m-elements) | |
| - [Block 3 Tensor Group : ~218M Elements](#block-3-tensor-group--218m-elements) | |
| - [Block 4 Tensor Group : ~218M Elements](#block-4-tensor-group--218m-elements) | |
| - [Block 5 Tensor Group : ~218M Elements](#block-5-tensor-group--218m-elements) | |
| - [Block 6 Tensor Group : ~218M Elements](#block-6-tensor-group--218m-elements) | |
| - [Block 7 Tensor Group : ~218M Elements](#block-7-tensor-group--218m-elements) | |
| - [Block 8 Tensor Group : ~218M Elements](#block-8-tensor-group--218m-elements) | |
| - [Block 9 Tensor Group : ~218M Elements](#block-9-tensor-group--218m-elements) | |
| - [Block 10 Tensor Group : ~218M Elements](#block-10-tensor-group--218m-elements) | |
| - [Block 11 Tensor Group : ~218M Elements](#block-11-tensor-group--218m-elements) | |
| - [Block 12 Tensor Group : ~218M Elements](#block-12-tensor-group--218m-elements) | |
| - [Block 13 Tensor Group : ~218M Elements](#block-13-tensor-group--218m-elements) | |
| - [Block 14 Tensor Group : ~218M Elements](#block-14-tensor-group--218m-elements) | |
| - [Block 15 Tensor Group : ~218M Elements](#block-15-tensor-group--218m-elements) | |
| - [Block 16 Tensor Group : ~218M Elements](#block-16-tensor-group--218m-elements) | |
| - [Block 17 Tensor Group : ~218M Elements](#block-17-tensor-group--218m-elements) | |
| - [Block 18 Tensor Group : ~218M Elements](#block-18-tensor-group--218m-elements) | |
| - [Block 19 Tensor Group : ~218M Elements](#block-19-tensor-group--218m-elements) | |
| - [Block 20 Tensor Group : ~218M Elements](#block-20-tensor-group--218m-elements) | |
| - [Block 21 Tensor Group : ~218M Elements](#block-21-tensor-group--218m-elements) | |
| - [Block 22 Tensor Group : ~218M Elements](#block-22-tensor-group--218m-elements) | |
| - [Block 23 Tensor Group : ~218M Elements](#block-23-tensor-group--218m-elements) | |
| - [Block 24 Tensor Group : ~218M Elements](#block-24-tensor-group--218m-elements) | |
| - [Block 25 Tensor Group : ~218M Elements](#block-25-tensor-group--218m-elements) | |
| - [Block 26 Tensor Group : ~218M Elements](#block-26-tensor-group--218m-elements) | |
| - [Block 27 Tensor Group : ~218M Elements](#block-27-tensor-group--218m-elements) | |
| - [Block 28 Tensor Group : ~218M Elements](#block-28-tensor-group--218m-elements) | |
| - [Block 29 Tensor Group : ~218M Elements](#block-29-tensor-group--218m-elements) | |
| - [Block 30 Tensor Group : ~218M Elements](#block-30-tensor-group--218m-elements) | |
| - [Block 31 Tensor Group : ~218M Elements](#block-31-tensor-group--218m-elements) | |
| ### Tensor Data Offset | |
| This table contains the offset and data segment relative to start of file | |
| | T_ID | Tensor Layer Name | Data Offset (B) | Data Size (B) | | |
| | ---: | :------------------------ | --------------: | ------------: | | |
| | 0 | output.weight | 0x779a80 | 0xd746000 | | |
| | 1 | output_norm.weight | 0xdebfa80 | 0x4000 | | |
| | 2 | rope_freqs.weight | 0xdec3a80 | 0x100 | | |
| | 3 | token_embd.weight | 0xdec3b80 | 0xd746000 | | |
| | 4 | blk.0.attn_k.weight | 0x1b609b80 | 0x1b8000 | | |
| | 5 | blk.0.attn_norm.weight | 0x1b7c1b80 | 0x4000 | | |
| | 6 | blk.0.attn_output.weight | 0x1b7c5b80 | 0x900000 | | |
| | 7 | blk.0.attn_q.weight | 0x1c0c5b80 | 0x6e0000 | | |
| | 8 | blk.0.attn_v.weight | 0x1c7a5b80 | 0x240000 | | |
| | 9 | blk.0.ffn_down.weight | 0x1c9e5b80 | 0x1f80000 | | |
| | 10 | blk.0.ffn_gate.weight | 0x1e965b80 | 0x1810000 | | |
| | 11 | blk.0.ffn_norm.weight | 0x20175b80 | 0x4000 | | |
| | 12 | blk.0.ffn_up.weight | 0x20179b80 | 0x1810000 | | |
| | 13 | blk.1.attn_k.weight | 0x21989b80 | 0x1b8000 | | |
| | 14 | blk.1.attn_norm.weight | 0x21b41b80 | 0x4000 | | |
| | 15 | blk.1.attn_output.weight | 0x21b45b80 | 0x900000 | | |
| | 16 | blk.1.attn_q.weight | 0x22445b80 | 0x6e0000 | | |
| | 17 | blk.1.attn_v.weight | 0x22b25b80 | 0x240000 | | |
| | 18 | blk.1.ffn_down.weight | 0x22d65b80 | 0x1f80000 | | |
| | 19 | blk.1.ffn_gate.weight | 0x24ce5b80 | 0x1810000 | | |
| | 20 | blk.1.ffn_norm.weight | 0x264f5b80 | 0x4000 | | |
| | 21 | blk.1.ffn_up.weight | 0x264f9b80 | 0x1810000 | | |
| | 22 | blk.2.attn_k.weight | 0x27d09b80 | 0x1b8000 | | |
| | 23 | blk.2.attn_norm.weight | 0x27ec1b80 | 0x4000 | | |
| | 24 | blk.2.attn_output.weight | 0x27ec5b80 | 0x900000 | | |
| | 25 | blk.2.attn_q.weight | 0x287c5b80 | 0x6e0000 | | |
| | 26 | blk.2.attn_v.weight | 0x28ea5b80 | 0x240000 | | |
| | 27 | blk.2.ffn_down.weight | 0x290e5b80 | 0x1f80000 | | |
| | 28 | blk.2.ffn_gate.weight | 0x2b065b80 | 0x1810000 | | |
| | 29 | blk.2.ffn_norm.weight | 0x2c875b80 | 0x4000 | | |
| | 30 | blk.2.ffn_up.weight | 0x2c879b80 | 0x1810000 | | |
| | 31 | blk.3.attn_k.weight | 0x2e089b80 | 0x1b8000 | | |
| | 32 | blk.3.attn_norm.weight | 0x2e241b80 | 0x4000 | | |
| | 33 | blk.3.attn_output.weight | 0x2e245b80 | 0x900000 | | |
| | 34 | blk.3.attn_q.weight | 0x2eb45b80 | 0x6e0000 | | |
| | 35 | blk.3.attn_v.weight | 0x2f225b80 | 0x240000 | | |
| | 36 | blk.3.ffn_down.weight | 0x2f465b80 | 0x1f80000 | | |
| | 37 | blk.3.ffn_gate.weight | 0x313e5b80 | 0x1810000 | | |
| | 38 | blk.3.ffn_norm.weight | 0x32bf5b80 | 0x4000 | | |
| | 39 | blk.3.ffn_up.weight | 0x32bf9b80 | 0x1810000 | | |
| | 40 | blk.4.attn_k.weight | 0x34409b80 | 0x1b8000 | | |
| | 41 | blk.4.attn_norm.weight | 0x345c1b80 | 0x4000 | | |
| | 42 | blk.4.attn_output.weight | 0x345c5b80 | 0x900000 | | |
| | 43 | blk.4.attn_q.weight | 0x34ec5b80 | 0x6e0000 | | |
| | 44 | blk.4.attn_v.weight | 0x355a5b80 | 0x240000 | | |
| | 45 | blk.4.ffn_down.weight | 0x357e5b80 | 0x1f80000 | | |
| | 46 | blk.4.ffn_gate.weight | 0x37765b80 | 0x1810000 | | |
| | 47 | blk.4.ffn_norm.weight | 0x38f75b80 | 0x4000 | | |
| | 48 | blk.4.ffn_up.weight | 0x38f79b80 | 0x1810000 | | |
| | 49 | blk.5.attn_k.weight | 0x3a789b80 | 0x1b8000 | | |
| | 50 | blk.5.attn_norm.weight | 0x3a941b80 | 0x4000 | | |
| | 51 | blk.5.attn_output.weight | 0x3a945b80 | 0x900000 | | |
| | 52 | blk.5.attn_q.weight | 0x3b245b80 | 0x6e0000 | | |
| | 53 | blk.5.attn_v.weight | 0x3b925b80 | 0x240000 | | |
| | 54 | blk.5.ffn_down.weight | 0x3bb65b80 | 0x1f80000 | | |
| | 55 | blk.5.ffn_gate.weight | 0x3dae5b80 | 0x1810000 | | |
| | 56 | blk.5.ffn_norm.weight | 0x3f2f5b80 | 0x4000 | | |
| | 57 | blk.5.ffn_up.weight | 0x3f2f9b80 | 0x1810000 | | |
| | 58 | blk.6.attn_k.weight | 0x40b09b80 | 0x1b8000 | | |
| | 59 | blk.6.attn_norm.weight | 0x40cc1b80 | 0x4000 | | |
| | 60 | blk.6.attn_output.weight | 0x40cc5b80 | 0x900000 | | |
| | 61 | blk.6.attn_q.weight | 0x415c5b80 | 0x6e0000 | | |
| | 62 | blk.6.attn_v.weight | 0x41ca5b80 | 0x240000 | | |
| | 63 | blk.6.ffn_down.weight | 0x41ee5b80 | 0x1f80000 | | |
| | 64 | blk.6.ffn_gate.weight | 0x43e65b80 | 0x1810000 | | |
| | 65 | blk.6.ffn_norm.weight | 0x45675b80 | 0x4000 | | |
| | 66 | blk.6.ffn_up.weight | 0x45679b80 | 0x1810000 | | |
| | 67 | blk.7.attn_k.weight | 0x46e89b80 | 0x1b8000 | | |
| | 68 | blk.7.attn_norm.weight | 0x47041b80 | 0x4000 | | |
| | 69 | blk.7.attn_output.weight | 0x47045b80 | 0x900000 | | |
| | 70 | blk.7.attn_q.weight | 0x47945b80 | 0x6e0000 | | |
| | 71 | blk.7.attn_v.weight | 0x48025b80 | 0x240000 | | |
| | 72 | blk.7.ffn_down.weight | 0x48265b80 | 0x1f80000 | | |
| | 73 | blk.7.ffn_gate.weight | 0x4a1e5b80 | 0x1810000 | | |
| | 74 | blk.7.ffn_norm.weight | 0x4b9f5b80 | 0x4000 | | |
| | 75 | blk.7.ffn_up.weight | 0x4b9f9b80 | 0x1810000 | | |
| | 76 | blk.8.attn_k.weight | 0x4d209b80 | 0x1b8000 | | |
| | 77 | blk.8.attn_norm.weight | 0x4d3c1b80 | 0x4000 | | |
| | 78 | blk.8.attn_output.weight | 0x4d3c5b80 | 0x900000 | | |
| | 79 | blk.8.attn_q.weight | 0x4dcc5b80 | 0x6e0000 | | |
| | 80 | blk.8.attn_v.weight | 0x4e3a5b80 | 0x240000 | | |
| | 81 | blk.8.ffn_down.weight | 0x4e5e5b80 | 0x1f80000 | | |
| | 82 | blk.8.ffn_gate.weight | 0x50565b80 | 0x1810000 | | |
| | 83 | blk.8.ffn_norm.weight | 0x51d75b80 | 0x4000 | | |
| | 84 | blk.8.ffn_up.weight | 0x51d79b80 | 0x1810000 | | |
| | 85 | blk.9.attn_k.weight | 0x53589b80 | 0x1b8000 | | |
| | 86 | blk.9.attn_norm.weight | 0x53741b80 | 0x4000 | | |
| | 87 | blk.9.attn_output.weight | 0x53745b80 | 0x900000 | | |
| | 88 | blk.9.attn_q.weight | 0x54045b80 | 0x6e0000 | | |
| | 89 | blk.9.attn_v.weight | 0x54725b80 | 0x240000 | | |
| | 90 | blk.9.ffn_down.weight | 0x54965b80 | 0x1f80000 | | |
| | 91 | blk.9.ffn_gate.weight | 0x568e5b80 | 0x1810000 | | |
| | 92 | blk.9.ffn_norm.weight | 0x580f5b80 | 0x4000 | | |
| | 93 | blk.9.ffn_up.weight | 0x580f9b80 | 0x1810000 | | |
| | 94 | blk.10.attn_k.weight | 0x59909b80 | 0x1b8000 | | |
| | 95 | blk.10.attn_norm.weight | 0x59ac1b80 | 0x4000 | | |
| | 96 | blk.10.attn_output.weight | 0x59ac5b80 | 0x900000 | | |
| | 97 | blk.10.attn_q.weight | 0x5a3c5b80 | 0x6e0000 | | |
| | 98 | blk.10.attn_v.weight | 0x5aaa5b80 | 0x240000 | | |
| | 99 | blk.10.ffn_down.weight | 0x5ace5b80 | 0x1f80000 | | |
| | 100 | blk.10.ffn_gate.weight | 0x5cc65b80 | 0x1810000 | | |
| | 101 | blk.10.ffn_norm.weight | 0x5e475b80 | 0x4000 | | |
| | 102 | blk.10.ffn_up.weight | 0x5e479b80 | 0x1810000 | | |
| | 103 | blk.11.attn_k.weight | 0x5fc89b80 | 0x1b8000 | | |
| | 104 | blk.11.attn_norm.weight | 0x5fe41b80 | 0x4000 | | |
| | 105 | blk.11.attn_output.weight | 0x5fe45b80 | 0x900000 | | |
| | 106 | blk.11.attn_q.weight | 0x60745b80 | 0x6e0000 | | |
| | 107 | blk.11.attn_v.weight | 0x60e25b80 | 0x240000 | | |
| | 108 | blk.11.ffn_down.weight | 0x61065b80 | 0x1f80000 | | |
| | 109 | blk.11.ffn_gate.weight | 0x62fe5b80 | 0x1810000 | | |
| | 110 | blk.11.ffn_norm.weight | 0x647f5b80 | 0x4000 | | |
| | 111 | blk.11.ffn_up.weight | 0x647f9b80 | 0x1810000 | | |
| | 112 | blk.12.attn_k.weight | 0x66009b80 | 0x1b8000 | | |
| | 113 | blk.12.attn_norm.weight | 0x661c1b80 | 0x4000 | | |
| | 114 | blk.12.attn_output.weight | 0x661c5b80 | 0x900000 | | |
| | 115 | blk.12.attn_q.weight | 0x66ac5b80 | 0x6e0000 | | |
| | 116 | blk.12.attn_v.weight | 0x671a5b80 | 0x240000 | | |
| | 117 | blk.12.ffn_down.weight | 0x673e5b80 | 0x1f80000 | | |
| | 118 | blk.12.ffn_gate.weight | 0x69365b80 | 0x1810000 | | |
| | 119 | blk.12.ffn_norm.weight | 0x6ab75b80 | 0x4000 | | |
| | 120 | blk.12.ffn_up.weight | 0x6ab79b80 | 0x1810000 | | |
| | 121 | blk.13.attn_k.weight | 0x6c389b80 | 0x1b8000 | | |
| | 122 | blk.13.attn_norm.weight | 0x6c541b80 | 0x4000 | | |
| | 123 | blk.13.attn_output.weight | 0x6c545b80 | 0x900000 | | |
| | 124 | blk.13.attn_q.weight | 0x6ce45b80 | 0x6e0000 | | |
| | 125 | blk.13.attn_v.weight | 0x6d525b80 | 0x240000 | | |
| | 126 | blk.13.ffn_down.weight | 0x6d765b80 | 0x1f80000 | | |
| | 127 | blk.13.ffn_gate.weight | 0x6f6e5b80 | 0x1810000 | | |
| | 128 | blk.13.ffn_norm.weight | 0x70ef5b80 | 0x4000 | | |
| | 129 | blk.13.ffn_up.weight | 0x70ef9b80 | 0x1810000 | | |
| | 130 | blk.14.attn_k.weight | 0x72709b80 | 0x1b8000 | | |
| | 131 | blk.14.attn_norm.weight | 0x728c1b80 | 0x4000 | | |
| | 132 | blk.14.attn_output.weight | 0x728c5b80 | 0x900000 | | |
| | 133 | blk.14.attn_q.weight | 0x731c5b80 | 0x6e0000 | | |
| | 134 | blk.14.attn_v.weight | 0x738a5b80 | 0x240000 | | |
| | 135 | blk.14.ffn_down.weight | 0x73ae5b80 | 0x1f80000 | | |
| | 136 | blk.14.ffn_gate.weight | 0x75a65b80 | 0x1810000 | | |
| | 137 | blk.14.ffn_norm.weight | 0x77275b80 | 0x4000 | | |
| | 138 | blk.14.ffn_up.weight | 0x77279b80 | 0x1810000 | | |
| | 139 | blk.15.attn_k.weight | 0x78a89b80 | 0x1b8000 | | |
| | 140 | blk.15.attn_norm.weight | 0x78c41b80 | 0x4000 | | |
| | 141 | blk.15.attn_output.weight | 0x78c45b80 | 0x900000 | | |
| | 142 | blk.15.attn_q.weight | 0x79545b80 | 0x6e0000 | | |
| | 143 | blk.15.attn_v.weight | 0x79c25b80 | 0x240000 | | |
| | 144 | blk.15.ffn_down.weight | 0x79e65b80 | 0x1f80000 | | |
| | 145 | blk.15.ffn_gate.weight | 0x7bde5b80 | 0x1810000 | | |
| | 146 | blk.15.ffn_norm.weight | 0x7d5f5b80 | 0x4000 | | |
| | 147 | blk.15.ffn_up.weight | 0x7d5f9b80 | 0x1810000 | | |
| | 148 | blk.16.attn_k.weight | 0x7ee09b80 | 0x150000 | | |
| | 149 | blk.16.attn_norm.weight | 0x7ef59b80 | 0x4000 | | |
| | 150 | blk.16.attn_output.weight | 0x7ef5db80 | 0x900000 | | |
| | 151 | blk.16.attn_q.weight | 0x7f85db80 | 0x540000 | | |
| | 152 | blk.16.attn_v.weight | 0x7fd9db80 | 0x1b8000 | | |
| | 153 | blk.16.ffn_down.weight | 0x7ff55b80 | 0x1f80000 | | |
| | 154 | blk.16.ffn_gate.weight | 0x81ed5b80 | 0x1260000 | | |
| | 155 | blk.16.ffn_norm.weight | 0x83135b80 | 0x4000 | | |
| | 156 | blk.16.ffn_up.weight | 0x83139b80 | 0x1260000 | | |
| | 157 | blk.17.attn_k.weight | 0x84399b80 | 0x150000 | | |
| | 158 | blk.17.attn_norm.weight | 0x844e9b80 | 0x4000 | | |
| | 159 | blk.17.attn_output.weight | 0x844edb80 | 0x900000 | | |
| | 160 | blk.17.attn_q.weight | 0x84dedb80 | 0x540000 | | |
| | 161 | blk.17.attn_v.weight | 0x8532db80 | 0x1b8000 | | |
| | 162 | blk.17.ffn_down.weight | 0x854e5b80 | 0x1f80000 | | |
| | 163 | blk.17.ffn_gate.weight | 0x87465b80 | 0x1260000 | | |
| | 164 | blk.17.ffn_norm.weight | 0x886c5b80 | 0x4000 | | |
| | 165 | blk.17.ffn_up.weight | 0x886c9b80 | 0x1260000 | | |
| | 166 | blk.18.attn_k.weight | 0x89929b80 | 0x150000 | | |
| | 167 | blk.18.attn_norm.weight | 0x89a79b80 | 0x4000 | | |
| | 168 | blk.18.attn_output.weight | 0x89a7db80 | 0x900000 | | |
| | 169 | blk.18.attn_q.weight | 0x8a37db80 | 0x540000 | | |
| | 170 | blk.18.attn_v.weight | 0x8a8bdb80 | 0x1b8000 | | |
| | 171 | blk.18.ffn_down.weight | 0x8aa75b80 | 0x1f80000 | | |
| | 172 | blk.18.ffn_gate.weight | 0x8c9f5b80 | 0x1260000 | | |
| | 173 | blk.18.ffn_norm.weight | 0x8dc55b80 | 0x4000 | | |
| | 174 | blk.18.ffn_up.weight | 0x8dc59b80 | 0x1260000 | | |
| | 175 | blk.19.attn_k.weight | 0x8eeb9b80 | 0x150000 | | |
| | 176 | blk.19.attn_norm.weight | 0x8f009b80 | 0x4000 | | |
| | 177 | blk.19.attn_output.weight | 0x8f00db80 | 0x900000 | | |
| | 178 | blk.19.attn_q.weight | 0x8f90db80 | 0x540000 | | |
| | 179 | blk.19.attn_v.weight | 0x8fe4db80 | 0x1b8000 | | |
| | 180 | blk.19.ffn_down.weight | 0x90005b80 | 0x1f80000 | | |
| | 181 | blk.19.ffn_gate.weight | 0x91f85b80 | 0x1260000 | | |
| | 182 | blk.19.ffn_norm.weight | 0x931e5b80 | 0x4000 | | |
| | 183 | blk.19.ffn_up.weight | 0x931e9b80 | 0x1260000 | | |
| | 184 | blk.20.attn_k.weight | 0x94449b80 | 0x150000 | | |
| | 185 | blk.20.attn_norm.weight | 0x94599b80 | 0x4000 | | |
| | 186 | blk.20.attn_output.weight | 0x9459db80 | 0x900000 | | |
| | 187 | blk.20.attn_q.weight | 0x94e9db80 | 0x540000 | | |
| | 188 | blk.20.attn_v.weight | 0x953ddb80 | 0x1b8000 | | |
| | 189 | blk.20.ffn_down.weight | 0x95595b80 | 0x1f80000 | | |
| | 190 | blk.20.ffn_gate.weight | 0x97515b80 | 0x1260000 | | |
| | 191 | blk.20.ffn_norm.weight | 0x98775b80 | 0x4000 | | |
| | 192 | blk.20.ffn_up.weight | 0x98779b80 | 0x1260000 | | |
| | 193 | blk.21.attn_k.weight | 0x999d9b80 | 0x150000 | | |
| | 194 | blk.21.attn_norm.weight | 0x99b29b80 | 0x4000 | | |
| | 195 | blk.21.attn_output.weight | 0x99b2db80 | 0x900000 | | |
| | 196 | blk.21.attn_q.weight | 0x9a42db80 | 0x540000 | | |
| | 197 | blk.21.attn_v.weight | 0x9a96db80 | 0x1b8000 | | |
| | 198 | blk.21.ffn_down.weight | 0x9ab25b80 | 0x1f80000 | | |
| | 199 | blk.21.ffn_gate.weight | 0x9caa5b80 | 0x1260000 | | |
| | 200 | blk.21.ffn_norm.weight | 0x9dd05b80 | 0x4000 | | |
| | 201 | blk.21.ffn_up.weight | 0x9dd09b80 | 0x1260000 | | |
| | 202 | blk.22.attn_k.weight | 0x9ef69b80 | 0x150000 | | |
| | 203 | blk.22.attn_norm.weight | 0x9f0b9b80 | 0x4000 | | |
| | 204 | blk.22.attn_output.weight | 0x9f0bdb80 | 0x900000 | | |
| | 205 | blk.22.attn_q.weight | 0x9f9bdb80 | 0x540000 | | |
| | 206 | blk.22.attn_v.weight | 0x9fefdb80 | 0x1b8000 | | |
| | 207 | blk.22.ffn_down.weight | 0xa00b5b80 | 0x1f80000 | | |
| | 208 | blk.22.ffn_gate.weight | 0xa2035b80 | 0x1260000 | | |
| | 209 | blk.22.ffn_norm.weight | 0xa3295b80 | 0x4000 | | |
| | 210 | blk.22.ffn_up.weight | 0xa3299b80 | 0x1260000 | | |
| | 211 | blk.23.attn_k.weight | 0xa44f9b80 | 0x150000 | | |
| | 212 | blk.23.attn_norm.weight | 0xa4649b80 | 0x4000 | | |
| | 213 | blk.23.attn_output.weight | 0xa464db80 | 0x900000 | | |
| | 214 | blk.23.attn_q.weight | 0xa4f4db80 | 0x540000 | | |
| | 215 | blk.23.attn_v.weight | 0xa548db80 | 0x1b8000 | | |
| | 216 | blk.23.ffn_down.weight | 0xa5645b80 | 0x1f80000 | | |
| | 217 | blk.23.ffn_gate.weight | 0xa75c5b80 | 0x1260000 | | |
| | 218 | blk.23.ffn_norm.weight | 0xa8825b80 | 0x4000 | | |
| | 219 | blk.23.ffn_up.weight | 0xa8829b80 | 0x1260000 | | |
| | 220 | blk.24.attn_k.weight | 0xa9a89b80 | 0x150000 | | |
| | 221 | blk.24.attn_norm.weight | 0xa9bd9b80 | 0x4000 | | |
| | 222 | blk.24.attn_output.weight | 0xa9bddb80 | 0x900000 | | |
| | 223 | blk.24.attn_q.weight | 0xaa4ddb80 | 0x540000 | | |
| | 224 | blk.24.attn_v.weight | 0xaaa1db80 | 0x1b8000 | | |
| | 225 | blk.24.ffn_down.weight | 0xaabd5b80 | 0x1f80000 | | |
| | 226 | blk.24.ffn_gate.weight | 0xacb55b80 | 0x1260000 | | |
| | 227 | blk.24.ffn_norm.weight | 0xaddb5b80 | 0x4000 | | |
| | 228 | blk.24.ffn_up.weight | 0xaddb9b80 | 0x1260000 | | |
| | 229 | blk.25.attn_k.weight | 0xaf019b80 | 0x150000 | | |
| | 230 | blk.25.attn_norm.weight | 0xaf169b80 | 0x4000 | | |
| | 231 | blk.25.attn_output.weight | 0xaf16db80 | 0x900000 | | |
| | 232 | blk.25.attn_q.weight | 0xafa6db80 | 0x540000 | | |
| | 233 | blk.25.attn_v.weight | 0xaffadb80 | 0x1b8000 | | |
| | 234 | blk.25.ffn_down.weight | 0xb0165b80 | 0x1f80000 | | |
| | 235 | blk.25.ffn_gate.weight | 0xb20e5b80 | 0x1260000 | | |
| | 236 | blk.25.ffn_norm.weight | 0xb3345b80 | 0x4000 | | |
| | 237 | blk.25.ffn_up.weight | 0xb3349b80 | 0x1260000 | | |
| | 238 | blk.26.attn_k.weight | 0xb45a9b80 | 0x150000 | | |
| | 239 | blk.26.attn_norm.weight | 0xb46f9b80 | 0x4000 | | |
| | 240 | blk.26.attn_output.weight | 0xb46fdb80 | 0x900000 | | |
| | 241 | blk.26.attn_q.weight | 0xb4ffdb80 | 0x540000 | | |
| | 242 | blk.26.attn_v.weight | 0xb553db80 | 0x1b8000 | | |
| | 243 | blk.26.ffn_down.weight | 0xb56f5b80 | 0x1f80000 | | |
| | 244 | blk.26.ffn_gate.weight | 0xb7675b80 | 0x1260000 | | |
| | 245 | blk.26.ffn_norm.weight | 0xb88d5b80 | 0x4000 | | |
| | 246 | blk.26.ffn_up.weight | 0xb88d9b80 | 0x1260000 | | |
| | 247 | blk.27.attn_k.weight | 0xb9b39b80 | 0x150000 | | |
| | 248 | blk.27.attn_norm.weight | 0xb9c89b80 | 0x4000 | | |
| | 249 | blk.27.attn_output.weight | 0xb9c8db80 | 0x900000 | | |
| | 250 | blk.27.attn_q.weight | 0xba58db80 | 0x540000 | | |
| | 251 | blk.27.attn_v.weight | 0xbaacdb80 | 0x1b8000 | | |
| | 252 | blk.27.ffn_down.weight | 0xbac85b80 | 0x1f80000 | | |
| | 253 | blk.27.ffn_gate.weight | 0xbcc05b80 | 0x1260000 | | |
| | 254 | blk.27.ffn_norm.weight | 0xbde65b80 | 0x4000 | | |
| | 255 | blk.27.ffn_up.weight | 0xbde69b80 | 0x1260000 | | |
| | 256 | blk.28.attn_k.weight | 0xbf0c9b80 | 0x150000 | | |
| | 257 | blk.28.attn_norm.weight | 0xbf219b80 | 0x4000 | | |
| | 258 | blk.28.attn_output.weight | 0xbf21db80 | 0x900000 | | |
| | 259 | blk.28.attn_q.weight | 0xbfb1db80 | 0x540000 | | |
| | 260 | blk.28.attn_v.weight | 0xc005db80 | 0x1b8000 | | |
| | 261 | blk.28.ffn_down.weight | 0xc0215b80 | 0x1f80000 | | |
| | 262 | blk.28.ffn_gate.weight | 0xc2195b80 | 0x1260000 | | |
| | 263 | blk.28.ffn_norm.weight | 0xc33f5b80 | 0x4000 | | |
| | 264 | blk.28.ffn_up.weight | 0xc33f9b80 | 0x1260000 | | |
| | 265 | blk.29.attn_k.weight | 0xc4659b80 | 0x150000 | | |
| | 266 | blk.29.attn_norm.weight | 0xc47a9b80 | 0x4000 | | |
| | 267 | blk.29.attn_output.weight | 0xc47adb80 | 0x900000 | | |
| | 268 | blk.29.attn_q.weight | 0xc50adb80 | 0x540000 | | |
| | 269 | blk.29.attn_v.weight | 0xc55edb80 | 0x1b8000 | | |
| | 270 | blk.29.ffn_down.weight | 0xc57a5b80 | 0x1f80000 | | |
| | 271 | blk.29.ffn_gate.weight | 0xc7725b80 | 0x1260000 | | |
| | 272 | blk.29.ffn_norm.weight | 0xc8985b80 | 0x4000 | | |
| | 273 | blk.29.ffn_up.weight | 0xc8989b80 | 0x1260000 | | |
| | 274 | blk.30.attn_k.weight | 0xc9be9b80 | 0x150000 | | |
| | 275 | blk.30.attn_norm.weight | 0xc9d39b80 | 0x4000 | | |
| | 276 | blk.30.attn_output.weight | 0xc9d3db80 | 0x900000 | | |
| | 277 | blk.30.attn_q.weight | 0xca63db80 | 0x540000 | | |
| | 278 | blk.30.attn_v.weight | 0xcab7db80 | 0x1b8000 | | |
| | 279 | blk.30.ffn_down.weight | 0xcad35b80 | 0x1f80000 | | |
| | 280 | blk.30.ffn_gate.weight | 0xcccb5b80 | 0x1260000 | | |
| | 281 | blk.30.ffn_norm.weight | 0xcdf15b80 | 0x4000 | | |
| | 282 | blk.30.ffn_up.weight | 0xcdf19b80 | 0x1260000 | | |
| | 283 | blk.31.attn_k.weight | 0xcf179b80 | 0x150000 | | |
| | 284 | blk.31.attn_norm.weight | 0xcf2c9b80 | 0x4000 | | |
| | 285 | blk.31.attn_output.weight | 0xcf2cdb80 | 0x900000 | | |
| | 286 | blk.31.attn_q.weight | 0xcfbcdb80 | 0x540000 | | |
| | 287 | blk.31.attn_v.weight | 0xd010db80 | 0x1b8000 | | |
| | 288 | blk.31.ffn_down.weight | 0xd02c5b80 | 0x1f80000 | | |
| | 289 | blk.31.ffn_gate.weight | 0xd2245b80 | 0x1260000 | | |
| | 290 | blk.31.ffn_norm.weight | 0xd34a5b80 | 0x4000 | | |
| | 291 | blk.31.ffn_up.weight | 0xd34a9b80 | 0x1260000 | | |
| ### <a name="base">Base Tensor Group : ~1B Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------- | :------------------------------- | :---------------- | :-------------------- | :--- | | |
| | 0 | output.weight | Output (W) | (~525M) 525336576 | 4096 x 128256 x 1 x 1 | Q3_K | | |
| | 1 | output_norm.weight | Output Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 2 | rope_freqs.weight | Rope_Freqs (W) | ( 64) 64 | 64 x 1 x 1 x 1 | F32 | | |
| | 3 | token_embd.weight | Token Embedding (W) | (~525M) 525336576 | 4096 x 128256 x 1 x 1 | Q3_K | | |
| - Total elements in base: ( ~1B) 1050677312 | |
| - Percentage of total elements: 13.08% | |
| ### <a name="blk_0">Block 0 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 4 | blk.0.attn_k.weight | Block 0 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 5 | blk.0.attn_norm.weight | Block 0 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 6 | blk.0.attn_output.weight | Block 0 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 7 | blk.0.attn_q.weight | Block 0 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 8 | blk.0.attn_v.weight | Block 0 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 9 | blk.0.ffn_down.weight | Block 0 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 10 | blk.0.ffn_gate.weight | Block 0 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 11 | blk.0.ffn_norm.weight | Block 0 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 12 | blk.0.ffn_up.weight | Block 0 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.0: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_1">Block 1 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 13 | blk.1.attn_k.weight | Block 1 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 14 | blk.1.attn_norm.weight | Block 1 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 15 | blk.1.attn_output.weight | Block 1 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 16 | blk.1.attn_q.weight | Block 1 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 17 | blk.1.attn_v.weight | Block 1 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 18 | blk.1.ffn_down.weight | Block 1 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 19 | blk.1.ffn_gate.weight | Block 1 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 20 | blk.1.ffn_norm.weight | Block 1 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 21 | blk.1.ffn_up.weight | Block 1 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.1: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_2">Block 2 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 22 | blk.2.attn_k.weight | Block 2 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 23 | blk.2.attn_norm.weight | Block 2 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 24 | blk.2.attn_output.weight | Block 2 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 25 | blk.2.attn_q.weight | Block 2 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 26 | blk.2.attn_v.weight | Block 2 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 27 | blk.2.ffn_down.weight | Block 2 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 28 | blk.2.ffn_gate.weight | Block 2 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 29 | blk.2.ffn_norm.weight | Block 2 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 30 | blk.2.ffn_up.weight | Block 2 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.2: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_3">Block 3 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 31 | blk.3.attn_k.weight | Block 3 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 32 | blk.3.attn_norm.weight | Block 3 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 33 | blk.3.attn_output.weight | Block 3 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 34 | blk.3.attn_q.weight | Block 3 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 35 | blk.3.attn_v.weight | Block 3 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 36 | blk.3.ffn_down.weight | Block 3 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 37 | blk.3.ffn_gate.weight | Block 3 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 38 | blk.3.ffn_norm.weight | Block 3 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 39 | blk.3.ffn_up.weight | Block 3 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.3: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_4">Block 4 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 40 | blk.4.attn_k.weight | Block 4 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 41 | blk.4.attn_norm.weight | Block 4 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 42 | blk.4.attn_output.weight | Block 4 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 43 | blk.4.attn_q.weight | Block 4 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 44 | blk.4.attn_v.weight | Block 4 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 45 | blk.4.ffn_down.weight | Block 4 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 46 | blk.4.ffn_gate.weight | Block 4 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 47 | blk.4.ffn_norm.weight | Block 4 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 48 | blk.4.ffn_up.weight | Block 4 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.4: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_5">Block 5 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 49 | blk.5.attn_k.weight | Block 5 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 50 | blk.5.attn_norm.weight | Block 5 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 51 | blk.5.attn_output.weight | Block 5 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 52 | blk.5.attn_q.weight | Block 5 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 53 | blk.5.attn_v.weight | Block 5 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 54 | blk.5.ffn_down.weight | Block 5 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 55 | blk.5.ffn_gate.weight | Block 5 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 56 | blk.5.ffn_norm.weight | Block 5 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 57 | blk.5.ffn_up.weight | Block 5 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.5: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_6">Block 6 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 58 | blk.6.attn_k.weight | Block 6 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 59 | blk.6.attn_norm.weight | Block 6 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 60 | blk.6.attn_output.weight | Block 6 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 61 | blk.6.attn_q.weight | Block 6 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 62 | blk.6.attn_v.weight | Block 6 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 63 | blk.6.ffn_down.weight | Block 6 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 64 | blk.6.ffn_gate.weight | Block 6 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 65 | blk.6.ffn_norm.weight | Block 6 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 66 | blk.6.ffn_up.weight | Block 6 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.6: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_7">Block 7 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 67 | blk.7.attn_k.weight | Block 7 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 68 | blk.7.attn_norm.weight | Block 7 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 69 | blk.7.attn_output.weight | Block 7 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 70 | blk.7.attn_q.weight | Block 7 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 71 | blk.7.attn_v.weight | Block 7 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 72 | blk.7.ffn_down.weight | Block 7 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 73 | blk.7.ffn_gate.weight | Block 7 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 74 | blk.7.ffn_norm.weight | Block 7 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 75 | blk.7.ffn_up.weight | Block 7 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.7: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_8">Block 8 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 76 | blk.8.attn_k.weight | Block 8 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 77 | blk.8.attn_norm.weight | Block 8 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 78 | blk.8.attn_output.weight | Block 8 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 79 | blk.8.attn_q.weight | Block 8 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 80 | blk.8.attn_v.weight | Block 8 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 81 | blk.8.ffn_down.weight | Block 8 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 82 | blk.8.ffn_gate.weight | Block 8 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 83 | blk.8.ffn_norm.weight | Block 8 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 84 | blk.8.ffn_up.weight | Block 8 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.8: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_9">Block 9 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :----------------------- | :--------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 85 | blk.9.attn_k.weight | Block 9 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 86 | blk.9.attn_norm.weight | Block 9 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 87 | blk.9.attn_output.weight | Block 9 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 88 | blk.9.attn_q.weight | Block 9 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 89 | blk.9.attn_v.weight | Block 9 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 90 | blk.9.ffn_down.weight | Block 9 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 91 | blk.9.ffn_gate.weight | Block 9 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 92 | blk.9.ffn_norm.weight | Block 9 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 93 | blk.9.ffn_up.weight | Block 9 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.9: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_10">Block 10 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 94 | blk.10.attn_k.weight | Block 10 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 95 | blk.10.attn_norm.weight | Block 10 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 96 | blk.10.attn_output.weight | Block 10 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 97 | blk.10.attn_q.weight | Block 10 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 98 | blk.10.attn_v.weight | Block 10 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 99 | blk.10.ffn_down.weight | Block 10 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 100 | blk.10.ffn_gate.weight | Block 10 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 101 | blk.10.ffn_norm.weight | Block 10 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 102 | blk.10.ffn_up.weight | Block 10 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.10: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_11">Block 11 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 103 | blk.11.attn_k.weight | Block 11 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 104 | blk.11.attn_norm.weight | Block 11 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 105 | blk.11.attn_output.weight | Block 11 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 106 | blk.11.attn_q.weight | Block 11 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 107 | blk.11.attn_v.weight | Block 11 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 108 | blk.11.ffn_down.weight | Block 11 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 109 | blk.11.ffn_gate.weight | Block 11 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 110 | blk.11.ffn_norm.weight | Block 11 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 111 | blk.11.ffn_up.weight | Block 11 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.11: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_12">Block 12 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 112 | blk.12.attn_k.weight | Block 12 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 113 | blk.12.attn_norm.weight | Block 12 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 114 | blk.12.attn_output.weight | Block 12 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 115 | blk.12.attn_q.weight | Block 12 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 116 | blk.12.attn_v.weight | Block 12 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 117 | blk.12.ffn_down.weight | Block 12 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 118 | blk.12.ffn_gate.weight | Block 12 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 119 | blk.12.ffn_norm.weight | Block 12 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 120 | blk.12.ffn_up.weight | Block 12 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.12: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_13">Block 13 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 121 | blk.13.attn_k.weight | Block 13 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 122 | blk.13.attn_norm.weight | Block 13 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 123 | blk.13.attn_output.weight | Block 13 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 124 | blk.13.attn_q.weight | Block 13 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 125 | blk.13.attn_v.weight | Block 13 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 126 | blk.13.ffn_down.weight | Block 13 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 127 | blk.13.ffn_gate.weight | Block 13 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 128 | blk.13.ffn_norm.weight | Block 13 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 129 | blk.13.ffn_up.weight | Block 13 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.13: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_14">Block 14 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 130 | blk.14.attn_k.weight | Block 14 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 131 | blk.14.attn_norm.weight | Block 14 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 132 | blk.14.attn_output.weight | Block 14 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 133 | blk.14.attn_q.weight | Block 14 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 134 | blk.14.attn_v.weight | Block 14 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 135 | blk.14.ffn_down.weight | Block 14 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 136 | blk.14.ffn_gate.weight | Block 14 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 137 | blk.14.ffn_norm.weight | Block 14 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 138 | blk.14.ffn_up.weight | Block 14 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.14: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_15">Block 15 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 139 | blk.15.attn_k.weight | Block 15 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 140 | blk.15.attn_norm.weight | Block 15 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 141 | blk.15.attn_output.weight | Block 15 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 142 | blk.15.attn_q.weight | Block 15 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q3_K | | |
| | 143 | blk.15.attn_v.weight | Block 15 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q4_K | | |
| | 144 | blk.15.ffn_down.weight | Block 15 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 145 | blk.15.ffn_gate.weight | Block 15 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| | 146 | blk.15.ffn_norm.weight | Block 15 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 147 | blk.15.ffn_up.weight | Block 15 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q3_K | | |
| - Total elements in blk.15: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_16">Block 16 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 148 | blk.16.attn_k.weight | Block 16 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 149 | blk.16.attn_norm.weight | Block 16 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 150 | blk.16.attn_output.weight | Block 16 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 151 | blk.16.attn_q.weight | Block 16 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 152 | blk.16.attn_v.weight | Block 16 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 153 | blk.16.ffn_down.weight | Block 16 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 154 | blk.16.ffn_gate.weight | Block 16 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 155 | blk.16.ffn_norm.weight | Block 16 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 156 | blk.16.ffn_up.weight | Block 16 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.16: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_17">Block 17 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 157 | blk.17.attn_k.weight | Block 17 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 158 | blk.17.attn_norm.weight | Block 17 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 159 | blk.17.attn_output.weight | Block 17 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 160 | blk.17.attn_q.weight | Block 17 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 161 | blk.17.attn_v.weight | Block 17 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 162 | blk.17.ffn_down.weight | Block 17 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 163 | blk.17.ffn_gate.weight | Block 17 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 164 | blk.17.ffn_norm.weight | Block 17 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 165 | blk.17.ffn_up.weight | Block 17 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.17: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_18">Block 18 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 166 | blk.18.attn_k.weight | Block 18 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 167 | blk.18.attn_norm.weight | Block 18 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 168 | blk.18.attn_output.weight | Block 18 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 169 | blk.18.attn_q.weight | Block 18 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 170 | blk.18.attn_v.weight | Block 18 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 171 | blk.18.ffn_down.weight | Block 18 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 172 | blk.18.ffn_gate.weight | Block 18 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 173 | blk.18.ffn_norm.weight | Block 18 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 174 | blk.18.ffn_up.weight | Block 18 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.18: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_19">Block 19 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 175 | blk.19.attn_k.weight | Block 19 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 176 | blk.19.attn_norm.weight | Block 19 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 177 | blk.19.attn_output.weight | Block 19 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 178 | blk.19.attn_q.weight | Block 19 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 179 | blk.19.attn_v.weight | Block 19 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 180 | blk.19.ffn_down.weight | Block 19 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 181 | blk.19.ffn_gate.weight | Block 19 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 182 | blk.19.ffn_norm.weight | Block 19 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 183 | blk.19.ffn_up.weight | Block 19 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.19: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_20">Block 20 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 184 | blk.20.attn_k.weight | Block 20 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 185 | blk.20.attn_norm.weight | Block 20 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 186 | blk.20.attn_output.weight | Block 20 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 187 | blk.20.attn_q.weight | Block 20 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 188 | blk.20.attn_v.weight | Block 20 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 189 | blk.20.ffn_down.weight | Block 20 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 190 | blk.20.ffn_gate.weight | Block 20 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 191 | blk.20.ffn_norm.weight | Block 20 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 192 | blk.20.ffn_up.weight | Block 20 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.20: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_21">Block 21 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 193 | blk.21.attn_k.weight | Block 21 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 194 | blk.21.attn_norm.weight | Block 21 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 195 | blk.21.attn_output.weight | Block 21 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 196 | blk.21.attn_q.weight | Block 21 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 197 | blk.21.attn_v.weight | Block 21 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 198 | blk.21.ffn_down.weight | Block 21 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 199 | blk.21.ffn_gate.weight | Block 21 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 200 | blk.21.ffn_norm.weight | Block 21 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 201 | blk.21.ffn_up.weight | Block 21 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.21: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_22">Block 22 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 202 | blk.22.attn_k.weight | Block 22 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 203 | blk.22.attn_norm.weight | Block 22 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 204 | blk.22.attn_output.weight | Block 22 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 205 | blk.22.attn_q.weight | Block 22 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 206 | blk.22.attn_v.weight | Block 22 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 207 | blk.22.ffn_down.weight | Block 22 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 208 | blk.22.ffn_gate.weight | Block 22 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 209 | blk.22.ffn_norm.weight | Block 22 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 210 | blk.22.ffn_up.weight | Block 22 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.22: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_23">Block 23 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 211 | blk.23.attn_k.weight | Block 23 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 212 | blk.23.attn_norm.weight | Block 23 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 213 | blk.23.attn_output.weight | Block 23 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 214 | blk.23.attn_q.weight | Block 23 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 215 | blk.23.attn_v.weight | Block 23 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 216 | blk.23.ffn_down.weight | Block 23 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 217 | blk.23.ffn_gate.weight | Block 23 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 218 | blk.23.ffn_norm.weight | Block 23 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 219 | blk.23.ffn_up.weight | Block 23 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.23: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_24">Block 24 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 220 | blk.24.attn_k.weight | Block 24 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 221 | blk.24.attn_norm.weight | Block 24 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 222 | blk.24.attn_output.weight | Block 24 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 223 | blk.24.attn_q.weight | Block 24 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 224 | blk.24.attn_v.weight | Block 24 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 225 | blk.24.ffn_down.weight | Block 24 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 226 | blk.24.ffn_gate.weight | Block 24 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 227 | blk.24.ffn_norm.weight | Block 24 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 228 | blk.24.ffn_up.weight | Block 24 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.24: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_25">Block 25 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 229 | blk.25.attn_k.weight | Block 25 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 230 | blk.25.attn_norm.weight | Block 25 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 231 | blk.25.attn_output.weight | Block 25 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 232 | blk.25.attn_q.weight | Block 25 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 233 | blk.25.attn_v.weight | Block 25 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 234 | blk.25.ffn_down.weight | Block 25 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 235 | blk.25.ffn_gate.weight | Block 25 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 236 | blk.25.ffn_norm.weight | Block 25 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 237 | blk.25.ffn_up.weight | Block 25 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.25: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_26">Block 26 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 238 | blk.26.attn_k.weight | Block 26 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 239 | blk.26.attn_norm.weight | Block 26 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 240 | blk.26.attn_output.weight | Block 26 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 241 | blk.26.attn_q.weight | Block 26 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 242 | blk.26.attn_v.weight | Block 26 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 243 | blk.26.ffn_down.weight | Block 26 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 244 | blk.26.ffn_gate.weight | Block 26 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 245 | blk.26.ffn_norm.weight | Block 26 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 246 | blk.26.ffn_up.weight | Block 26 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.26: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_27">Block 27 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 247 | blk.27.attn_k.weight | Block 27 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 248 | blk.27.attn_norm.weight | Block 27 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 249 | blk.27.attn_output.weight | Block 27 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 250 | blk.27.attn_q.weight | Block 27 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 251 | blk.27.attn_v.weight | Block 27 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 252 | blk.27.ffn_down.weight | Block 27 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 253 | blk.27.ffn_gate.weight | Block 27 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 254 | blk.27.ffn_norm.weight | Block 27 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 255 | blk.27.ffn_up.weight | Block 27 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.27: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_28">Block 28 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 256 | blk.28.attn_k.weight | Block 28 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 257 | blk.28.attn_norm.weight | Block 28 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 258 | blk.28.attn_output.weight | Block 28 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 259 | blk.28.attn_q.weight | Block 28 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 260 | blk.28.attn_v.weight | Block 28 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 261 | blk.28.ffn_down.weight | Block 28 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 262 | blk.28.ffn_gate.weight | Block 28 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 263 | blk.28.ffn_norm.weight | Block 28 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 264 | blk.28.ffn_up.weight | Block 28 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.28: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_29">Block 29 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 265 | blk.29.attn_k.weight | Block 29 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 266 | blk.29.attn_norm.weight | Block 29 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 267 | blk.29.attn_output.weight | Block 29 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 268 | blk.29.attn_q.weight | Block 29 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 269 | blk.29.attn_v.weight | Block 29 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 270 | blk.29.ffn_down.weight | Block 29 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 271 | blk.29.ffn_gate.weight | Block 29 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 272 | blk.29.ffn_norm.weight | Block 29 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 273 | blk.29.ffn_up.weight | Block 29 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.29: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_30">Block 30 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 274 | blk.30.attn_k.weight | Block 30 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 275 | blk.30.attn_norm.weight | Block 30 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 276 | blk.30.attn_output.weight | Block 30 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 277 | blk.30.attn_q.weight | Block 30 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 278 | blk.30.attn_v.weight | Block 30 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 279 | blk.30.ffn_down.weight | Block 30 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 280 | blk.30.ffn_gate.weight | Block 30 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 281 | blk.30.ffn_norm.weight | Block 30 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 282 | blk.30.ffn_up.weight | Block 30 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.30: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |
| ### <a name="blk_31">Block 31 Tensor Group : ~218M Elements</a> | |
| | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | | |
| | ---: | :------------------------ | :---------------------------------------------- | :-------------- | :-------------------- | :--- | | |
| | 283 | blk.31.attn_k.weight | Block 31 Attention Key (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q2_K | | |
| | 284 | blk.31.attn_norm.weight | Block 31 Attention Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 285 | blk.31.attn_output.weight | Block 31 Attention Output (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q4_K | | |
| | 286 | blk.31.attn_q.weight | Block 31 Attention Query (W) | (~17M) 16777216 | 4096 x 4096 x 1 x 1 | Q2_K | | |
| | 287 | blk.31.attn_v.weight | Block 31 Attention Value (W) | ( ~4M) 4194304 | 4096 x 1024 x 1 x 1 | Q3_K | | |
| | 288 | blk.31.ffn_down.weight | Block 31 Feed-Forward Network "Down" (W) | (~59M) 58720256 | 14336 x 4096 x 1 x 1 | Q4_K | | |
| | 289 | blk.31.ffn_gate.weight | Block 31 Feed-Forward Network "Gate" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| | 290 | blk.31.ffn_norm.weight | Block 31 Feed-Forward Network Normalization (W) | ( ~4K) 4096 | 4096 x 1 x 1 x 1 | F32 | | |
| | 291 | blk.31.ffn_up.weight | Block 31 Feed-Forward Network "Up" (W) | (~59M) 58720256 | 4096 x 14336 x 1 x 1 | Q2_K | | |
| - Total elements in blk.31: (~218M) 218112000 | |
| - Percentage of total elements: 2.72% | |