diff --git "a/scores/Qwen3-30B-A3B-pruned-F16.md" "b/scores/Qwen3-30B-A3B-pruned-F16.md"
new file mode 100644--- /dev/null
+++ "b/scores/Qwen3-30B-A3B-pruned-F16.md"
@@ -0,0 +1,1719 @@
+# Qwen3-30B-A3B-F16.gguf - GGUF Internal File Dump
+
+- Endian: LITTLE endian
+
+## Key Value Metadata Store
+
+There are 43 key-value pairs in this file
+
+| POS | TYPE | Count | Key | Value |
+|----:|:---------|-------:|:------------------------------------------|:--------------------------------------------------------------------|
+| 1 | UINT32 | 1 | GGUF.version | 3 |
+| 2 | UINT64 | 1 | GGUF.tensor_count | 579 |
+| 3 | UINT64 | 1 | GGUF.kv_count | 40 |
+| 4 | STRING | 1 | general.architecture | `qwen3moe` |
+| 5 | STRING | 1 | general.type | `model` |
+| 6 | STRING | 1 | general.name | `Qwen3 30B A3B` |
+| 7 | STRING | 1 | general.basename | `Qwen3` |
+| 8 | STRING | 1 | general.size_label | `30B-A3B` |
+| 9 | STRING | 1 | general.license | `apache-2.0` |
+| 10 | STRING | 1 | general.license.link | `https://huggingface.co/Qwen/Qwen3-30B-A3B/blob/main/LICENSE` |
+| 11 | UINT32 | 1 | general.base_model.count | 1 |
+| 12 | STRING | 1 | general.base_model.0.name | `Qwen3 30B A3B Base` |
+| 13 | STRING | 1 | general.base_model.0.organization | `Qwen` |
+| 14 | STRING | 1 | general.base_model.0.repo_url | `https://huggingface.co/Qwen/Qwen3-30B-A3B-Base` |
+| 15 | [STRING] | 1 | general.tags | [ `text-generation` ] |
+| 16 | UINT32 | 1 | qwen3moe.block_count | 48 |
+| 17 | UINT32 | 1 | qwen3moe.context_length | 40960 |
+| 18 | UINT32 | 1 | qwen3moe.embedding_length | 2048 |
+| 19 | UINT32 | 1 | qwen3moe.feed_forward_length | 6144 |
+| 20 | UINT32 | 1 | qwen3moe.attention.head_count | 32 |
+| 21 | UINT32 | 1 | qwen3moe.attention.head_count_kv | 4 |
+| 22 | FLOAT32 | 1 | qwen3moe.rope.freq_base | 1000000.0 |
+| 23 | FLOAT32 | 1 | qwen3moe.attention.layer_norm_rms_epsilon | 1e-06 |
+| 24 | UINT32 | 1 | qwen3moe.expert_used_count | 8 |
+| 25 | UINT32 | 1 | qwen3moe.attention.key_length | 128 |
+| 26 | UINT32 | 1 | qwen3moe.attention.value_length | 128 |
+| 27 | UINT32 | 1 | general.file_type | 1 |
+| 28 | UINT32 | 1 | qwen3moe.expert_count | 128 |
+| 29 | UINT32 | 1 | qwen3moe.expert_feed_forward_length | 768 |
+| 30 | UINT32 | 1 | general.quantization_version | 2 |
+| 31 | STRING | 1 | tokenizer.ggml.model | `gpt2` |
+| 32 | STRING | 1 | tokenizer.ggml.pre | `qwen2` |
+| 33 | [STRING] | 151936 | tokenizer.ggml.tokens | [ `!`, `"`, `#`, `$`, `%`, ... ] |
+| 34 | [INT32] | 151936 | tokenizer.ggml.token_type | [ 1, 1, 1, 1, 1, 1, 1, ... ] |
+| 35 | [STRING] | 151387 | tokenizer.ggml.merges | [ `Ġ Ġ`, `ĠĠ ĠĠ`, `i n`, `Ġ t`, `ĠĠĠĠ ĠĠĠĠ`, ... ] |
+| 36 | UINT32 | 1 | tokenizer.ggml.eos_token_id | 151645 |
+| 37 | UINT32 | 1 | tokenizer.ggml.padding_token_id | 151643 |
+| 38 | UINT32 | 1 | tokenizer.ggml.bos_token_id | 151643 |
+| 39 | BOOL | 1 | tokenizer.ggml.add_bos_token | False |
+| 40 | STRING | 1 | tokenizer.chat_template | `{%- if tools %}{{- '<|im_`...`{%- endif %}{%- endif %}` |
+| 41 | UINT16 | 1 | split.no | 0 |
+| 42 | INT32 | 1 | split.tensors.count | 579 |
+| 43 | UINT16 | 1 | split.count | 0 |
+
+## Tensors Overview ~31B Elements
+
+Total number of elements in all tensors: 30532122624 Elements
+
+- [Qwen3-30B-A3B-F16.gguf - GGUF Internal File Dump](#qwen3-30b-a3b-f16gguf---gguf-internal-file-dump)
+ - [Key Value Metadata Store](#key-value-metadata-store)
+ - [Tensors Overview ~31B Elements](#tensors-overview-31b-elements)
+ - [Tensor Data Offset](#tensor-data-offset)
+ - [Base Tensor Group : ~622M Elements](#base-tensor-group--622m-elements)
+ - [Block 0 Tensor Group : ~623M Elements](#block-0-tensor-group--623m-elements)
+ - [Block 1 Tensor Group : ~623M Elements](#block-1-tensor-group--623m-elements)
+ - [Block 2 Tensor Group : ~623M Elements](#block-2-tensor-group--623m-elements)
+ - [Block 3 Tensor Group : ~623M Elements](#block-3-tensor-group--623m-elements)
+ - [Block 4 Tensor Group : ~623M Elements](#block-4-tensor-group--623m-elements)
+ - [Block 5 Tensor Group : ~623M Elements](#block-5-tensor-group--623m-elements)
+ - [Block 6 Tensor Group : ~623M Elements](#block-6-tensor-group--623m-elements)
+ - [Block 7 Tensor Group : ~623M Elements](#block-7-tensor-group--623m-elements)
+ - [Block 8 Tensor Group : ~623M Elements](#block-8-tensor-group--623m-elements)
+ - [Block 9 Tensor Group : ~623M Elements](#block-9-tensor-group--623m-elements)
+ - [Block 10 Tensor Group : ~623M Elements](#block-10-tensor-group--623m-elements)
+ - [Block 11 Tensor Group : ~623M Elements](#block-11-tensor-group--623m-elements)
+ - [Block 12 Tensor Group : ~623M Elements](#block-12-tensor-group--623m-elements)
+ - [Block 13 Tensor Group : ~623M Elements](#block-13-tensor-group--623m-elements)
+ - [Block 14 Tensor Group : ~623M Elements](#block-14-tensor-group--623m-elements)
+ - [Block 15 Tensor Group : ~623M Elements](#block-15-tensor-group--623m-elements)
+ - [Block 16 Tensor Group : ~623M Elements](#block-16-tensor-group--623m-elements)
+ - [Block 17 Tensor Group : ~623M Elements](#block-17-tensor-group--623m-elements)
+ - [Block 18 Tensor Group : ~623M Elements](#block-18-tensor-group--623m-elements)
+ - [Block 19 Tensor Group : ~623M Elements](#block-19-tensor-group--623m-elements)
+ - [Block 20 Tensor Group : ~623M Elements](#block-20-tensor-group--623m-elements)
+ - [Block 21 Tensor Group : ~623M Elements](#block-21-tensor-group--623m-elements)
+ - [Block 22 Tensor Group : ~623M Elements](#block-22-tensor-group--623m-elements)
+ - [Block 23 Tensor Group : ~623M Elements](#block-23-tensor-group--623m-elements)
+ - [Block 24 Tensor Group : ~623M Elements](#block-24-tensor-group--623m-elements)
+ - [Block 25 Tensor Group : ~623M Elements](#block-25-tensor-group--623m-elements)
+ - [Block 26 Tensor Group : ~623M Elements](#block-26-tensor-group--623m-elements)
+ - [Block 27 Tensor Group : ~623M Elements](#block-27-tensor-group--623m-elements)
+ - [Block 28 Tensor Group : ~623M Elements](#block-28-tensor-group--623m-elements)
+ - [Block 29 Tensor Group : ~623M Elements](#block-29-tensor-group--623m-elements)
+ - [Block 30 Tensor Group : ~623M Elements](#block-30-tensor-group--623m-elements)
+ - [Block 31 Tensor Group : ~623M Elements](#block-31-tensor-group--623m-elements)
+ - [Block 32 Tensor Group : ~623M Elements](#block-32-tensor-group--623m-elements)
+ - [Block 33 Tensor Group : ~623M Elements](#block-33-tensor-group--623m-elements)
+ - [Block 34 Tensor Group : ~623M Elements](#block-34-tensor-group--623m-elements)
+ - [Block 35 Tensor Group : ~623M Elements](#block-35-tensor-group--623m-elements)
+ - [Block 36 Tensor Group : ~623M Elements](#block-36-tensor-group--623m-elements)
+ - [Block 37 Tensor Group : ~623M Elements](#block-37-tensor-group--623m-elements)
+ - [Block 38 Tensor Group : ~623M Elements](#block-38-tensor-group--623m-elements)
+ - [Block 39 Tensor Group : ~623M Elements](#block-39-tensor-group--623m-elements)
+ - [Block 40 Tensor Group : ~623M Elements](#block-40-tensor-group--623m-elements)
+ - [Block 41 Tensor Group : ~623M Elements](#block-41-tensor-group--623m-elements)
+ - [Block 42 Tensor Group : ~623M Elements](#block-42-tensor-group--623m-elements)
+ - [Block 43 Tensor Group : ~623M Elements](#block-43-tensor-group--623m-elements)
+ - [Block 44 Tensor Group : ~623M Elements](#block-44-tensor-group--623m-elements)
+ - [Block 45 Tensor Group : ~623M Elements](#block-45-tensor-group--623m-elements)
+ - [Block 46 Tensor Group : ~623M Elements](#block-46-tensor-group--623m-elements)
+ - [Block 47 Tensor Group : ~623M Elements](#block-47-tensor-group--623m-elements)
+
+### Tensor Data Offset
+
+This table contains the offset and data segment relative to start of file
+
+| T_ID | Tensor Layer Name | Data Offset (B) | Data Size (B) |
+|-----:|:----------------------------|-----------------:|-----------------:|
+| 0 | token_embd.weight | 0x5b1800 | 0x25180000 |
+| 1 | blk.0.attn_norm.weight | 0x25731800 | 0x2000 |
+| 2 | blk.0.ffn_down_exps.weight | 0x25733800 | 0x18000000 |
+| 3 | blk.0.ffn_gate_exps.weight | 0x3d733800 | 0x18000000 |
+| 4 | blk.0.ffn_up_exps.weight | 0x55733800 | 0x18000000 |
+| 5 | blk.0.ffn_gate_inp.weight | 0x6d733800 | 0x100000 |
+| 6 | blk.0.ffn_norm.weight | 0x6d833800 | 0x2000 |
+| 7 | blk.0.attn_k_norm.weight | 0x6d835800 | 0x200 |
+| 8 | blk.0.attn_k.weight | 0x6d835a00 | 0x200000 |
+| 9 | blk.0.attn_output.weight | 0x6da35a00 | 0x1000000 |
+| 10 | blk.0.attn_q_norm.weight | 0x6ea35a00 | 0x200 |
+| 11 | blk.0.attn_q.weight | 0x6ea35c00 | 0x1000000 |
+| 12 | blk.0.attn_v.weight | 0x6fa35c00 | 0x200000 |
+| 13 | blk.1.attn_norm.weight | 0x6fc35c00 | 0x2000 |
+| 14 | blk.1.ffn_down_exps.weight | 0x6fc37c00 | 0x18000000 |
+| 15 | blk.1.ffn_gate_exps.weight | 0x87c37c00 | 0x18000000 |
+| 16 | blk.1.ffn_up_exps.weight | 0x9fc37c00 | 0x18000000 |
+| 17 | blk.1.ffn_gate_inp.weight | 0xb7c37c00 | 0x100000 |
+| 18 | blk.1.ffn_norm.weight | 0xb7d37c00 | 0x2000 |
+| 19 | blk.1.attn_k_norm.weight | 0xb7d39c00 | 0x200 |
+| 20 | blk.1.attn_k.weight | 0xb7d39e00 | 0x200000 |
+| 21 | blk.1.attn_output.weight | 0xb7f39e00 | 0x1000000 |
+| 22 | blk.1.attn_q_norm.weight | 0xb8f39e00 | 0x200 |
+| 23 | blk.1.attn_q.weight | 0xb8f3a000 | 0x1000000 |
+| 24 | blk.1.attn_v.weight | 0xb9f3a000 | 0x200000 |
+| 25 | blk.2.ffn_gate_inp.weight | 0xba13a000 | 0x100000 |
+| 26 | blk.2.attn_k_norm.weight | 0xba23a000 | 0x200 |
+| 27 | blk.2.attn_k.weight | 0xba23a200 | 0x200000 |
+| 28 | blk.2.attn_output.weight | 0xba43a200 | 0x1000000 |
+| 29 | blk.2.attn_q_norm.weight | 0xbb43a200 | 0x200 |
+| 30 | blk.2.attn_q.weight | 0xbb43a400 | 0x1000000 |
+| 31 | blk.2.attn_v.weight | 0xbc43a400 | 0x200000 |
+| 32 | blk.2.attn_norm.weight | 0xbc63a400 | 0x2000 |
+| 33 | blk.2.ffn_down_exps.weight | 0xbc63c400 | 0x18000000 |
+| 34 | blk.2.ffn_gate_exps.weight | 0xd463c400 | 0x18000000 |
+| 35 | blk.2.ffn_up_exps.weight | 0xec63c400 | 0x18000000 |
+| 36 | blk.2.ffn_norm.weight | 0x10463c400 | 0x2000 |
+| 37 | blk.3.attn_norm.weight | 0x10463e400 | 0x2000 |
+| 38 | blk.3.ffn_down_exps.weight | 0x104640400 | 0x18000000 |
+| 39 | blk.3.ffn_gate_exps.weight | 0x11c640400 | 0x18000000 |
+| 40 | blk.3.ffn_up_exps.weight | 0x134640400 | 0x18000000 |
+| 41 | blk.3.ffn_gate_inp.weight | 0x14c640400 | 0x100000 |
+| 42 | blk.3.ffn_norm.weight | 0x14c740400 | 0x2000 |
+| 43 | blk.3.attn_k_norm.weight | 0x14c742400 | 0x200 |
+| 44 | blk.3.attn_k.weight | 0x14c742600 | 0x200000 |
+| 45 | blk.3.attn_output.weight | 0x14c942600 | 0x1000000 |
+| 46 | blk.3.attn_q_norm.weight | 0x14d942600 | 0x200 |
+| 47 | blk.3.attn_q.weight | 0x14d942800 | 0x1000000 |
+| 48 | blk.3.attn_v.weight | 0x14e942800 | 0x200000 |
+| 49 | blk.4.attn_norm.weight | 0x14eb42800 | 0x2000 |
+| 50 | blk.4.ffn_down_exps.weight | 0x14eb44800 | 0x18000000 |
+| 51 | blk.4.ffn_gate_exps.weight | 0x166b44800 | 0x18000000 |
+| 52 | blk.4.ffn_up_exps.weight | 0x17eb44800 | 0x18000000 |
+| 53 | blk.4.ffn_gate_inp.weight | 0x196b44800 | 0x100000 |
+| 54 | blk.4.ffn_norm.weight | 0x196c44800 | 0x2000 |
+| 55 | blk.4.attn_k_norm.weight | 0x196c46800 | 0x200 |
+| 56 | blk.4.attn_k.weight | 0x196c46a00 | 0x200000 |
+| 57 | blk.4.attn_output.weight | 0x196e46a00 | 0x1000000 |
+| 58 | blk.4.attn_q_norm.weight | 0x197e46a00 | 0x200 |
+| 59 | blk.4.attn_q.weight | 0x197e46c00 | 0x1000000 |
+| 60 | blk.4.attn_v.weight | 0x198e46c00 | 0x200000 |
+| 61 | blk.5.ffn_gate_inp.weight | 0x199046c00 | 0x100000 |
+| 62 | blk.5.attn_k_norm.weight | 0x199146c00 | 0x200 |
+| 63 | blk.5.attn_k.weight | 0x199146e00 | 0x200000 |
+| 64 | blk.5.attn_output.weight | 0x199346e00 | 0x1000000 |
+| 65 | blk.5.attn_q_norm.weight | 0x19a346e00 | 0x200 |
+| 66 | blk.5.attn_q.weight | 0x19a347000 | 0x1000000 |
+| 67 | blk.5.attn_v.weight | 0x19b347000 | 0x200000 |
+| 68 | blk.5.attn_norm.weight | 0x19b547000 | 0x2000 |
+| 69 | blk.5.ffn_down_exps.weight | 0x19b549000 | 0x18000000 |
+| 70 | blk.5.ffn_gate_exps.weight | 0x1b3549000 | 0x18000000 |
+| 71 | blk.5.ffn_up_exps.weight | 0x1cb549000 | 0x18000000 |
+| 72 | blk.5.ffn_norm.weight | 0x1e3549000 | 0x2000 |
+| 73 | blk.6.attn_norm.weight | 0x1e354b000 | 0x2000 |
+| 74 | blk.6.ffn_down_exps.weight | 0x1e354d000 | 0x18000000 |
+| 75 | blk.6.ffn_gate_exps.weight | 0x1fb54d000 | 0x18000000 |
+| 76 | blk.6.ffn_up_exps.weight | 0x21354d000 | 0x18000000 |
+| 77 | blk.6.ffn_gate_inp.weight | 0x22b54d000 | 0x100000 |
+| 78 | blk.6.ffn_norm.weight | 0x22b64d000 | 0x2000 |
+| 79 | blk.6.attn_k_norm.weight | 0x22b64f000 | 0x200 |
+| 80 | blk.6.attn_k.weight | 0x22b64f200 | 0x200000 |
+| 81 | blk.6.attn_output.weight | 0x22b84f200 | 0x1000000 |
+| 82 | blk.6.attn_q_norm.weight | 0x22c84f200 | 0x200 |
+| 83 | blk.6.attn_q.weight | 0x22c84f400 | 0x1000000 |
+| 84 | blk.6.attn_v.weight | 0x22d84f400 | 0x200000 |
+| 85 | blk.7.attn_norm.weight | 0x22da4f400 | 0x2000 |
+| 86 | blk.7.ffn_down_exps.weight | 0x22da51400 | 0x18000000 |
+| 87 | blk.7.ffn_gate_exps.weight | 0x245a51400 | 0x18000000 |
+| 88 | blk.7.ffn_up_exps.weight | 0x25da51400 | 0x18000000 |
+| 89 | blk.7.ffn_gate_inp.weight | 0x275a51400 | 0x100000 |
+| 90 | blk.7.ffn_norm.weight | 0x275b51400 | 0x2000 |
+| 91 | blk.7.attn_k_norm.weight | 0x275b53400 | 0x200 |
+| 92 | blk.7.attn_k.weight | 0x275b53600 | 0x200000 |
+| 93 | blk.7.attn_output.weight | 0x275d53600 | 0x1000000 |
+| 94 | blk.7.attn_q_norm.weight | 0x276d53600 | 0x200 |
+| 95 | blk.7.attn_q.weight | 0x276d53800 | 0x1000000 |
+| 96 | blk.7.attn_v.weight | 0x277d53800 | 0x200000 |
+| 97 | blk.8.attn_norm.weight | 0x277f53800 | 0x2000 |
+| 98 | blk.8.ffn_down_exps.weight | 0x277f55800 | 0x18000000 |
+| 99 | blk.8.ffn_gate_exps.weight | 0x28ff55800 | 0x18000000 |
+| 100 | blk.8.ffn_up_exps.weight | 0x2a7f55800 | 0x18000000 |
+| 101 | blk.8.ffn_gate_inp.weight | 0x2bff55800 | 0x100000 |
+| 102 | blk.8.ffn_norm.weight | 0x2c0055800 | 0x2000 |
+| 103 | blk.8.attn_k_norm.weight | 0x2c0057800 | 0x200 |
+| 104 | blk.8.attn_k.weight | 0x2c0057a00 | 0x200000 |
+| 105 | blk.8.attn_output.weight | 0x2c0257a00 | 0x1000000 |
+| 106 | blk.8.attn_q_norm.weight | 0x2c1257a00 | 0x200 |
+| 107 | blk.8.attn_q.weight | 0x2c1257c00 | 0x1000000 |
+| 108 | blk.8.attn_v.weight | 0x2c2257c00 | 0x200000 |
+| 109 | blk.9.ffn_gate_inp.weight | 0x2c2457c00 | 0x100000 |
+| 110 | blk.9.attn_k_norm.weight | 0x2c2557c00 | 0x200 |
+| 111 | blk.9.attn_k.weight | 0x2c2557e00 | 0x200000 |
+| 112 | blk.9.attn_output.weight | 0x2c2757e00 | 0x1000000 |
+| 113 | blk.9.attn_q_norm.weight | 0x2c3757e00 | 0x200 |
+| 114 | blk.9.attn_q.weight | 0x2c3758000 | 0x1000000 |
+| 115 | blk.9.attn_v.weight | 0x2c4758000 | 0x200000 |
+| 116 | blk.10.attn_norm.weight | 0x2c4958000 | 0x2000 |
+| 117 | blk.10.ffn_down_exps.weight | 0x2c495a000 | 0x18000000 |
+| 118 | blk.10.ffn_gate_exps.weight | 0x2dc95a000 | 0x18000000 |
+| 119 | blk.10.ffn_up_exps.weight | 0x2f495a000 | 0x18000000 |
+| 120 | blk.10.ffn_gate_inp.weight | 0x30c95a000 | 0x100000 |
+| 121 | blk.10.ffn_norm.weight | 0x30ca5a000 | 0x2000 |
+| 122 | blk.10.attn_k_norm.weight | 0x30ca5c000 | 0x200 |
+| 123 | blk.10.attn_k.weight | 0x30ca5c200 | 0x200000 |
+| 124 | blk.10.attn_output.weight | 0x30cc5c200 | 0x1000000 |
+| 125 | blk.10.attn_q_norm.weight | 0x30dc5c200 | 0x200 |
+| 126 | blk.10.attn_q.weight | 0x30dc5c400 | 0x1000000 |
+| 127 | blk.10.attn_v.weight | 0x30ec5c400 | 0x200000 |
+| 128 | blk.11.attn_norm.weight | 0x30ee5c400 | 0x2000 |
+| 129 | blk.11.ffn_down_exps.weight | 0x30ee5e400 | 0x18000000 |
+| 130 | blk.11.ffn_gate_exps.weight | 0x326e5e400 | 0x18000000 |
+| 131 | blk.11.ffn_up_exps.weight | 0x33ee5e400 | 0x18000000 |
+| 132 | blk.11.ffn_gate_inp.weight | 0x356e5e400 | 0x100000 |
+| 133 | blk.11.ffn_norm.weight | 0x356f5e400 | 0x2000 |
+| 134 | blk.11.attn_k_norm.weight | 0x356f60400 | 0x200 |
+| 135 | blk.11.attn_k.weight | 0x356f60600 | 0x200000 |
+| 136 | blk.11.attn_output.weight | 0x357160600 | 0x1000000 |
+| 137 | blk.11.attn_q_norm.weight | 0x358160600 | 0x200 |
+| 138 | blk.11.attn_q.weight | 0x358160800 | 0x1000000 |
+| 139 | blk.11.attn_v.weight | 0x359160800 | 0x200000 |
+| 140 | blk.12.ffn_gate_inp.weight | 0x359360800 | 0x100000 |
+| 141 | blk.12.attn_k_norm.weight | 0x359460800 | 0x200 |
+| 142 | blk.12.attn_k.weight | 0x359460a00 | 0x200000 |
+| 143 | blk.12.attn_output.weight | 0x359660a00 | 0x1000000 |
+| 144 | blk.12.attn_q_norm.weight | 0x35a660a00 | 0x200 |
+| 145 | blk.12.attn_q.weight | 0x35a660c00 | 0x1000000 |
+| 146 | blk.12.attn_v.weight | 0x35b660c00 | 0x200000 |
+| 147 | blk.9.attn_norm.weight | 0x35b860c00 | 0x2000 |
+| 148 | blk.9.ffn_down_exps.weight | 0x35b862c00 | 0x18000000 |
+| 149 | blk.9.ffn_gate_exps.weight | 0x373862c00 | 0x18000000 |
+| 150 | blk.9.ffn_up_exps.weight | 0x38b862c00 | 0x18000000 |
+| 151 | blk.9.ffn_norm.weight | 0x3a3862c00 | 0x2000 |
+| 152 | blk.12.attn_norm.weight | 0x3a3864c00 | 0x2000 |
+| 153 | blk.12.ffn_down_exps.weight | 0x3a3866c00 | 0x18000000 |
+| 154 | blk.12.ffn_gate_exps.weight | 0x3bb866c00 | 0x18000000 |
+| 155 | blk.12.ffn_up_exps.weight | 0x3d3866c00 | 0x18000000 |
+| 156 | blk.12.ffn_norm.weight | 0x3eb866c00 | 0x2000 |
+| 157 | blk.13.attn_norm.weight | 0x3eb868c00 | 0x2000 |
+| 158 | blk.13.ffn_down_exps.weight | 0x3eb86ac00 | 0x18000000 |
+| 159 | blk.13.ffn_gate_exps.weight | 0x40386ac00 | 0x18000000 |
+| 160 | blk.13.ffn_up_exps.weight | 0x41b86ac00 | 0x18000000 |
+| 161 | blk.13.ffn_gate_inp.weight | 0x43386ac00 | 0x100000 |
+| 162 | blk.13.ffn_norm.weight | 0x43396ac00 | 0x2000 |
+| 163 | blk.13.attn_k_norm.weight | 0x43396cc00 | 0x200 |
+| 164 | blk.13.attn_k.weight | 0x43396ce00 | 0x200000 |
+| 165 | blk.13.attn_output.weight | 0x433b6ce00 | 0x1000000 |
+| 166 | blk.13.attn_q_norm.weight | 0x434b6ce00 | 0x200 |
+| 167 | blk.13.attn_q.weight | 0x434b6d000 | 0x1000000 |
+| 168 | blk.13.attn_v.weight | 0x435b6d000 | 0x200000 |
+| 169 | blk.14.attn_norm.weight | 0x435d6d000 | 0x2000 |
+| 170 | blk.14.ffn_down_exps.weight | 0x435d6f000 | 0x18000000 |
+| 171 | blk.14.ffn_gate_exps.weight | 0x44dd6f000 | 0x18000000 |
+| 172 | blk.14.ffn_up_exps.weight | 0x465d6f000 | 0x18000000 |
+| 173 | blk.14.ffn_gate_inp.weight | 0x47dd6f000 | 0x100000 |
+| 174 | blk.14.ffn_norm.weight | 0x47de6f000 | 0x2000 |
+| 175 | blk.14.attn_k_norm.weight | 0x47de71000 | 0x200 |
+| 176 | blk.14.attn_k.weight | 0x47de71200 | 0x200000 |
+| 177 | blk.14.attn_output.weight | 0x47e071200 | 0x1000000 |
+| 178 | blk.14.attn_q_norm.weight | 0x47f071200 | 0x200 |
+| 179 | blk.14.attn_q.weight | 0x47f071400 | 0x1000000 |
+| 180 | blk.14.attn_v.weight | 0x480071400 | 0x200000 |
+| 181 | blk.15.ffn_gate_inp.weight | 0x480271400 | 0x100000 |
+| 182 | blk.15.attn_k_norm.weight | 0x480371400 | 0x200 |
+| 183 | blk.15.attn_k.weight | 0x480371600 | 0x200000 |
+| 184 | blk.15.attn_output.weight | 0x480571600 | 0x1000000 |
+| 185 | blk.15.attn_q_norm.weight | 0x481571600 | 0x200 |
+| 186 | blk.15.attn_q.weight | 0x481571800 | 0x1000000 |
+| 187 | blk.15.attn_v.weight | 0x482571800 | 0x200000 |
+| 188 | blk.15.attn_norm.weight | 0x482771800 | 0x2000 |
+| 189 | blk.15.ffn_down_exps.weight | 0x482773800 | 0x18000000 |
+| 190 | blk.15.ffn_gate_exps.weight | 0x49a773800 | 0x18000000 |
+| 191 | blk.15.ffn_up_exps.weight | 0x4b2773800 | 0x18000000 |
+| 192 | blk.15.ffn_norm.weight | 0x4ca773800 | 0x2000 |
+| 193 | blk.16.attn_norm.weight | 0x4ca775800 | 0x2000 |
+| 194 | blk.16.ffn_down_exps.weight | 0x4ca777800 | 0x18000000 |
+| 195 | blk.16.ffn_gate_exps.weight | 0x4e2777800 | 0x18000000 |
+| 196 | blk.16.ffn_up_exps.weight | 0x4fa777800 | 0x18000000 |
+| 197 | blk.16.ffn_gate_inp.weight | 0x512777800 | 0x100000 |
+| 198 | blk.16.ffn_norm.weight | 0x512877800 | 0x2000 |
+| 199 | blk.16.attn_k_norm.weight | 0x512879800 | 0x200 |
+| 200 | blk.16.attn_k.weight | 0x512879a00 | 0x200000 |
+| 201 | blk.16.attn_output.weight | 0x512a79a00 | 0x1000000 |
+| 202 | blk.16.attn_q_norm.weight | 0x513a79a00 | 0x200 |
+| 203 | blk.16.attn_q.weight | 0x513a79c00 | 0x1000000 |
+| 204 | blk.16.attn_v.weight | 0x514a79c00 | 0x200000 |
+| 205 | blk.17.attn_norm.weight | 0x514c79c00 | 0x2000 |
+| 206 | blk.17.ffn_down_exps.weight | 0x514c7bc00 | 0x18000000 |
+| 207 | blk.17.ffn_gate_exps.weight | 0x52cc7bc00 | 0x18000000 |
+| 208 | blk.17.ffn_up_exps.weight | 0x544c7bc00 | 0x18000000 |
+| 209 | blk.17.ffn_gate_inp.weight | 0x55cc7bc00 | 0x100000 |
+| 210 | blk.17.ffn_norm.weight | 0x55cd7bc00 | 0x2000 |
+| 211 | blk.17.attn_k_norm.weight | 0x55cd7dc00 | 0x200 |
+| 212 | blk.17.attn_k.weight | 0x55cd7de00 | 0x200000 |
+| 213 | blk.17.attn_output.weight | 0x55cf7de00 | 0x1000000 |
+| 214 | blk.17.attn_q_norm.weight | 0x55df7de00 | 0x200 |
+| 215 | blk.17.attn_q.weight | 0x55df7e000 | 0x1000000 |
+| 216 | blk.17.attn_v.weight | 0x55ef7e000 | 0x200000 |
+| 217 | blk.18.ffn_gate_inp.weight | 0x55f17e000 | 0x100000 |
+| 218 | blk.18.attn_k_norm.weight | 0x55f27e000 | 0x200 |
+| 219 | blk.18.attn_k.weight | 0x55f27e200 | 0x200000 |
+| 220 | blk.18.attn_output.weight | 0x55f47e200 | 0x1000000 |
+| 221 | blk.18.attn_q_norm.weight | 0x56047e200 | 0x200 |
+| 222 | blk.18.attn_q.weight | 0x56047e400 | 0x1000000 |
+| 223 | blk.18.attn_v.weight | 0x56147e400 | 0x200000 |
+| 224 | blk.18.attn_norm.weight | 0x56167e400 | 0x2000 |
+| 225 | blk.18.ffn_down_exps.weight | 0x561680400 | 0x18000000 |
+| 226 | blk.18.ffn_gate_exps.weight | 0x579680400 | 0x18000000 |
+| 227 | blk.18.ffn_up_exps.weight | 0x591680400 | 0x18000000 |
+| 228 | blk.18.ffn_norm.weight | 0x5a9680400 | 0x2000 |
+| 229 | blk.19.attn_norm.weight | 0x5a9682400 | 0x2000 |
+| 230 | blk.19.ffn_down_exps.weight | 0x5a9684400 | 0x18000000 |
+| 231 | blk.19.ffn_gate_exps.weight | 0x5c1684400 | 0x18000000 |
+| 232 | blk.19.ffn_up_exps.weight | 0x5d9684400 | 0x18000000 |
+| 233 | blk.19.ffn_gate_inp.weight | 0x5f1684400 | 0x100000 |
+| 234 | blk.19.ffn_norm.weight | 0x5f1784400 | 0x2000 |
+| 235 | blk.19.attn_k_norm.weight | 0x5f1786400 | 0x200 |
+| 236 | blk.19.attn_k.weight | 0x5f1786600 | 0x200000 |
+| 237 | blk.19.attn_output.weight | 0x5f1986600 | 0x1000000 |
+| 238 | blk.19.attn_q_norm.weight | 0x5f2986600 | 0x200 |
+| 239 | blk.19.attn_q.weight | 0x5f2986800 | 0x1000000 |
+| 240 | blk.19.attn_v.weight | 0x5f3986800 | 0x200000 |
+| 241 | blk.20.attn_norm.weight | 0x5f3b86800 | 0x2000 |
+| 242 | blk.20.ffn_down_exps.weight | 0x5f3b88800 | 0x18000000 |
+| 243 | blk.20.ffn_gate_exps.weight | 0x60bb88800 | 0x18000000 |
+| 244 | blk.20.ffn_up_exps.weight | 0x623b88800 | 0x18000000 |
+| 245 | blk.20.ffn_gate_inp.weight | 0x63bb88800 | 0x100000 |
+| 246 | blk.20.ffn_norm.weight | 0x63bc88800 | 0x2000 |
+| 247 | blk.20.attn_k_norm.weight | 0x63bc8a800 | 0x200 |
+| 248 | blk.20.attn_k.weight | 0x63bc8aa00 | 0x200000 |
+| 249 | blk.20.attn_output.weight | 0x63be8aa00 | 0x1000000 |
+| 250 | blk.20.attn_q_norm.weight | 0x63ce8aa00 | 0x200 |
+| 251 | blk.20.attn_q.weight | 0x63ce8ac00 | 0x1000000 |
+| 252 | blk.20.attn_v.weight | 0x63de8ac00 | 0x200000 |
+| 253 | blk.21.ffn_gate_inp.weight | 0x63e08ac00 | 0x100000 |
+| 254 | blk.21.attn_k_norm.weight | 0x63e18ac00 | 0x200 |
+| 255 | blk.21.attn_k.weight | 0x63e18ae00 | 0x200000 |
+| 256 | blk.21.attn_output.weight | 0x63e38ae00 | 0x1000000 |
+| 257 | blk.21.attn_q_norm.weight | 0x63f38ae00 | 0x200 |
+| 258 | blk.21.attn_q.weight | 0x63f38b000 | 0x1000000 |
+| 259 | blk.21.attn_v.weight | 0x64038b000 | 0x200000 |
+| 260 | blk.21.attn_norm.weight | 0x64058b000 | 0x2000 |
+| 261 | blk.21.ffn_down_exps.weight | 0x64058d000 | 0x18000000 |
+| 262 | blk.21.ffn_gate_exps.weight | 0x65858d000 | 0x18000000 |
+| 263 | blk.21.ffn_up_exps.weight | 0x67058d000 | 0x18000000 |
+| 264 | blk.21.ffn_norm.weight | 0x68858d000 | 0x2000 |
+| 265 | blk.22.attn_norm.weight | 0x68858f000 | 0x2000 |
+| 266 | blk.22.ffn_down_exps.weight | 0x688591000 | 0x18000000 |
+| 267 | blk.22.ffn_gate_exps.weight | 0x6a0591000 | 0x18000000 |
+| 268 | blk.22.ffn_up_exps.weight | 0x6b8591000 | 0x18000000 |
+| 269 | blk.22.ffn_gate_inp.weight | 0x6d0591000 | 0x100000 |
+| 270 | blk.22.ffn_norm.weight | 0x6d0691000 | 0x2000 |
+| 271 | blk.22.attn_k_norm.weight | 0x6d0693000 | 0x200 |
+| 272 | blk.22.attn_k.weight | 0x6d0693200 | 0x200000 |
+| 273 | blk.22.attn_output.weight | 0x6d0893200 | 0x1000000 |
+| 274 | blk.22.attn_q_norm.weight | 0x6d1893200 | 0x200 |
+| 275 | blk.22.attn_q.weight | 0x6d1893400 | 0x1000000 |
+| 276 | blk.22.attn_v.weight | 0x6d2893400 | 0x200000 |
+| 277 | blk.23.attn_norm.weight | 0x6d2a93400 | 0x2000 |
+| 278 | blk.23.ffn_down_exps.weight | 0x6d2a95400 | 0x18000000 |
+| 279 | blk.23.ffn_gate_exps.weight | 0x6eaa95400 | 0x18000000 |
+| 280 | blk.23.ffn_up_exps.weight | 0x702a95400 | 0x18000000 |
+| 281 | blk.23.ffn_gate_inp.weight | 0x71aa95400 | 0x100000 |
+| 282 | blk.23.ffn_norm.weight | 0x71ab95400 | 0x2000 |
+| 283 | blk.23.attn_k_norm.weight | 0x71ab97400 | 0x200 |
+| 284 | blk.23.attn_k.weight | 0x71ab97600 | 0x200000 |
+| 285 | blk.23.attn_output.weight | 0x71ad97600 | 0x1000000 |
+| 286 | blk.23.attn_q_norm.weight | 0x71bd97600 | 0x200 |
+| 287 | blk.23.attn_q.weight | 0x71bd97800 | 0x1000000 |
+| 288 | blk.23.attn_v.weight | 0x71cd97800 | 0x200000 |
+| 289 | blk.24.attn_norm.weight | 0x71cf97800 | 0x2000 |
+| 290 | blk.24.ffn_down_exps.weight | 0x71cf99800 | 0x18000000 |
+| 291 | blk.24.ffn_gate_exps.weight | 0x734f99800 | 0x18000000 |
+| 292 | blk.24.ffn_up_exps.weight | 0x74cf99800 | 0x18000000 |
+| 293 | blk.24.ffn_gate_inp.weight | 0x764f99800 | 0x100000 |
+| 294 | blk.24.ffn_norm.weight | 0x765099800 | 0x2000 |
+| 295 | blk.24.attn_k_norm.weight | 0x76509b800 | 0x200 |
+| 296 | blk.24.attn_k.weight | 0x76509ba00 | 0x200000 |
+| 297 | blk.24.attn_output.weight | 0x76529ba00 | 0x1000000 |
+| 298 | blk.24.attn_q_norm.weight | 0x76629ba00 | 0x200 |
+| 299 | blk.24.attn_q.weight | 0x76629bc00 | 0x1000000 |
+| 300 | blk.24.attn_v.weight | 0x76729bc00 | 0x200000 |
+| 301 | blk.25.ffn_gate_inp.weight | 0x76749bc00 | 0x100000 |
+| 302 | blk.25.attn_k_norm.weight | 0x76759bc00 | 0x200 |
+| 303 | blk.25.attn_k.weight | 0x76759be00 | 0x200000 |
+| 304 | blk.25.attn_output.weight | 0x76779be00 | 0x1000000 |
+| 305 | blk.25.attn_q_norm.weight | 0x76879be00 | 0x200 |
+| 306 | blk.25.attn_q.weight | 0x76879c000 | 0x1000000 |
+| 307 | blk.25.attn_v.weight | 0x76979c000 | 0x200000 |
+| 308 | blk.25.attn_norm.weight | 0x76999c000 | 0x2000 |
+| 309 | blk.25.ffn_down_exps.weight | 0x76999e000 | 0x18000000 |
+| 310 | blk.25.ffn_gate_exps.weight | 0x78199e000 | 0x18000000 |
+| 311 | blk.25.ffn_up_exps.weight | 0x79999e000 | 0x18000000 |
+| 312 | blk.25.ffn_norm.weight | 0x7b199e000 | 0x2000 |
+| 313 | blk.26.attn_norm.weight | 0x7b19a0000 | 0x2000 |
+| 314 | blk.26.ffn_down_exps.weight | 0x7b19a2000 | 0x18000000 |
+| 315 | blk.26.ffn_gate_exps.weight | 0x7c99a2000 | 0x18000000 |
+| 316 | blk.26.ffn_up_exps.weight | 0x7e19a2000 | 0x18000000 |
+| 317 | blk.26.ffn_gate_inp.weight | 0x7f99a2000 | 0x100000 |
+| 318 | blk.26.ffn_norm.weight | 0x7f9aa2000 | 0x2000 |
+| 319 | blk.26.attn_k_norm.weight | 0x7f9aa4000 | 0x200 |
+| 320 | blk.26.attn_k.weight | 0x7f9aa4200 | 0x200000 |
+| 321 | blk.26.attn_output.weight | 0x7f9ca4200 | 0x1000000 |
+| 322 | blk.26.attn_q_norm.weight | 0x7faca4200 | 0x200 |
+| 323 | blk.26.attn_q.weight | 0x7faca4400 | 0x1000000 |
+| 324 | blk.26.attn_v.weight | 0x7fbca4400 | 0x200000 |
+| 325 | blk.27.attn_norm.weight | 0x7fbea4400 | 0x2000 |
+| 326 | blk.27.ffn_down_exps.weight | 0x7fbea6400 | 0x18000000 |
+| 327 | blk.27.ffn_gate_exps.weight | 0x813ea6400 | 0x18000000 |
+| 328 | blk.27.ffn_up_exps.weight | 0x82bea6400 | 0x18000000 |
+| 329 | blk.27.ffn_gate_inp.weight | 0x843ea6400 | 0x100000 |
+| 330 | blk.27.ffn_norm.weight | 0x843fa6400 | 0x2000 |
+| 331 | blk.27.attn_k_norm.weight | 0x843fa8400 | 0x200 |
+| 332 | blk.27.attn_k.weight | 0x843fa8600 | 0x200000 |
+| 333 | blk.27.attn_output.weight | 0x8441a8600 | 0x1000000 |
+| 334 | blk.27.attn_q_norm.weight | 0x8451a8600 | 0x200 |
+| 335 | blk.27.attn_q.weight | 0x8451a8800 | 0x1000000 |
+| 336 | blk.27.attn_v.weight | 0x8461a8800 | 0x200000 |
+| 337 | blk.28.ffn_gate_inp.weight | 0x8463a8800 | 0x100000 |
+| 338 | blk.28.attn_k_norm.weight | 0x8464a8800 | 0x200 |
+| 339 | blk.28.attn_k.weight | 0x8464a8a00 | 0x200000 |
+| 340 | blk.28.attn_output.weight | 0x8466a8a00 | 0x1000000 |
+| 341 | blk.28.attn_q_norm.weight | 0x8476a8a00 | 0x200 |
+| 342 | blk.28.attn_q.weight | 0x8476a8c00 | 0x1000000 |
+| 343 | blk.28.attn_v.weight | 0x8486a8c00 | 0x200000 |
+| 344 | blk.28.attn_norm.weight | 0x8488a8c00 | 0x2000 |
+| 345 | blk.28.ffn_down_exps.weight | 0x8488aac00 | 0x18000000 |
+| 346 | blk.28.ffn_gate_exps.weight | 0x8608aac00 | 0x18000000 |
+| 347 | blk.28.ffn_up_exps.weight | 0x8788aac00 | 0x18000000 |
+| 348 | blk.28.ffn_norm.weight | 0x8908aac00 | 0x2000 |
+| 349 | blk.29.attn_norm.weight | 0x8908acc00 | 0x2000 |
+| 350 | blk.29.ffn_down_exps.weight | 0x8908aec00 | 0x18000000 |
+| 351 | blk.29.ffn_gate_exps.weight | 0x8a88aec00 | 0x18000000 |
+| 352 | blk.29.ffn_up_exps.weight | 0x8c08aec00 | 0x18000000 |
+| 353 | blk.29.ffn_gate_inp.weight | 0x8d88aec00 | 0x100000 |
+| 354 | blk.29.ffn_norm.weight | 0x8d89aec00 | 0x2000 |
+| 355 | blk.29.attn_k_norm.weight | 0x8d89b0c00 | 0x200 |
+| 356 | blk.29.attn_k.weight | 0x8d89b0e00 | 0x200000 |
+| 357 | blk.29.attn_output.weight | 0x8d8bb0e00 | 0x1000000 |
+| 358 | blk.29.attn_q_norm.weight | 0x8d9bb0e00 | 0x200 |
+| 359 | blk.29.attn_q.weight | 0x8d9bb1000 | 0x1000000 |
+| 360 | blk.29.attn_v.weight | 0x8dabb1000 | 0x200000 |
+| 361 | blk.30.attn_norm.weight | 0x8dadb1000 | 0x2000 |
+| 362 | blk.30.ffn_down_exps.weight | 0x8dadb3000 | 0x18000000 |
+| 363 | blk.30.ffn_gate_exps.weight | 0x8f2db3000 | 0x18000000 |
+| 364 | blk.30.ffn_up_exps.weight | 0x90adb3000 | 0x18000000 |
+| 365 | blk.30.ffn_gate_inp.weight | 0x922db3000 | 0x100000 |
+| 366 | blk.30.ffn_norm.weight | 0x922eb3000 | 0x2000 |
+| 367 | blk.30.attn_k_norm.weight | 0x922eb5000 | 0x200 |
+| 368 | blk.30.attn_k.weight | 0x922eb5200 | 0x200000 |
+| 369 | blk.30.attn_output.weight | 0x9230b5200 | 0x1000000 |
+| 370 | blk.30.attn_q_norm.weight | 0x9240b5200 | 0x200 |
+| 371 | blk.30.attn_q.weight | 0x9240b5400 | 0x1000000 |
+| 372 | blk.30.attn_v.weight | 0x9250b5400 | 0x200000 |
+| 373 | blk.31.ffn_gate_inp.weight | 0x9252b5400 | 0x100000 |
+| 374 | blk.31.attn_k_norm.weight | 0x9253b5400 | 0x200 |
+| 375 | blk.31.attn_k.weight | 0x9253b5600 | 0x200000 |
+| 376 | blk.31.attn_output.weight | 0x9255b5600 | 0x1000000 |
+| 377 | blk.31.attn_q_norm.weight | 0x9265b5600 | 0x200 |
+| 378 | blk.31.attn_q.weight | 0x9265b5800 | 0x1000000 |
+| 379 | blk.31.attn_v.weight | 0x9275b5800 | 0x200000 |
+| 380 | blk.31.attn_norm.weight | 0x9277b5800 | 0x2000 |
+| 381 | blk.31.ffn_down_exps.weight | 0x9277b7800 | 0x18000000 |
+| 382 | blk.31.ffn_gate_exps.weight | 0x93f7b7800 | 0x18000000 |
+| 383 | blk.31.ffn_up_exps.weight | 0x9577b7800 | 0x18000000 |
+| 384 | blk.31.ffn_norm.weight | 0x96f7b7800 | 0x2000 |
+| 385 | blk.32.attn_norm.weight | 0x96f7b9800 | 0x2000 |
+| 386 | blk.32.ffn_down_exps.weight | 0x96f7bb800 | 0x18000000 |
+| 387 | blk.32.ffn_gate_exps.weight | 0x9877bb800 | 0x18000000 |
+| 388 | blk.32.ffn_up_exps.weight | 0x99f7bb800 | 0x18000000 |
+| 389 | blk.32.ffn_gate_inp.weight | 0x9b77bb800 | 0x100000 |
+| 390 | blk.32.ffn_norm.weight | 0x9b78bb800 | 0x2000 |
+| 391 | blk.32.attn_k_norm.weight | 0x9b78bd800 | 0x200 |
+| 392 | blk.32.attn_k.weight | 0x9b78bda00 | 0x200000 |
+| 393 | blk.32.attn_output.weight | 0x9b7abda00 | 0x1000000 |
+| 394 | blk.32.attn_q_norm.weight | 0x9b8abda00 | 0x200 |
+| 395 | blk.32.attn_q.weight | 0x9b8abdc00 | 0x1000000 |
+| 396 | blk.32.attn_v.weight | 0x9b9abdc00 | 0x200000 |
+| 397 | blk.33.attn_norm.weight | 0x9b9cbdc00 | 0x2000 |
+| 398 | blk.33.ffn_down_exps.weight | 0x9b9cbfc00 | 0x18000000 |
+| 399 | blk.33.ffn_gate_exps.weight | 0x9d1cbfc00 | 0x18000000 |
+| 400 | blk.33.ffn_up_exps.weight | 0x9e9cbfc00 | 0x18000000 |
+| 401 | blk.33.ffn_gate_inp.weight | 0xa01cbfc00 | 0x100000 |
+| 402 | blk.33.ffn_norm.weight | 0xa01dbfc00 | 0x2000 |
+| 403 | blk.33.attn_k_norm.weight | 0xa01dc1c00 | 0x200 |
+| 404 | blk.33.attn_k.weight | 0xa01dc1e00 | 0x200000 |
+| 405 | blk.33.attn_output.weight | 0xa01fc1e00 | 0x1000000 |
+| 406 | blk.33.attn_q_norm.weight | 0xa02fc1e00 | 0x200 |
+| 407 | blk.33.attn_q.weight | 0xa02fc2000 | 0x1000000 |
+| 408 | blk.33.attn_v.weight | 0xa03fc2000 | 0x200000 |
+| 409 | blk.34.ffn_gate_inp.weight | 0xa041c2000 | 0x100000 |
+| 410 | blk.34.attn_k_norm.weight | 0xa042c2000 | 0x200 |
+| 411 | blk.34.attn_k.weight | 0xa042c2200 | 0x200000 |
+| 412 | blk.34.attn_output.weight | 0xa044c2200 | 0x1000000 |
+| 413 | blk.34.attn_q_norm.weight | 0xa054c2200 | 0x200 |
+| 414 | blk.34.attn_q.weight | 0xa054c2400 | 0x1000000 |
+| 415 | blk.34.attn_v.weight | 0xa064c2400 | 0x200000 |
+| 416 | blk.34.attn_norm.weight | 0xa066c2400 | 0x2000 |
+| 417 | blk.34.ffn_down_exps.weight | 0xa066c4400 | 0x18000000 |
+| 418 | blk.34.ffn_gate_exps.weight | 0xa1e6c4400 | 0x18000000 |
+| 419 | blk.34.ffn_up_exps.weight | 0xa366c4400 | 0x18000000 |
+| 420 | blk.34.ffn_norm.weight | 0xa4e6c4400 | 0x2000 |
+| 421 | blk.35.attn_norm.weight | 0xa4e6c6400 | 0x2000 |
+| 422 | blk.35.ffn_down_exps.weight | 0xa4e6c8400 | 0x18000000 |
+| 423 | blk.35.ffn_gate_exps.weight | 0xa666c8400 | 0x18000000 |
+| 424 | blk.35.ffn_up_exps.weight | 0xa7e6c8400 | 0x18000000 |
+| 425 | blk.35.ffn_gate_inp.weight | 0xa966c8400 | 0x100000 |
+| 426 | blk.35.ffn_norm.weight | 0xa967c8400 | 0x2000 |
+| 427 | blk.35.attn_k_norm.weight | 0xa967ca400 | 0x200 |
+| 428 | blk.35.attn_k.weight | 0xa967ca600 | 0x200000 |
+| 429 | blk.35.attn_output.weight | 0xa969ca600 | 0x1000000 |
+| 430 | blk.35.attn_q_norm.weight | 0xa979ca600 | 0x200 |
+| 431 | blk.35.attn_q.weight | 0xa979ca800 | 0x1000000 |
+| 432 | blk.35.attn_v.weight | 0xa989ca800 | 0x200000 |
+| 433 | blk.36.attn_norm.weight | 0xa98bca800 | 0x2000 |
+| 434 | blk.36.ffn_down_exps.weight | 0xa98bcc800 | 0x18000000 |
+| 435 | blk.36.ffn_gate_exps.weight | 0xab0bcc800 | 0x18000000 |
+| 436 | blk.36.ffn_up_exps.weight | 0xac8bcc800 | 0x18000000 |
+| 437 | blk.36.ffn_gate_inp.weight | 0xae0bcc800 | 0x100000 |
+| 438 | blk.36.ffn_norm.weight | 0xae0ccc800 | 0x2000 |
+| 439 | blk.36.attn_k_norm.weight | 0xae0cce800 | 0x200 |
+| 440 | blk.36.attn_k.weight | 0xae0ccea00 | 0x200000 |
+| 441 | blk.36.attn_output.weight | 0xae0ecea00 | 0x1000000 |
+| 442 | blk.36.attn_q_norm.weight | 0xae1ecea00 | 0x200 |
+| 443 | blk.36.attn_q.weight | 0xae1ecec00 | 0x1000000 |
+| 444 | blk.36.attn_v.weight | 0xae2ecec00 | 0x200000 |
+| 445 | blk.37.attn_norm.weight | 0xae30cec00 | 0x2000 |
+| 446 | blk.37.ffn_down_exps.weight | 0xae30d0c00 | 0x18000000 |
+| 447 | blk.37.ffn_gate_exps.weight | 0xafb0d0c00 | 0x18000000 |
+| 448 | blk.37.ffn_up_exps.weight | 0xb130d0c00 | 0x18000000 |
+| 449 | blk.37.ffn_gate_inp.weight | 0xb2b0d0c00 | 0x100000 |
+| 450 | blk.37.ffn_norm.weight | 0xb2b1d0c00 | 0x2000 |
+| 451 | blk.37.attn_k_norm.weight | 0xb2b1d2c00 | 0x200 |
+| 452 | blk.37.attn_k.weight | 0xb2b1d2e00 | 0x200000 |
+| 453 | blk.37.attn_output.weight | 0xb2b3d2e00 | 0x1000000 |
+| 454 | blk.37.attn_q_norm.weight | 0xb2c3d2e00 | 0x200 |
+| 455 | blk.37.attn_q.weight | 0xb2c3d3000 | 0x1000000 |
+| 456 | blk.37.attn_v.weight | 0xb2d3d3000 | 0x200000 |
+| 457 | blk.38.attn_norm.weight | 0xb2d5d3000 | 0x2000 |
+| 458 | blk.38.ffn_down_exps.weight | 0xb2d5d5000 | 0x18000000 |
+| 459 | blk.38.ffn_gate_exps.weight | 0xb455d5000 | 0x18000000 |
+| 460 | blk.38.ffn_up_exps.weight | 0xb5d5d5000 | 0x18000000 |
+| 461 | blk.38.ffn_gate_inp.weight | 0xb755d5000 | 0x100000 |
+| 462 | blk.38.ffn_norm.weight | 0xb756d5000 | 0x2000 |
+| 463 | blk.38.attn_k_norm.weight | 0xb756d7000 | 0x200 |
+| 464 | blk.38.attn_k.weight | 0xb756d7200 | 0x200000 |
+| 465 | blk.38.attn_output.weight | 0xb758d7200 | 0x1000000 |
+| 466 | blk.38.attn_q_norm.weight | 0xb768d7200 | 0x200 |
+| 467 | blk.38.attn_q.weight | 0xb768d7400 | 0x1000000 |
+| 468 | blk.38.attn_v.weight | 0xb778d7400 | 0x200000 |
+| 469 | blk.39.attn_norm.weight | 0xb77ad7400 | 0x2000 |
+| 470 | blk.39.ffn_down_exps.weight | 0xb77ad9400 | 0x18000000 |
+| 471 | blk.39.ffn_gate_exps.weight | 0xb8fad9400 | 0x18000000 |
+| 472 | blk.39.ffn_up_exps.weight | 0xba7ad9400 | 0x18000000 |
+| 473 | blk.39.ffn_gate_inp.weight | 0xbbfad9400 | 0x100000 |
+| 474 | blk.39.ffn_norm.weight | 0xbbfbd9400 | 0x2000 |
+| 475 | blk.39.attn_k_norm.weight | 0xbbfbdb400 | 0x200 |
+| 476 | blk.39.attn_k.weight | 0xbbfbdb600 | 0x200000 |
+| 477 | blk.39.attn_output.weight | 0xbbfddb600 | 0x1000000 |
+| 478 | blk.39.attn_q_norm.weight | 0xbc0ddb600 | 0x200 |
+| 479 | blk.39.attn_q.weight | 0xbc0ddb800 | 0x1000000 |
+| 480 | blk.39.attn_v.weight | 0xbc1ddb800 | 0x200000 |
+| 481 | blk.40.attn_norm.weight | 0xbc1fdb800 | 0x2000 |
+| 482 | blk.40.ffn_down_exps.weight | 0xbc1fdd800 | 0x18000000 |
+| 483 | blk.40.ffn_gate_exps.weight | 0xbd9fdd800 | 0x18000000 |
+| 484 | blk.40.ffn_up_exps.weight | 0xbf1fdd800 | 0x18000000 |
+| 485 | blk.40.ffn_gate_inp.weight | 0xc09fdd800 | 0x100000 |
+| 486 | blk.40.ffn_norm.weight | 0xc0a0dd800 | 0x2000 |
+| 487 | blk.40.attn_k_norm.weight | 0xc0a0df800 | 0x200 |
+| 488 | blk.40.attn_k.weight | 0xc0a0dfa00 | 0x200000 |
+| 489 | blk.40.attn_output.weight | 0xc0a2dfa00 | 0x1000000 |
+| 490 | blk.40.attn_q_norm.weight | 0xc0b2dfa00 | 0x200 |
+| 491 | blk.40.attn_q.weight | 0xc0b2dfc00 | 0x1000000 |
+| 492 | blk.40.attn_v.weight | 0xc0c2dfc00 | 0x200000 |
+| 493 | blk.41.ffn_gate_inp.weight | 0xc0c4dfc00 | 0x100000 |
+| 494 | blk.41.attn_k_norm.weight | 0xc0c5dfc00 | 0x200 |
+| 495 | blk.41.attn_k.weight | 0xc0c5dfe00 | 0x200000 |
+| 496 | blk.41.attn_output.weight | 0xc0c7dfe00 | 0x1000000 |
+| 497 | blk.41.attn_q_norm.weight | 0xc0d7dfe00 | 0x200 |
+| 498 | blk.41.attn_q.weight | 0xc0d7e0000 | 0x1000000 |
+| 499 | blk.41.attn_v.weight | 0xc0e7e0000 | 0x200000 |
+| 500 | blk.41.attn_norm.weight | 0xc0e9e0000 | 0x2000 |
+| 501 | blk.41.ffn_down_exps.weight | 0xc0e9e2000 | 0x18000000 |
+| 502 | blk.41.ffn_gate_exps.weight | 0xc269e2000 | 0x18000000 |
+| 503 | blk.41.ffn_up_exps.weight | 0xc3e9e2000 | 0x18000000 |
+| 504 | blk.41.ffn_norm.weight | 0xc569e2000 | 0x2000 |
+| 505 | blk.42.attn_norm.weight | 0xc569e4000 | 0x2000 |
+| 506 | blk.42.ffn_down_exps.weight | 0xc569e6000 | 0x18000000 |
+| 507 | blk.42.ffn_gate_exps.weight | 0xc6e9e6000 | 0x18000000 |
+| 508 | blk.42.ffn_up_exps.weight | 0xc869e6000 | 0x18000000 |
+| 509 | blk.42.ffn_gate_inp.weight | 0xc9e9e6000 | 0x100000 |
+| 510 | blk.42.ffn_norm.weight | 0xc9eae6000 | 0x2000 |
+| 511 | blk.42.attn_k_norm.weight | 0xc9eae8000 | 0x200 |
+| 512 | blk.42.attn_k.weight | 0xc9eae8200 | 0x200000 |
+| 513 | blk.42.attn_output.weight | 0xc9ece8200 | 0x1000000 |
+| 514 | blk.42.attn_q_norm.weight | 0xc9fce8200 | 0x200 |
+| 515 | blk.42.attn_q.weight | 0xc9fce8400 | 0x1000000 |
+| 516 | blk.42.attn_v.weight | 0xca0ce8400 | 0x200000 |
+| 517 | blk.43.attn_norm.weight | 0xca0ee8400 | 0x2000 |
+| 518 | blk.43.ffn_down_exps.weight | 0xca0eea400 | 0x18000000 |
+| 519 | blk.43.ffn_gate_exps.weight | 0xcb8eea400 | 0x18000000 |
+| 520 | blk.43.ffn_up_exps.weight | 0xcd0eea400 | 0x18000000 |
+| 521 | blk.43.ffn_gate_inp.weight | 0xce8eea400 | 0x100000 |
+| 522 | blk.43.ffn_norm.weight | 0xce8fea400 | 0x2000 |
+| 523 | blk.43.attn_k_norm.weight | 0xce8fec400 | 0x200 |
+| 524 | blk.43.attn_k.weight | 0xce8fec600 | 0x200000 |
+| 525 | blk.43.attn_output.weight | 0xce91ec600 | 0x1000000 |
+| 526 | blk.43.attn_q_norm.weight | 0xcea1ec600 | 0x200 |
+| 527 | blk.43.attn_q.weight | 0xcea1ec800 | 0x1000000 |
+| 528 | blk.43.attn_v.weight | 0xceb1ec800 | 0x200000 |
+| 529 | blk.44.ffn_gate_inp.weight | 0xceb3ec800 | 0x100000 |
+| 530 | blk.44.attn_k_norm.weight | 0xceb4ec800 | 0x200 |
+| 531 | blk.44.attn_k.weight | 0xceb4eca00 | 0x200000 |
+| 532 | blk.44.attn_output.weight | 0xceb6eca00 | 0x1000000 |
+| 533 | blk.44.attn_q_norm.weight | 0xcec6eca00 | 0x200 |
+| 534 | blk.44.attn_q.weight | 0xcec6ecc00 | 0x1000000 |
+| 535 | blk.44.attn_v.weight | 0xced6ecc00 | 0x200000 |
+| 536 | blk.44.attn_norm.weight | 0xced8ecc00 | 0x2000 |
+| 537 | blk.44.ffn_down_exps.weight | 0xced8eec00 | 0x18000000 |
+| 538 | blk.44.ffn_gate_exps.weight | 0xd058eec00 | 0x18000000 |
+| 539 | blk.44.ffn_up_exps.weight | 0xd1d8eec00 | 0x18000000 |
+| 540 | blk.44.ffn_norm.weight | 0xd358eec00 | 0x2000 |
+| 541 | blk.45.attn_norm.weight | 0xd358f0c00 | 0x2000 |
+| 542 | blk.45.ffn_down_exps.weight | 0xd358f2c00 | 0x18000000 |
+| 543 | blk.45.ffn_gate_exps.weight | 0xd4d8f2c00 | 0x18000000 |
+| 544 | blk.45.ffn_up_exps.weight | 0xd658f2c00 | 0x18000000 |
+| 545 | blk.45.ffn_gate_inp.weight | 0xd7d8f2c00 | 0x100000 |
+| 546 | blk.45.ffn_norm.weight | 0xd7d9f2c00 | 0x2000 |
+| 547 | blk.45.attn_k_norm.weight | 0xd7d9f4c00 | 0x200 |
+| 548 | blk.45.attn_k.weight | 0xd7d9f4e00 | 0x200000 |
+| 549 | blk.45.attn_output.weight | 0xd7dbf4e00 | 0x1000000 |
+| 550 | blk.45.attn_q_norm.weight | 0xd7ebf4e00 | 0x200 |
+| 551 | blk.45.attn_q.weight | 0xd7ebf5000 | 0x1000000 |
+| 552 | blk.45.attn_v.weight | 0xd7fbf5000 | 0x200000 |
+| 553 | blk.46.attn_norm.weight | 0xd7fdf5000 | 0x2000 |
+| 554 | blk.46.ffn_down_exps.weight | 0xd7fdf7000 | 0x18000000 |
+| 555 | blk.46.ffn_gate_exps.weight | 0xd97df7000 | 0x18000000 |
+| 556 | blk.46.ffn_up_exps.weight | 0xdafdf7000 | 0x18000000 |
+| 557 | blk.46.ffn_gate_inp.weight | 0xdc7df7000 | 0x100000 |
+| 558 | blk.46.ffn_norm.weight | 0xdc7ef7000 | 0x2000 |
+| 559 | blk.46.attn_k_norm.weight | 0xdc7ef9000 | 0x200 |
+| 560 | blk.46.attn_k.weight | 0xdc7ef9200 | 0x200000 |
+| 561 | blk.46.attn_output.weight | 0xdc80f9200 | 0x1000000 |
+| 562 | blk.46.attn_q_norm.weight | 0xdc90f9200 | 0x200 |
+| 563 | blk.46.attn_q.weight | 0xdc90f9400 | 0x1000000 |
+| 564 | blk.46.attn_v.weight | 0xdca0f9400 | 0x200000 |
+| 565 | blk.47.ffn_gate_inp.weight | 0xdca2f9400 | 0x100000 |
+| 566 | blk.47.attn_k_norm.weight | 0xdca3f9400 | 0x200 |
+| 567 | blk.47.attn_k.weight | 0xdca3f9600 | 0x200000 |
+| 568 | blk.47.attn_output.weight | 0xdca5f9600 | 0x1000000 |
+| 569 | blk.47.attn_q_norm.weight | 0xdcb5f9600 | 0x200 |
+| 570 | blk.47.attn_q.weight | 0xdcb5f9800 | 0x1000000 |
+| 571 | blk.47.attn_v.weight | 0xdcc5f9800 | 0x200000 |
+| 572 | output.weight | 0xdcc7f9800 | 0x25180000 |
+| 573 | blk.47.attn_norm.weight | 0xdf1979800 | 0x2000 |
+| 574 | blk.47.ffn_down_exps.weight | 0xdf197b800 | 0x18000000 |
+| 575 | blk.47.ffn_gate_exps.weight | 0xe0997b800 | 0x18000000 |
+| 576 | blk.47.ffn_up_exps.weight | 0xe2197b800 | 0x18000000 |
+| 577 | blk.47.ffn_norm.weight | 0xe3997b800 | 0x2000 |
+| 578 | output_norm.weight | 0xe3997d800 | 0x2000 |
+
+### Base Tensor Group : ~622M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:-------------------|:---------------------------------|:------------------|:----------------------|:-----|
+| 0 | token_embd.weight | Token Embedding (W) | (~311M) 311164928 | 2048 x 151936 x 1 x 1 | F16 |
+| 572 | output.weight | Output (W) | (~311M) 311164928 | 2048 x 151936 x 1 x 1 | F16 |
+| 578 | output_norm.weight | Output Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in base: (~622M) 622331904
+- Percentage of total elements: 2.04%
+
+
+### Block 0 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 1 | blk.0.attn_norm.weight | Block 0 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 2 | blk.0.ffn_down_exps.weight | Block 0 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 3 | blk.0.ffn_gate_exps.weight | Block 0 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 4 | blk.0.ffn_up_exps.weight | Block 0 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 5 | blk.0.ffn_gate_inp.weight | Block 0 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 6 | blk.0.ffn_norm.weight | Block 0 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 7 | blk.0.attn_k_norm.weight | Block 0 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 8 | blk.0.attn_k.weight | Block 0 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 9 | blk.0.attn_output.weight | Block 0 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 10 | blk.0.attn_q_norm.weight | Block 0 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 11 | blk.0.attn_q.weight | Block 0 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 12 | blk.0.attn_v.weight | Block 0 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.0: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 1 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 13 | blk.1.attn_norm.weight | Block 1 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 14 | blk.1.ffn_down_exps.weight | Block 1 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 15 | blk.1.ffn_gate_exps.weight | Block 1 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 16 | blk.1.ffn_up_exps.weight | Block 1 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 17 | blk.1.ffn_gate_inp.weight | Block 1 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 18 | blk.1.ffn_norm.weight | Block 1 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 19 | blk.1.attn_k_norm.weight | Block 1 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 20 | blk.1.attn_k.weight | Block 1 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 21 | blk.1.attn_output.weight | Block 1 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 22 | blk.1.attn_q_norm.weight | Block 1 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 23 | blk.1.attn_q.weight | Block 1 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 24 | blk.1.attn_v.weight | Block 1 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.1: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 2 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 25 | blk.2.ffn_gate_inp.weight | Block 2 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 26 | blk.2.attn_k_norm.weight | Block 2 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 27 | blk.2.attn_k.weight | Block 2 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 28 | blk.2.attn_output.weight | Block 2 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 29 | blk.2.attn_q_norm.weight | Block 2 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 30 | blk.2.attn_q.weight | Block 2 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 31 | blk.2.attn_v.weight | Block 2 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 32 | blk.2.attn_norm.weight | Block 2 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 33 | blk.2.ffn_down_exps.weight | Block 2 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 34 | blk.2.ffn_gate_exps.weight | Block 2 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 35 | blk.2.ffn_up_exps.weight | Block 2 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 36 | blk.2.ffn_norm.weight | Block 2 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.2: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 3 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 37 | blk.3.attn_norm.weight | Block 3 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 38 | blk.3.ffn_down_exps.weight | Block 3 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 39 | blk.3.ffn_gate_exps.weight | Block 3 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 40 | blk.3.ffn_up_exps.weight | Block 3 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 41 | blk.3.ffn_gate_inp.weight | Block 3 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 42 | blk.3.ffn_norm.weight | Block 3 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 43 | blk.3.attn_k_norm.weight | Block 3 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 44 | blk.3.attn_k.weight | Block 3 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 45 | blk.3.attn_output.weight | Block 3 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 46 | blk.3.attn_q_norm.weight | Block 3 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 47 | blk.3.attn_q.weight | Block 3 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 48 | blk.3.attn_v.weight | Block 3 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.3: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 4 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 49 | blk.4.attn_norm.weight | Block 4 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 50 | blk.4.ffn_down_exps.weight | Block 4 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 51 | blk.4.ffn_gate_exps.weight | Block 4 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 52 | blk.4.ffn_up_exps.weight | Block 4 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 53 | blk.4.ffn_gate_inp.weight | Block 4 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 54 | blk.4.ffn_norm.weight | Block 4 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 55 | blk.4.attn_k_norm.weight | Block 4 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 56 | blk.4.attn_k.weight | Block 4 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 57 | blk.4.attn_output.weight | Block 4 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 58 | blk.4.attn_q_norm.weight | Block 4 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 59 | blk.4.attn_q.weight | Block 4 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 60 | blk.4.attn_v.weight | Block 4 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.4: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 5 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 61 | blk.5.ffn_gate_inp.weight | Block 5 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 62 | blk.5.attn_k_norm.weight | Block 5 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 63 | blk.5.attn_k.weight | Block 5 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 64 | blk.5.attn_output.weight | Block 5 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 65 | blk.5.attn_q_norm.weight | Block 5 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 66 | blk.5.attn_q.weight | Block 5 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 67 | blk.5.attn_v.weight | Block 5 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 68 | blk.5.attn_norm.weight | Block 5 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 69 | blk.5.ffn_down_exps.weight | Block 5 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 70 | blk.5.ffn_gate_exps.weight | Block 5 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 71 | blk.5.ffn_up_exps.weight | Block 5 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 72 | blk.5.ffn_norm.weight | Block 5 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.5: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 6 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 73 | blk.6.attn_norm.weight | Block 6 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 74 | blk.6.ffn_down_exps.weight | Block 6 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 75 | blk.6.ffn_gate_exps.weight | Block 6 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 76 | blk.6.ffn_up_exps.weight | Block 6 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 77 | blk.6.ffn_gate_inp.weight | Block 6 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 78 | blk.6.ffn_norm.weight | Block 6 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 79 | blk.6.attn_k_norm.weight | Block 6 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 80 | blk.6.attn_k.weight | Block 6 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 81 | blk.6.attn_output.weight | Block 6 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 82 | blk.6.attn_q_norm.weight | Block 6 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 83 | blk.6.attn_q.weight | Block 6 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 84 | blk.6.attn_v.weight | Block 6 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.6: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 7 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 85 | blk.7.attn_norm.weight | Block 7 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 86 | blk.7.ffn_down_exps.weight | Block 7 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 87 | blk.7.ffn_gate_exps.weight | Block 7 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 88 | blk.7.ffn_up_exps.weight | Block 7 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 89 | blk.7.ffn_gate_inp.weight | Block 7 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 90 | blk.7.ffn_norm.weight | Block 7 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 91 | blk.7.attn_k_norm.weight | Block 7 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 92 | blk.7.attn_k.weight | Block 7 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 93 | blk.7.attn_output.weight | Block 7 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 94 | blk.7.attn_q_norm.weight | Block 7 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 95 | blk.7.attn_q.weight | Block 7 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 96 | blk.7.attn_v.weight | Block 7 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.7: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 8 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 97 | blk.8.attn_norm.weight | Block 8 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 98 | blk.8.ffn_down_exps.weight | Block 8 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 99 | blk.8.ffn_gate_exps.weight | Block 8 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 100 | blk.8.ffn_up_exps.weight | Block 8 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 101 | blk.8.ffn_gate_inp.weight | Block 8 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 102 | blk.8.ffn_norm.weight | Block 8 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 103 | blk.8.attn_k_norm.weight | Block 8 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 104 | blk.8.attn_k.weight | Block 8 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 105 | blk.8.attn_output.weight | Block 8 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 106 | blk.8.attn_q_norm.weight | Block 8 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 107 | blk.8.attn_q.weight | Block 8 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 108 | blk.8.attn_v.weight | Block 8 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.8: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 9 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 109 | blk.9.ffn_gate_inp.weight | Block 9 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 110 | blk.9.attn_k_norm.weight | Block 9 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 111 | blk.9.attn_k.weight | Block 9 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 112 | blk.9.attn_output.weight | Block 9 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 113 | blk.9.attn_q_norm.weight | Block 9 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 114 | blk.9.attn_q.weight | Block 9 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 115 | blk.9.attn_v.weight | Block 9 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 147 | blk.9.attn_norm.weight | Block 9 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 148 | blk.9.ffn_down_exps.weight | Block 9 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 149 | blk.9.ffn_gate_exps.weight | Block 9 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 150 | blk.9.ffn_up_exps.weight | Block 9 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 151 | blk.9.ffn_norm.weight | Block 9 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.9: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 10 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 116 | blk.10.attn_norm.weight | Block 10 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 117 | blk.10.ffn_down_exps.weight | Block 10 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 118 | blk.10.ffn_gate_exps.weight | Block 10 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 119 | blk.10.ffn_up_exps.weight | Block 10 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 120 | blk.10.ffn_gate_inp.weight | Block 10 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 121 | blk.10.ffn_norm.weight | Block 10 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 122 | blk.10.attn_k_norm.weight | Block 10 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 123 | blk.10.attn_k.weight | Block 10 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 124 | blk.10.attn_output.weight | Block 10 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 125 | blk.10.attn_q_norm.weight | Block 10 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 126 | blk.10.attn_q.weight | Block 10 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 127 | blk.10.attn_v.weight | Block 10 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.10: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 11 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 128 | blk.11.attn_norm.weight | Block 11 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 129 | blk.11.ffn_down_exps.weight | Block 11 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 130 | blk.11.ffn_gate_exps.weight | Block 11 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 131 | blk.11.ffn_up_exps.weight | Block 11 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 132 | blk.11.ffn_gate_inp.weight | Block 11 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 133 | blk.11.ffn_norm.weight | Block 11 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 134 | blk.11.attn_k_norm.weight | Block 11 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 135 | blk.11.attn_k.weight | Block 11 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 136 | blk.11.attn_output.weight | Block 11 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 137 | blk.11.attn_q_norm.weight | Block 11 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 138 | blk.11.attn_q.weight | Block 11 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 139 | blk.11.attn_v.weight | Block 11 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.11: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 12 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 140 | blk.12.ffn_gate_inp.weight | Block 12 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 141 | blk.12.attn_k_norm.weight | Block 12 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 142 | blk.12.attn_k.weight | Block 12 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 143 | blk.12.attn_output.weight | Block 12 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 144 | blk.12.attn_q_norm.weight | Block 12 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 145 | blk.12.attn_q.weight | Block 12 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 146 | blk.12.attn_v.weight | Block 12 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 152 | blk.12.attn_norm.weight | Block 12 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 153 | blk.12.ffn_down_exps.weight | Block 12 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 154 | blk.12.ffn_gate_exps.weight | Block 12 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 155 | blk.12.ffn_up_exps.weight | Block 12 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 156 | blk.12.ffn_norm.weight | Block 12 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.12: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 13 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 157 | blk.13.attn_norm.weight | Block 13 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 158 | blk.13.ffn_down_exps.weight | Block 13 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 159 | blk.13.ffn_gate_exps.weight | Block 13 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 160 | blk.13.ffn_up_exps.weight | Block 13 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 161 | blk.13.ffn_gate_inp.weight | Block 13 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 162 | blk.13.ffn_norm.weight | Block 13 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 163 | blk.13.attn_k_norm.weight | Block 13 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 164 | blk.13.attn_k.weight | Block 13 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 165 | blk.13.attn_output.weight | Block 13 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 166 | blk.13.attn_q_norm.weight | Block 13 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 167 | blk.13.attn_q.weight | Block 13 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 168 | blk.13.attn_v.weight | Block 13 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.13: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 14 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 169 | blk.14.attn_norm.weight | Block 14 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 170 | blk.14.ffn_down_exps.weight | Block 14 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 171 | blk.14.ffn_gate_exps.weight | Block 14 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 172 | blk.14.ffn_up_exps.weight | Block 14 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 173 | blk.14.ffn_gate_inp.weight | Block 14 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 174 | blk.14.ffn_norm.weight | Block 14 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 175 | blk.14.attn_k_norm.weight | Block 14 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 176 | blk.14.attn_k.weight | Block 14 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 177 | blk.14.attn_output.weight | Block 14 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 178 | blk.14.attn_q_norm.weight | Block 14 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 179 | blk.14.attn_q.weight | Block 14 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 180 | blk.14.attn_v.weight | Block 14 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.14: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 15 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 181 | blk.15.ffn_gate_inp.weight | Block 15 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 182 | blk.15.attn_k_norm.weight | Block 15 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 183 | blk.15.attn_k.weight | Block 15 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 184 | blk.15.attn_output.weight | Block 15 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 185 | blk.15.attn_q_norm.weight | Block 15 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 186 | blk.15.attn_q.weight | Block 15 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 187 | blk.15.attn_v.weight | Block 15 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 188 | blk.15.attn_norm.weight | Block 15 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 189 | blk.15.ffn_down_exps.weight | Block 15 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 190 | blk.15.ffn_gate_exps.weight | Block 15 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 191 | blk.15.ffn_up_exps.weight | Block 15 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 192 | blk.15.ffn_norm.weight | Block 15 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.15: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 16 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 193 | blk.16.attn_norm.weight | Block 16 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 194 | blk.16.ffn_down_exps.weight | Block 16 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 195 | blk.16.ffn_gate_exps.weight | Block 16 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 196 | blk.16.ffn_up_exps.weight | Block 16 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 197 | blk.16.ffn_gate_inp.weight | Block 16 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 198 | blk.16.ffn_norm.weight | Block 16 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 199 | blk.16.attn_k_norm.weight | Block 16 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 200 | blk.16.attn_k.weight | Block 16 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 201 | blk.16.attn_output.weight | Block 16 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 202 | blk.16.attn_q_norm.weight | Block 16 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 203 | blk.16.attn_q.weight | Block 16 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 204 | blk.16.attn_v.weight | Block 16 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.16: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 17 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 205 | blk.17.attn_norm.weight | Block 17 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 206 | blk.17.ffn_down_exps.weight | Block 17 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 207 | blk.17.ffn_gate_exps.weight | Block 17 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 208 | blk.17.ffn_up_exps.weight | Block 17 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 209 | blk.17.ffn_gate_inp.weight | Block 17 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 210 | blk.17.ffn_norm.weight | Block 17 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 211 | blk.17.attn_k_norm.weight | Block 17 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 212 | blk.17.attn_k.weight | Block 17 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 213 | blk.17.attn_output.weight | Block 17 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 214 | blk.17.attn_q_norm.weight | Block 17 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 215 | blk.17.attn_q.weight | Block 17 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 216 | blk.17.attn_v.weight | Block 17 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.17: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 18 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 217 | blk.18.ffn_gate_inp.weight | Block 18 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 218 | blk.18.attn_k_norm.weight | Block 18 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 219 | blk.18.attn_k.weight | Block 18 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 220 | blk.18.attn_output.weight | Block 18 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 221 | blk.18.attn_q_norm.weight | Block 18 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 222 | blk.18.attn_q.weight | Block 18 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 223 | blk.18.attn_v.weight | Block 18 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 224 | blk.18.attn_norm.weight | Block 18 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 225 | blk.18.ffn_down_exps.weight | Block 18 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 226 | blk.18.ffn_gate_exps.weight | Block 18 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 227 | blk.18.ffn_up_exps.weight | Block 18 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 228 | blk.18.ffn_norm.weight | Block 18 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.18: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 19 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 229 | blk.19.attn_norm.weight | Block 19 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 230 | blk.19.ffn_down_exps.weight | Block 19 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 231 | blk.19.ffn_gate_exps.weight | Block 19 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 232 | blk.19.ffn_up_exps.weight | Block 19 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 233 | blk.19.ffn_gate_inp.weight | Block 19 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 234 | blk.19.ffn_norm.weight | Block 19 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 235 | blk.19.attn_k_norm.weight | Block 19 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 236 | blk.19.attn_k.weight | Block 19 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 237 | blk.19.attn_output.weight | Block 19 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 238 | blk.19.attn_q_norm.weight | Block 19 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 239 | blk.19.attn_q.weight | Block 19 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 240 | blk.19.attn_v.weight | Block 19 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.19: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 20 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 241 | blk.20.attn_norm.weight | Block 20 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 242 | blk.20.ffn_down_exps.weight | Block 20 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 243 | blk.20.ffn_gate_exps.weight | Block 20 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 244 | blk.20.ffn_up_exps.weight | Block 20 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 245 | blk.20.ffn_gate_inp.weight | Block 20 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 246 | blk.20.ffn_norm.weight | Block 20 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 247 | blk.20.attn_k_norm.weight | Block 20 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 248 | blk.20.attn_k.weight | Block 20 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 249 | blk.20.attn_output.weight | Block 20 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 250 | blk.20.attn_q_norm.weight | Block 20 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 251 | blk.20.attn_q.weight | Block 20 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 252 | blk.20.attn_v.weight | Block 20 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.20: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 21 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 253 | blk.21.ffn_gate_inp.weight | Block 21 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 254 | blk.21.attn_k_norm.weight | Block 21 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 255 | blk.21.attn_k.weight | Block 21 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 256 | blk.21.attn_output.weight | Block 21 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 257 | blk.21.attn_q_norm.weight | Block 21 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 258 | blk.21.attn_q.weight | Block 21 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 259 | blk.21.attn_v.weight | Block 21 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 260 | blk.21.attn_norm.weight | Block 21 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 261 | blk.21.ffn_down_exps.weight | Block 21 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 262 | blk.21.ffn_gate_exps.weight | Block 21 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 263 | blk.21.ffn_up_exps.weight | Block 21 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 264 | blk.21.ffn_norm.weight | Block 21 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.21: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 22 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 265 | blk.22.attn_norm.weight | Block 22 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 266 | blk.22.ffn_down_exps.weight | Block 22 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 267 | blk.22.ffn_gate_exps.weight | Block 22 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 268 | blk.22.ffn_up_exps.weight | Block 22 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 269 | blk.22.ffn_gate_inp.weight | Block 22 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 270 | blk.22.ffn_norm.weight | Block 22 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 271 | blk.22.attn_k_norm.weight | Block 22 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 272 | blk.22.attn_k.weight | Block 22 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 273 | blk.22.attn_output.weight | Block 22 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 274 | blk.22.attn_q_norm.weight | Block 22 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 275 | blk.22.attn_q.weight | Block 22 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 276 | blk.22.attn_v.weight | Block 22 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.22: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 23 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 277 | blk.23.attn_norm.weight | Block 23 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 278 | blk.23.ffn_down_exps.weight | Block 23 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 279 | blk.23.ffn_gate_exps.weight | Block 23 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 280 | blk.23.ffn_up_exps.weight | Block 23 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 281 | blk.23.ffn_gate_inp.weight | Block 23 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 282 | blk.23.ffn_norm.weight | Block 23 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 283 | blk.23.attn_k_norm.weight | Block 23 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 284 | blk.23.attn_k.weight | Block 23 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 285 | blk.23.attn_output.weight | Block 23 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 286 | blk.23.attn_q_norm.weight | Block 23 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 287 | blk.23.attn_q.weight | Block 23 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 288 | blk.23.attn_v.weight | Block 23 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.23: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 24 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 289 | blk.24.attn_norm.weight | Block 24 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 290 | blk.24.ffn_down_exps.weight | Block 24 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 291 | blk.24.ffn_gate_exps.weight | Block 24 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 292 | blk.24.ffn_up_exps.weight | Block 24 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 293 | blk.24.ffn_gate_inp.weight | Block 24 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 294 | blk.24.ffn_norm.weight | Block 24 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 295 | blk.24.attn_k_norm.weight | Block 24 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 296 | blk.24.attn_k.weight | Block 24 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 297 | blk.24.attn_output.weight | Block 24 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 298 | blk.24.attn_q_norm.weight | Block 24 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 299 | blk.24.attn_q.weight | Block 24 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 300 | blk.24.attn_v.weight | Block 24 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.24: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 25 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 301 | blk.25.ffn_gate_inp.weight | Block 25 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 302 | blk.25.attn_k_norm.weight | Block 25 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 303 | blk.25.attn_k.weight | Block 25 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 304 | blk.25.attn_output.weight | Block 25 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 305 | blk.25.attn_q_norm.weight | Block 25 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 306 | blk.25.attn_q.weight | Block 25 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 307 | blk.25.attn_v.weight | Block 25 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 308 | blk.25.attn_norm.weight | Block 25 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 309 | blk.25.ffn_down_exps.weight | Block 25 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 310 | blk.25.ffn_gate_exps.weight | Block 25 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 311 | blk.25.ffn_up_exps.weight | Block 25 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 312 | blk.25.ffn_norm.weight | Block 25 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.25: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 26 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 313 | blk.26.attn_norm.weight | Block 26 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 314 | blk.26.ffn_down_exps.weight | Block 26 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 315 | blk.26.ffn_gate_exps.weight | Block 26 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 316 | blk.26.ffn_up_exps.weight | Block 26 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 317 | blk.26.ffn_gate_inp.weight | Block 26 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 318 | blk.26.ffn_norm.weight | Block 26 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 319 | blk.26.attn_k_norm.weight | Block 26 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 320 | blk.26.attn_k.weight | Block 26 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 321 | blk.26.attn_output.weight | Block 26 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 322 | blk.26.attn_q_norm.weight | Block 26 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 323 | blk.26.attn_q.weight | Block 26 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 324 | blk.26.attn_v.weight | Block 26 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.26: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 27 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 325 | blk.27.attn_norm.weight | Block 27 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 326 | blk.27.ffn_down_exps.weight | Block 27 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 327 | blk.27.ffn_gate_exps.weight | Block 27 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 328 | blk.27.ffn_up_exps.weight | Block 27 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 329 | blk.27.ffn_gate_inp.weight | Block 27 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 330 | blk.27.ffn_norm.weight | Block 27 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 331 | blk.27.attn_k_norm.weight | Block 27 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 332 | blk.27.attn_k.weight | Block 27 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 333 | blk.27.attn_output.weight | Block 27 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 334 | blk.27.attn_q_norm.weight | Block 27 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 335 | blk.27.attn_q.weight | Block 27 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 336 | blk.27.attn_v.weight | Block 27 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.27: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 28 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 337 | blk.28.ffn_gate_inp.weight | Block 28 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 338 | blk.28.attn_k_norm.weight | Block 28 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 339 | blk.28.attn_k.weight | Block 28 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 340 | blk.28.attn_output.weight | Block 28 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 341 | blk.28.attn_q_norm.weight | Block 28 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 342 | blk.28.attn_q.weight | Block 28 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 343 | blk.28.attn_v.weight | Block 28 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 344 | blk.28.attn_norm.weight | Block 28 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 345 | blk.28.ffn_down_exps.weight | Block 28 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 346 | blk.28.ffn_gate_exps.weight | Block 28 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 347 | blk.28.ffn_up_exps.weight | Block 28 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 348 | blk.28.ffn_norm.weight | Block 28 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.28: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 29 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 349 | blk.29.attn_norm.weight | Block 29 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 350 | blk.29.ffn_down_exps.weight | Block 29 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 351 | blk.29.ffn_gate_exps.weight | Block 29 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 352 | blk.29.ffn_up_exps.weight | Block 29 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 353 | blk.29.ffn_gate_inp.weight | Block 29 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 354 | blk.29.ffn_norm.weight | Block 29 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 355 | blk.29.attn_k_norm.weight | Block 29 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 356 | blk.29.attn_k.weight | Block 29 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 357 | blk.29.attn_output.weight | Block 29 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 358 | blk.29.attn_q_norm.weight | Block 29 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 359 | blk.29.attn_q.weight | Block 29 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 360 | blk.29.attn_v.weight | Block 29 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.29: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 30 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 361 | blk.30.attn_norm.weight | Block 30 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 362 | blk.30.ffn_down_exps.weight | Block 30 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 363 | blk.30.ffn_gate_exps.weight | Block 30 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 364 | blk.30.ffn_up_exps.weight | Block 30 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 365 | blk.30.ffn_gate_inp.weight | Block 30 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 366 | blk.30.ffn_norm.weight | Block 30 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 367 | blk.30.attn_k_norm.weight | Block 30 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 368 | blk.30.attn_k.weight | Block 30 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 369 | blk.30.attn_output.weight | Block 30 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 370 | blk.30.attn_q_norm.weight | Block 30 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 371 | blk.30.attn_q.weight | Block 30 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 372 | blk.30.attn_v.weight | Block 30 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.30: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 31 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 373 | blk.31.ffn_gate_inp.weight | Block 31 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 374 | blk.31.attn_k_norm.weight | Block 31 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 375 | blk.31.attn_k.weight | Block 31 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 376 | blk.31.attn_output.weight | Block 31 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 377 | blk.31.attn_q_norm.weight | Block 31 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 378 | blk.31.attn_q.weight | Block 31 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 379 | blk.31.attn_v.weight | Block 31 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 380 | blk.31.attn_norm.weight | Block 31 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 381 | blk.31.ffn_down_exps.weight | Block 31 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 382 | blk.31.ffn_gate_exps.weight | Block 31 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 383 | blk.31.ffn_up_exps.weight | Block 31 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 384 | blk.31.ffn_norm.weight | Block 31 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.31: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 32 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 385 | blk.32.attn_norm.weight | Block 32 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 386 | blk.32.ffn_down_exps.weight | Block 32 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 387 | blk.32.ffn_gate_exps.weight | Block 32 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 388 | blk.32.ffn_up_exps.weight | Block 32 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 389 | blk.32.ffn_gate_inp.weight | Block 32 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 390 | blk.32.ffn_norm.weight | Block 32 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 391 | blk.32.attn_k_norm.weight | Block 32 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 392 | blk.32.attn_k.weight | Block 32 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 393 | blk.32.attn_output.weight | Block 32 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 394 | blk.32.attn_q_norm.weight | Block 32 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 395 | blk.32.attn_q.weight | Block 32 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 396 | blk.32.attn_v.weight | Block 32 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.32: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 33 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 397 | blk.33.attn_norm.weight | Block 33 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 398 | blk.33.ffn_down_exps.weight | Block 33 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 399 | blk.33.ffn_gate_exps.weight | Block 33 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 400 | blk.33.ffn_up_exps.weight | Block 33 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 401 | blk.33.ffn_gate_inp.weight | Block 33 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 402 | blk.33.ffn_norm.weight | Block 33 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 403 | blk.33.attn_k_norm.weight | Block 33 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 404 | blk.33.attn_k.weight | Block 33 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 405 | blk.33.attn_output.weight | Block 33 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 406 | blk.33.attn_q_norm.weight | Block 33 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 407 | blk.33.attn_q.weight | Block 33 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 408 | blk.33.attn_v.weight | Block 33 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.33: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 34 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 409 | blk.34.ffn_gate_inp.weight | Block 34 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 410 | blk.34.attn_k_norm.weight | Block 34 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 411 | blk.34.attn_k.weight | Block 34 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 412 | blk.34.attn_output.weight | Block 34 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 413 | blk.34.attn_q_norm.weight | Block 34 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 414 | blk.34.attn_q.weight | Block 34 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 415 | blk.34.attn_v.weight | Block 34 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 416 | blk.34.attn_norm.weight | Block 34 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 417 | blk.34.ffn_down_exps.weight | Block 34 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 418 | blk.34.ffn_gate_exps.weight | Block 34 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 419 | blk.34.ffn_up_exps.weight | Block 34 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 420 | blk.34.ffn_norm.weight | Block 34 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.34: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 35 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 421 | blk.35.attn_norm.weight | Block 35 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 422 | blk.35.ffn_down_exps.weight | Block 35 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 423 | blk.35.ffn_gate_exps.weight | Block 35 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 424 | blk.35.ffn_up_exps.weight | Block 35 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 425 | blk.35.ffn_gate_inp.weight | Block 35 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 426 | blk.35.ffn_norm.weight | Block 35 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 427 | blk.35.attn_k_norm.weight | Block 35 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 428 | blk.35.attn_k.weight | Block 35 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 429 | blk.35.attn_output.weight | Block 35 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 430 | blk.35.attn_q_norm.weight | Block 35 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 431 | blk.35.attn_q.weight | Block 35 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 432 | blk.35.attn_v.weight | Block 35 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.35: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 36 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 433 | blk.36.attn_norm.weight | Block 36 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 434 | blk.36.ffn_down_exps.weight | Block 36 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 435 | blk.36.ffn_gate_exps.weight | Block 36 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 436 | blk.36.ffn_up_exps.weight | Block 36 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 437 | blk.36.ffn_gate_inp.weight | Block 36 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 438 | blk.36.ffn_norm.weight | Block 36 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 439 | blk.36.attn_k_norm.weight | Block 36 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 440 | blk.36.attn_k.weight | Block 36 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 441 | blk.36.attn_output.weight | Block 36 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 442 | blk.36.attn_q_norm.weight | Block 36 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 443 | blk.36.attn_q.weight | Block 36 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 444 | blk.36.attn_v.weight | Block 36 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.36: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 37 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 445 | blk.37.attn_norm.weight | Block 37 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 446 | blk.37.ffn_down_exps.weight | Block 37 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 447 | blk.37.ffn_gate_exps.weight | Block 37 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 448 | blk.37.ffn_up_exps.weight | Block 37 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 449 | blk.37.ffn_gate_inp.weight | Block 37 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 450 | blk.37.ffn_norm.weight | Block 37 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 451 | blk.37.attn_k_norm.weight | Block 37 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 452 | blk.37.attn_k.weight | Block 37 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 453 | blk.37.attn_output.weight | Block 37 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 454 | blk.37.attn_q_norm.weight | Block 37 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 455 | blk.37.attn_q.weight | Block 37 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 456 | blk.37.attn_v.weight | Block 37 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.37: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 38 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 457 | blk.38.attn_norm.weight | Block 38 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 458 | blk.38.ffn_down_exps.weight | Block 38 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 459 | blk.38.ffn_gate_exps.weight | Block 38 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 460 | blk.38.ffn_up_exps.weight | Block 38 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 461 | blk.38.ffn_gate_inp.weight | Block 38 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 462 | blk.38.ffn_norm.weight | Block 38 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 463 | blk.38.attn_k_norm.weight | Block 38 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 464 | blk.38.attn_k.weight | Block 38 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 465 | blk.38.attn_output.weight | Block 38 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 466 | blk.38.attn_q_norm.weight | Block 38 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 467 | blk.38.attn_q.weight | Block 38 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 468 | blk.38.attn_v.weight | Block 38 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.38: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 39 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 469 | blk.39.attn_norm.weight | Block 39 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 470 | blk.39.ffn_down_exps.weight | Block 39 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 471 | blk.39.ffn_gate_exps.weight | Block 39 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 472 | blk.39.ffn_up_exps.weight | Block 39 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 473 | blk.39.ffn_gate_inp.weight | Block 39 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 474 | blk.39.ffn_norm.weight | Block 39 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 475 | blk.39.attn_k_norm.weight | Block 39 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 476 | blk.39.attn_k.weight | Block 39 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 477 | blk.39.attn_output.weight | Block 39 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 478 | blk.39.attn_q_norm.weight | Block 39 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 479 | blk.39.attn_q.weight | Block 39 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 480 | blk.39.attn_v.weight | Block 39 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.39: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 40 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 481 | blk.40.attn_norm.weight | Block 40 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 482 | blk.40.ffn_down_exps.weight | Block 40 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 483 | blk.40.ffn_gate_exps.weight | Block 40 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 484 | blk.40.ffn_up_exps.weight | Block 40 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 485 | blk.40.ffn_gate_inp.weight | Block 40 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 486 | blk.40.ffn_norm.weight | Block 40 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 487 | blk.40.attn_k_norm.weight | Block 40 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 488 | blk.40.attn_k.weight | Block 40 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 489 | blk.40.attn_output.weight | Block 40 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 490 | blk.40.attn_q_norm.weight | Block 40 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 491 | blk.40.attn_q.weight | Block 40 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 492 | blk.40.attn_v.weight | Block 40 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.40: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 41 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 493 | blk.41.ffn_gate_inp.weight | Block 41 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 494 | blk.41.attn_k_norm.weight | Block 41 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 495 | blk.41.attn_k.weight | Block 41 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 496 | blk.41.attn_output.weight | Block 41 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 497 | blk.41.attn_q_norm.weight | Block 41 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 498 | blk.41.attn_q.weight | Block 41 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 499 | blk.41.attn_v.weight | Block 41 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 500 | blk.41.attn_norm.weight | Block 41 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 501 | blk.41.ffn_down_exps.weight | Block 41 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 502 | blk.41.ffn_gate_exps.weight | Block 41 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 503 | blk.41.ffn_up_exps.weight | Block 41 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 504 | blk.41.ffn_norm.weight | Block 41 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.41: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 42 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 505 | blk.42.attn_norm.weight | Block 42 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 506 | blk.42.ffn_down_exps.weight | Block 42 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 507 | blk.42.ffn_gate_exps.weight | Block 42 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 508 | blk.42.ffn_up_exps.weight | Block 42 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 509 | blk.42.ffn_gate_inp.weight | Block 42 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 510 | blk.42.ffn_norm.weight | Block 42 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 511 | blk.42.attn_k_norm.weight | Block 42 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 512 | blk.42.attn_k.weight | Block 42 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 513 | blk.42.attn_output.weight | Block 42 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 514 | blk.42.attn_q_norm.weight | Block 42 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 515 | blk.42.attn_q.weight | Block 42 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 516 | blk.42.attn_v.weight | Block 42 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.42: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 43 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 517 | blk.43.attn_norm.weight | Block 43 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 518 | blk.43.ffn_down_exps.weight | Block 43 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 519 | blk.43.ffn_gate_exps.weight | Block 43 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 520 | blk.43.ffn_up_exps.weight | Block 43 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 521 | blk.43.ffn_gate_inp.weight | Block 43 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 522 | blk.43.ffn_norm.weight | Block 43 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 523 | blk.43.attn_k_norm.weight | Block 43 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 524 | blk.43.attn_k.weight | Block 43 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 525 | blk.43.attn_output.weight | Block 43 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 526 | blk.43.attn_q_norm.weight | Block 43 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 527 | blk.43.attn_q.weight | Block 43 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 528 | blk.43.attn_v.weight | Block 43 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.43: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 44 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 529 | blk.44.ffn_gate_inp.weight | Block 44 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 530 | blk.44.attn_k_norm.weight | Block 44 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 531 | blk.44.attn_k.weight | Block 44 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 532 | blk.44.attn_output.weight | Block 44 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 533 | blk.44.attn_q_norm.weight | Block 44 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 534 | blk.44.attn_q.weight | Block 44 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 535 | blk.44.attn_v.weight | Block 44 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 536 | blk.44.attn_norm.weight | Block 44 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 537 | blk.44.ffn_down_exps.weight | Block 44 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 538 | blk.44.ffn_gate_exps.weight | Block 44 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 539 | blk.44.ffn_up_exps.weight | Block 44 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 540 | blk.44.ffn_norm.weight | Block 44 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.44: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 45 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 541 | blk.45.attn_norm.weight | Block 45 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 542 | blk.45.ffn_down_exps.weight | Block 45 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 543 | blk.45.ffn_gate_exps.weight | Block 45 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 544 | blk.45.ffn_up_exps.weight | Block 45 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 545 | blk.45.ffn_gate_inp.weight | Block 45 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 546 | blk.45.ffn_norm.weight | Block 45 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 547 | blk.45.attn_k_norm.weight | Block 45 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 548 | blk.45.attn_k.weight | Block 45 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 549 | blk.45.attn_output.weight | Block 45 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 550 | blk.45.attn_q_norm.weight | Block 45 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 551 | blk.45.attn_q.weight | Block 45 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 552 | blk.45.attn_v.weight | Block 45 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.45: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 46 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 553 | blk.46.attn_norm.weight | Block 46 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 554 | blk.46.ffn_down_exps.weight | Block 46 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 555 | blk.46.ffn_gate_exps.weight | Block 46 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 556 | blk.46.ffn_up_exps.weight | Block 46 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 557 | blk.46.ffn_gate_inp.weight | Block 46 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 558 | blk.46.ffn_norm.weight | Block 46 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 559 | blk.46.attn_k_norm.weight | Block 46 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 560 | blk.46.attn_k.weight | Block 46 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 561 | blk.46.attn_output.weight | Block 46 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 562 | blk.46.attn_q_norm.weight | Block 46 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 563 | blk.46.attn_q.weight | Block 46 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 564 | blk.46.attn_v.weight | Block 46 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+
+- Total elements in blk.46: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+### Block 47 Tensor Group : ~623M Elements
+
+| T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type |
+|-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-----|
+| 565 | blk.47.ffn_gate_inp.weight | Block 47 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 |
+| 566 | blk.47.attn_k_norm.weight | Block 47 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 567 | blk.47.attn_k.weight | Block 47 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 568 | blk.47.attn_output.weight | Block 47 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | F16 |
+| 569 | blk.47.attn_q_norm.weight | Block 47 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 |
+| 570 | blk.47.attn_q.weight | Block 47 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | F16 |
+| 571 | blk.47.attn_v.weight | Block 47 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | F16 |
+| 573 | blk.47.attn_norm.weight | Block 47 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+| 574 | blk.47.ffn_down_exps.weight | Block 47 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | F16 |
+| 575 | blk.47.ffn_gate_exps.weight | Block 47 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 576 | blk.47.ffn_up_exps.weight | Block 47 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | F16 |
+| 577 | blk.47.ffn_norm.weight | Block 47 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 |
+
+- Total elements in blk.47: (~623M) 623120640
+- Percentage of total elements: 2.04%
+
+
+