monology commited on
Commit
04163b2
·
verified ·
1 Parent(s): 8f23adb

Upload 3 files

Browse files
Files changed (3) hide show
  1. README.md +6 -196
  2. config.json +1 -0
  3. quanto_qmap.json +462 -0
README.md CHANGED
@@ -1,199 +1,9 @@
1
  ---
2
- library_name: transformers
3
- tags: []
4
  ---
5
 
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
- This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
 
1
  ---
2
+ tags:
3
+ - model_hub_mixin
4
  ---
5
 
6
+ This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
7
+ - Code: [More Information Needed]
8
+ - Paper: [More Information Needed]
9
+ - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
config.json CHANGED
@@ -28,6 +28,7 @@
28
  "partial_rotary_factor": 0.5,
29
  "rms_norm_eps": 1e-06,
30
  "rope_theta": 10000.0,
 
31
  "torch_dtype": "float16",
32
  "transformers_version": "4.51.3",
33
  "use_cache": true,
 
28
  "partial_rotary_factor": 0.5,
29
  "rms_norm_eps": 1e-06,
30
  "rope_theta": 10000.0,
31
+ "tie_word_embeddings": false,
32
  "torch_dtype": "float16",
33
  "transformers_version": "4.51.3",
34
  "use_cache": true,
quanto_qmap.json ADDED
@@ -0,0 +1,462 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model.layers.0.mlp_block.gate_proj": {
3
+ "weights": "qint8",
4
+ "activations": "none"
5
+ },
6
+ "model.layers.0.mlp_block.up_proj": {
7
+ "weights": "qint8",
8
+ "activations": "none"
9
+ },
10
+ "model.layers.0.mlp_block.down_proj": {
11
+ "weights": "qint8",
12
+ "activations": "none"
13
+ },
14
+ "model.layers.1.mlp_block.gate_proj": {
15
+ "weights": "qint8",
16
+ "activations": "none"
17
+ },
18
+ "model.layers.1.mlp_block.up_proj": {
19
+ "weights": "qint8",
20
+ "activations": "none"
21
+ },
22
+ "model.layers.1.mlp_block.down_proj": {
23
+ "weights": "qint8",
24
+ "activations": "none"
25
+ },
26
+ "model.layers.2.mlp_block.gate_proj": {
27
+ "weights": "qint8",
28
+ "activations": "none"
29
+ },
30
+ "model.layers.2.mlp_block.up_proj": {
31
+ "weights": "qint8",
32
+ "activations": "none"
33
+ },
34
+ "model.layers.2.mlp_block.down_proj": {
35
+ "weights": "qint8",
36
+ "activations": "none"
37
+ },
38
+ "model.layers.3.mlp_block.gate_proj": {
39
+ "weights": "qint8",
40
+ "activations": "none"
41
+ },
42
+ "model.layers.3.mlp_block.up_proj": {
43
+ "weights": "qint8",
44
+ "activations": "none"
45
+ },
46
+ "model.layers.3.mlp_block.down_proj": {
47
+ "weights": "qint8",
48
+ "activations": "none"
49
+ },
50
+ "model.layers.4.mlp_block.gate_proj": {
51
+ "weights": "qint8",
52
+ "activations": "none"
53
+ },
54
+ "model.layers.4.mlp_block.up_proj": {
55
+ "weights": "qint8",
56
+ "activations": "none"
57
+ },
58
+ "model.layers.4.mlp_block.down_proj": {
59
+ "weights": "qint8",
60
+ "activations": "none"
61
+ },
62
+ "model.layers.5.mlp_block.gate_proj": {
63
+ "weights": "qint8",
64
+ "activations": "none"
65
+ },
66
+ "model.layers.5.mlp_block.up_proj": {
67
+ "weights": "qint8",
68
+ "activations": "none"
69
+ },
70
+ "model.layers.5.mlp_block.down_proj": {
71
+ "weights": "qint8",
72
+ "activations": "none"
73
+ },
74
+ "model.layers.6.mlp_block.gate_proj": {
75
+ "weights": "qint8",
76
+ "activations": "none"
77
+ },
78
+ "model.layers.6.mlp_block.up_proj": {
79
+ "weights": "qint8",
80
+ "activations": "none"
81
+ },
82
+ "model.layers.6.mlp_block.down_proj": {
83
+ "weights": "qint8",
84
+ "activations": "none"
85
+ },
86
+ "model.layers.7.mlp_block.gate_proj": {
87
+ "weights": "qint8",
88
+ "activations": "none"
89
+ },
90
+ "model.layers.7.mlp_block.up_proj": {
91
+ "weights": "qint8",
92
+ "activations": "none"
93
+ },
94
+ "model.layers.7.mlp_block.down_proj": {
95
+ "weights": "qint8",
96
+ "activations": "none"
97
+ },
98
+ "model.layers.8.mlp_block.gate_proj": {
99
+ "weights": "qint8",
100
+ "activations": "none"
101
+ },
102
+ "model.layers.8.mlp_block.up_proj": {
103
+ "weights": "qint8",
104
+ "activations": "none"
105
+ },
106
+ "model.layers.8.mlp_block.down_proj": {
107
+ "weights": "qint8",
108
+ "activations": "none"
109
+ },
110
+ "model.layers.9.mlp_block.gate_proj": {
111
+ "weights": "qint8",
112
+ "activations": "none"
113
+ },
114
+ "model.layers.9.mlp_block.up_proj": {
115
+ "weights": "qint8",
116
+ "activations": "none"
117
+ },
118
+ "model.layers.9.mlp_block.down_proj": {
119
+ "weights": "qint8",
120
+ "activations": "none"
121
+ },
122
+ "model.layers.10.mlp_block.gate_proj": {
123
+ "weights": "qint8",
124
+ "activations": "none"
125
+ },
126
+ "model.layers.10.mlp_block.up_proj": {
127
+ "weights": "qint8",
128
+ "activations": "none"
129
+ },
130
+ "model.layers.10.mlp_block.down_proj": {
131
+ "weights": "qint8",
132
+ "activations": "none"
133
+ },
134
+ "model.layers.11.mlp_block.gate_proj": {
135
+ "weights": "qint8",
136
+ "activations": "none"
137
+ },
138
+ "model.layers.11.mlp_block.up_proj": {
139
+ "weights": "qint8",
140
+ "activations": "none"
141
+ },
142
+ "model.layers.11.mlp_block.down_proj": {
143
+ "weights": "qint8",
144
+ "activations": "none"
145
+ },
146
+ "model.layers.12.mlp_block.gate_proj": {
147
+ "weights": "qint8",
148
+ "activations": "none"
149
+ },
150
+ "model.layers.12.mlp_block.up_proj": {
151
+ "weights": "qint8",
152
+ "activations": "none"
153
+ },
154
+ "model.layers.12.mlp_block.down_proj": {
155
+ "weights": "qint8",
156
+ "activations": "none"
157
+ },
158
+ "model.layers.13.mlp_block.gate_proj": {
159
+ "weights": "qint8",
160
+ "activations": "none"
161
+ },
162
+ "model.layers.13.mlp_block.up_proj": {
163
+ "weights": "qint8",
164
+ "activations": "none"
165
+ },
166
+ "model.layers.13.mlp_block.down_proj": {
167
+ "weights": "qint8",
168
+ "activations": "none"
169
+ },
170
+ "model.layers.14.mlp_block.gate_proj": {
171
+ "weights": "qint8",
172
+ "activations": "none"
173
+ },
174
+ "model.layers.14.mlp_block.up_proj": {
175
+ "weights": "qint8",
176
+ "activations": "none"
177
+ },
178
+ "model.layers.14.mlp_block.down_proj": {
179
+ "weights": "qint8",
180
+ "activations": "none"
181
+ },
182
+ "model.layers.15.mlp_block.gate_proj": {
183
+ "weights": "qint8",
184
+ "activations": "none"
185
+ },
186
+ "model.layers.15.mlp_block.up_proj": {
187
+ "weights": "qint8",
188
+ "activations": "none"
189
+ },
190
+ "model.layers.15.mlp_block.down_proj": {
191
+ "weights": "qint8",
192
+ "activations": "none"
193
+ },
194
+ "model.layers.16.mlp_block.gate_proj": {
195
+ "weights": "qint8",
196
+ "activations": "none"
197
+ },
198
+ "model.layers.16.mlp_block.up_proj": {
199
+ "weights": "qint8",
200
+ "activations": "none"
201
+ },
202
+ "model.layers.16.mlp_block.down_proj": {
203
+ "weights": "qint8",
204
+ "activations": "none"
205
+ },
206
+ "model.layers.17.mlp_block.gate_proj": {
207
+ "weights": "qint8",
208
+ "activations": "none"
209
+ },
210
+ "model.layers.17.mlp_block.up_proj": {
211
+ "weights": "qint8",
212
+ "activations": "none"
213
+ },
214
+ "model.layers.17.mlp_block.down_proj": {
215
+ "weights": "qint8",
216
+ "activations": "none"
217
+ },
218
+ "model.layers.18.mlp_block.gate_proj": {
219
+ "weights": "qint8",
220
+ "activations": "none"
221
+ },
222
+ "model.layers.18.mlp_block.up_proj": {
223
+ "weights": "qint8",
224
+ "activations": "none"
225
+ },
226
+ "model.layers.18.mlp_block.down_proj": {
227
+ "weights": "qint8",
228
+ "activations": "none"
229
+ },
230
+ "model.layers.19.mlp_block.gate_proj": {
231
+ "weights": "qint8",
232
+ "activations": "none"
233
+ },
234
+ "model.layers.19.mlp_block.up_proj": {
235
+ "weights": "qint8",
236
+ "activations": "none"
237
+ },
238
+ "model.layers.19.mlp_block.down_proj": {
239
+ "weights": "qint8",
240
+ "activations": "none"
241
+ },
242
+ "model.layers.20.mlp_block.gate_proj": {
243
+ "weights": "qint8",
244
+ "activations": "none"
245
+ },
246
+ "model.layers.20.mlp_block.up_proj": {
247
+ "weights": "qint8",
248
+ "activations": "none"
249
+ },
250
+ "model.layers.20.mlp_block.down_proj": {
251
+ "weights": "qint8",
252
+ "activations": "none"
253
+ },
254
+ "model.layers.21.mlp_block.gate_proj": {
255
+ "weights": "qint8",
256
+ "activations": "none"
257
+ },
258
+ "model.layers.21.mlp_block.up_proj": {
259
+ "weights": "qint8",
260
+ "activations": "none"
261
+ },
262
+ "model.layers.21.mlp_block.down_proj": {
263
+ "weights": "qint8",
264
+ "activations": "none"
265
+ },
266
+ "model.layers.22.mlp_block.gate_proj": {
267
+ "weights": "qint8",
268
+ "activations": "none"
269
+ },
270
+ "model.layers.22.mlp_block.up_proj": {
271
+ "weights": "qint8",
272
+ "activations": "none"
273
+ },
274
+ "model.layers.22.mlp_block.down_proj": {
275
+ "weights": "qint8",
276
+ "activations": "none"
277
+ },
278
+ "model.layers.23.mlp_block.gate_proj": {
279
+ "weights": "qint8",
280
+ "activations": "none"
281
+ },
282
+ "model.layers.23.mlp_block.up_proj": {
283
+ "weights": "qint8",
284
+ "activations": "none"
285
+ },
286
+ "model.layers.23.mlp_block.down_proj": {
287
+ "weights": "qint8",
288
+ "activations": "none"
289
+ },
290
+ "model.layers.24.mlp_block.gate_proj": {
291
+ "weights": "qint8",
292
+ "activations": "none"
293
+ },
294
+ "model.layers.24.mlp_block.up_proj": {
295
+ "weights": "qint8",
296
+ "activations": "none"
297
+ },
298
+ "model.layers.24.mlp_block.down_proj": {
299
+ "weights": "qint8",
300
+ "activations": "none"
301
+ },
302
+ "model.layers.25.mlp_block.gate_proj": {
303
+ "weights": "qint8",
304
+ "activations": "none"
305
+ },
306
+ "model.layers.25.mlp_block.up_proj": {
307
+ "weights": "qint8",
308
+ "activations": "none"
309
+ },
310
+ "model.layers.25.mlp_block.down_proj": {
311
+ "weights": "qint8",
312
+ "activations": "none"
313
+ },
314
+ "model.layers.26.mlp_block.gate_proj": {
315
+ "weights": "qint8",
316
+ "activations": "none"
317
+ },
318
+ "model.layers.26.mlp_block.up_proj": {
319
+ "weights": "qint8",
320
+ "activations": "none"
321
+ },
322
+ "model.layers.26.mlp_block.down_proj": {
323
+ "weights": "qint8",
324
+ "activations": "none"
325
+ },
326
+ "model.layers.27.mlp_block.gate_proj": {
327
+ "weights": "qint8",
328
+ "activations": "none"
329
+ },
330
+ "model.layers.27.mlp_block.up_proj": {
331
+ "weights": "qint8",
332
+ "activations": "none"
333
+ },
334
+ "model.layers.27.mlp_block.down_proj": {
335
+ "weights": "qint8",
336
+ "activations": "none"
337
+ },
338
+ "model.layers.28.mlp_block.gate_proj": {
339
+ "weights": "qint8",
340
+ "activations": "none"
341
+ },
342
+ "model.layers.28.mlp_block.up_proj": {
343
+ "weights": "qint8",
344
+ "activations": "none"
345
+ },
346
+ "model.layers.28.mlp_block.down_proj": {
347
+ "weights": "qint8",
348
+ "activations": "none"
349
+ },
350
+ "model.layers.29.mlp_block.gate_proj": {
351
+ "weights": "qint8",
352
+ "activations": "none"
353
+ },
354
+ "model.layers.29.mlp_block.up_proj": {
355
+ "weights": "qint8",
356
+ "activations": "none"
357
+ },
358
+ "model.layers.29.mlp_block.down_proj": {
359
+ "weights": "qint8",
360
+ "activations": "none"
361
+ },
362
+ "model.layers.30.mlp_block.gate_proj": {
363
+ "weights": "qint8",
364
+ "activations": "none"
365
+ },
366
+ "model.layers.30.mlp_block.up_proj": {
367
+ "weights": "qint8",
368
+ "activations": "none"
369
+ },
370
+ "model.layers.30.mlp_block.down_proj": {
371
+ "weights": "qint8",
372
+ "activations": "none"
373
+ },
374
+ "model.layers.31.mlp_block.gate_proj": {
375
+ "weights": "qint8",
376
+ "activations": "none"
377
+ },
378
+ "model.layers.31.mlp_block.up_proj": {
379
+ "weights": "qint8",
380
+ "activations": "none"
381
+ },
382
+ "model.layers.31.mlp_block.down_proj": {
383
+ "weights": "qint8",
384
+ "activations": "none"
385
+ },
386
+ "model.layers.32.mlp_block.gate_proj": {
387
+ "weights": "qint8",
388
+ "activations": "none"
389
+ },
390
+ "model.layers.32.mlp_block.up_proj": {
391
+ "weights": "qint8",
392
+ "activations": "none"
393
+ },
394
+ "model.layers.32.mlp_block.down_proj": {
395
+ "weights": "qint8",
396
+ "activations": "none"
397
+ },
398
+ "model.layers.33.mlp_block.gate_proj": {
399
+ "weights": "qint8",
400
+ "activations": "none"
401
+ },
402
+ "model.layers.33.mlp_block.up_proj": {
403
+ "weights": "qint8",
404
+ "activations": "none"
405
+ },
406
+ "model.layers.33.mlp_block.down_proj": {
407
+ "weights": "qint8",
408
+ "activations": "none"
409
+ },
410
+ "model.layers.34.mlp_block.gate_proj": {
411
+ "weights": "qint8",
412
+ "activations": "none"
413
+ },
414
+ "model.layers.34.mlp_block.up_proj": {
415
+ "weights": "qint8",
416
+ "activations": "none"
417
+ },
418
+ "model.layers.34.mlp_block.down_proj": {
419
+ "weights": "qint8",
420
+ "activations": "none"
421
+ },
422
+ "model.layers.35.mlp_block.gate_proj": {
423
+ "weights": "qint8",
424
+ "activations": "none"
425
+ },
426
+ "model.layers.35.mlp_block.up_proj": {
427
+ "weights": "qint8",
428
+ "activations": "none"
429
+ },
430
+ "model.layers.35.mlp_block.down_proj": {
431
+ "weights": "qint8",
432
+ "activations": "none"
433
+ },
434
+ "model.layers.36.mlp_block.gate_proj": {
435
+ "weights": "qint8",
436
+ "activations": "none"
437
+ },
438
+ "model.layers.36.mlp_block.up_proj": {
439
+ "weights": "qint8",
440
+ "activations": "none"
441
+ },
442
+ "model.layers.36.mlp_block.down_proj": {
443
+ "weights": "qint8",
444
+ "activations": "none"
445
+ },
446
+ "model.layers.37.mlp_block.gate_proj": {
447
+ "weights": "qint8",
448
+ "activations": "none"
449
+ },
450
+ "model.layers.37.mlp_block.up_proj": {
451
+ "weights": "qint8",
452
+ "activations": "none"
453
+ },
454
+ "model.layers.37.mlp_block.down_proj": {
455
+ "weights": "qint8",
456
+ "activations": "none"
457
+ },
458
+ "lm_head": {
459
+ "weights": "qint8",
460
+ "activations": "none"
461
+ }
462
+ }