Commit
·
476c22a
1
Parent(s):
2820e4b
Update README.md
Browse files
README.md
CHANGED
|
@@ -23,12 +23,21 @@ tasks:
|
|
| 23 |
## GPTQ参数
|
| 24 |
该GPTQ文件都是用AutoGPTQ生成的。
|
| 25 |
|
| 26 |
-
- Bits: 4
|
| 27 |
-
- GS: 128
|
| 28 |
- Act Order: True
|
| 29 |
-
- Damp %: 0.
|
| 30 |
- GPTQ dataset: 中文、英文混合数据集
|
| 31 |
- Sequence Length: 4096
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
<!-- README_GPTQ.md-provided-files end -->
|
| 34 |
## 如何在Python代码中使用此GPTQ模型
|
|
|
|
| 23 |
## GPTQ参数
|
| 24 |
该GPTQ文件都是用AutoGPTQ生成的。
|
| 25 |
|
| 26 |
+
- Bits: 4/8
|
| 27 |
+
- GS: 32/128
|
| 28 |
- Act Order: True
|
| 29 |
+
- Damp %: 0.1
|
| 30 |
- GPTQ dataset: 中文、英文混合数据集
|
| 31 |
- Sequence Length: 4096
|
| 32 |
+
|
| 33 |
+
| 模型版本 | agieval | ceval | cmmlu | size | 推理速度 |
|
| 34 |
+
|---|---|---|---|---|---|
|
| 35 |
+
| [Baichuan2-7B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat) | 34.63 | 54.28 | 54.14 | 15g | |
|
| 36 |
+
| [Baichuan2-7B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat-4bits) | 34.46 | 51.01 | 51.55 | 5.4g | |
|
| 37 |
+
| [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/4bit-32g) | 34.11 | 52.74 | 52.92 | 5.8g | |
|
| 38 |
+
| [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/main) | 33.42 | 52.17 | 52.05 | 5.4g | |
|
| 39 |
+
| [GPTQ-8bit-32g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-32g) | | | | 9.2g | |
|
| 40 |
+
| [GPTQ-8bit-128g](https://huggingface.co/csdc-atl/Baichuan2-7B-Chat-GPTQ-Int4/tree/8bit-128g) | | | | 8.7g | |
|
| 41 |
|
| 42 |
<!-- README_GPTQ.md-provided-files end -->
|
| 43 |
## 如何在Python代码中使用此GPTQ模型
|