ChatGLM3-6B
๐ป Github Repo โข ๐ฆ Twitter โข ๐ [GLM@ACL 22] [GitHub] โข ๐ [GLM-130B@ICLR 23] [GitHub]
๐ Join our Slack and WeChat
๐Experience the larger-scale ChatGLM model at chatglm.cn
GLM-4 ๅผๆบๆจกๅ
ๆไปฌๅทฒ็ปๅๅธๆๆฐ็ GLM-4 ๆจกๅ๏ผ่ฏฅๆจกๅๅจๅคไธชๆๆ ไธๆไบๆฐ็็ช็ ด๏ผๆจๅฏไปฅๅจไปฅไธไธคไธชๆธ ้ไฝ้ชๆไปฌ็ๆๆฐๆจกๅใ
- GLM-4 ๅผๆบๆจกๅ ๆไปฌๅทฒ็ปๅผๆบไบ GLM-4-9B ็ณปๅๆจกๅ๏ผๅจๅ้กนๆๆ ็ๆต่ฏไธๆๆๆพๆๅ๏ผๆฌข่ฟๅฐ่ฏใ
ไป็ป (Introduction)
ChatGLM3-6B ๆฏ ChatGLM ็ณปๅๆๆฐไธไปฃ็ๅผๆบๆจกๅ๏ผๅจไฟ็ไบๅไธคไปฃๆจกๅๅฏน่ฏๆต็ ใ้จ็ฝฒ้จๆงไฝ็ญไผๅคไผ็ง็นๆง็ๅบ็กไธ๏ผChatGLM3-6B ๅผๅ ฅไบๅฆไธ็นๆง๏ผ
- ๆดๅผบๅคง็ๅบ็กๆจกๅ๏ผ ChatGLM3-6B ็ๅบ็กๆจกๅ ChatGLM3-6B-Base ้็จไบๆดๅคๆ ท็่ฎญ็ปๆฐๆฎใๆดๅ ๅ็่ฎญ็ปๆญฅๆฐๅๆดๅ็็่ฎญ็ป็ญ็ฅใๅจ่ฏญไนใๆฐๅญฆใๆจ็ใไปฃ็ ใ็ฅ่ฏ็ญไธๅ่งๅบฆ็ๆฐๆฎ้ไธๆต่ฏๆพ็คบ๏ผChatGLM3-6B-Base ๅ ทๆๅจ 10B ไปฅไธ็้ข่ฎญ็ปๆจกๅไธญๆๅผบ็ๆง่ฝใ
- ๆดๅฎๆด็ๅ่ฝๆฏๆ๏ผ ChatGLM3-6B ้็จไบๅ จๆฐ่ฎพ่ฎก็ Prompt ๆ ผๅผ๏ผ้คๆญฃๅธธ็ๅค่ฝฎๅฏน่ฏๅคใๅๆถๅ็ๆฏๆๅทฅๅ ท่ฐ็จ๏ผFunction Call๏ผใไปฃ็ ๆง่ก๏ผCode Interpreter๏ผๅ Agent ไปปๅก็ญๅคๆๅบๆฏใ
- ๆดๅ จ้ข็ๅผๆบๅบๅ๏ผ ้คไบๅฏน่ฏๆจกๅ ChatGLM3-6B ๅค๏ผ่ฟๅผๆบไบๅบ็กๆจกๅ ChatGLM-6B-Baseใ้ฟๆๆฌๅฏน่ฏๆจกๅ ChatGLM3-6B-32Kใไปฅไธๆๆๆ้ๅฏนๅญฆๆฏ็ ็ฉถๅฎๅ จๅผๆพ๏ผๅจๅกซๅ้ฎๅท่ฟ่ก็ป่ฎฐๅไบฆๅ ่ฎธๅ ่ดนๅไธไฝฟ็จใ
ChatGLM3-6B is the latest open-source model in the ChatGLM series. While retaining many excellent features such as smooth dialogue and low deployment threshold from the previous two generations, ChatGLM3-6B introduces the following features:
- More Powerful Base Model: The base model of ChatGLM3-6B, ChatGLM3-6B-Base, employs a more diverse training dataset, more sufficient training steps, and a more reasonable training strategy. Evaluations on datasets such as semantics, mathematics, reasoning, code, knowledge, etc., show that ChatGLM3-6B-Base has the strongest performance among pre-trained models under 10B.
- More Comprehensive Function Support: ChatGLM3-6B adopts a newly designed Prompt format, in addition to the normal multi-turn dialogue. It also natively supports function call, code interpreter, and complex scenarios such as agent tasks.
- More Comprehensive Open-source Series: In addition to the dialogue model ChatGLM3-6B, the base model ChatGLM-6B-Base and the long-text dialogue model ChatGLM3-6B-32K are also open-sourced. All the weights are fully open for academic research, and after completing the questionnaire registration, they are also allowed for free commercial use.
่ฝฏไปถไพ่ต (Dependencies)
pip install protobuf transformers==4.30.2 cpm_kernels torch>=2.0 gradio mdtex2html sentencepiece accelerate
ไปฃ็ ่ฐ็จ (Code Usage)
ๅฏไปฅ้่ฟๅฆไธไปฃ็ ่ฐ็จ ChatGLM3-6B ๆจกๅๆฅ็ๆๅฏน่ฏ๏ผ
You can generate dialogue by invoking the ChatGLM3-6B model with the following code:
>>> from transformers import AutoTokenizer, AutoModel
>>> tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True)
>>> model = AutoModel.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True).half().cuda()
>>> model = model.eval()
>>> response, history = model.chat(tokenizer, "ไฝ ๅฅฝ", history=[])
>>> print(response)
ไฝ ๅฅฝ๐!ๆๆฏไบบๅทฅๆบ่ฝๅฉๆ ChatGLM-6B,ๅพ้ซๅ
ด่งๅฐไฝ ,ๆฌข่ฟ้ฎๆไปปไฝ้ฎ้ขใ
>>> response, history = model.chat(tokenizer, "ๆไธ็กไธ็ๅบ่ฏฅๆไนๅ", history=history)
>>> print(response)
ๆไธ็กไธ็ๅฏ่ฝไผ่ฎฉไฝ ๆๅฐ็ฆ่ๆไธ่ๆ,ไฝไปฅไธๆฏไธไบๅฏไปฅๅธฎๅฉไฝ ๅ
ฅ็ก็ๆนๆณ:
1. ๅถๅฎ่งๅพ็็ก็ ๆถ้ด่กจ:ไฟๆ่งๅพ็็ก็ ๆถ้ด่กจๅฏไปฅๅธฎๅฉไฝ ๅปบ็ซๅฅๅบท็็ก็ ไน ๆฏ,ไฝฟไฝ ๆดๅฎนๆๅ
ฅ็กใๅฐฝ้ๅจๆฏๅคฉ็็ธๅๆถ้ดไธๅบ,ๅนถๅจๅไธๆถ้ด่ตทๅบใ
2. ๅ้ ไธไธช่้็็ก็ ็ฏๅข:็กฎไฟ็ก็ ็ฏๅข่้,ๅฎ้,้ปๆไธๆธฉๅบฆ้ๅฎใๅฏไปฅไฝฟ็จ่้็ๅบไธ็จๅ,ๅนถไฟๆๆฟ้ด้้ฃใ
3. ๆพๆพ่บซๅฟ:ๅจ็กๅๅไบๆพๆพ็ๆดปๅจ,ไพๅฆๆณกไธช็ญๆฐดๆพก,ๅฌไบ่ฝปๆ็้ณไน,้
่ฏปไธไบๆ่ถฃ็ไนฆ็ฑ็ญ,ๆๅฉไบ็ผ่งฃ็ดงๅผ ๅ็ฆ่,ไฝฟไฝ ๆดๅฎนๆๅ
ฅ็กใ
4. ้ฟๅ
้ฅฎ็จๅซๆๅๅกๅ ็้ฅฎๆ:ๅๅกๅ ๆฏไธ็งๅบๆฟๆง็ฉ่ดจ,ไผๅฝฑๅไฝ ็็ก็ ่ดจ้ใๅฐฝ้้ฟๅ
ๅจ็กๅ้ฅฎ็จๅซๆๅๅกๅ ็้ฅฎๆ,ไพๅฆๅๅก,่ถๅๅฏไนใ
5. ้ฟๅ
ๅจๅบไธๅไธ็ก็ ๆ ๅ
ณ็ไบๆ
:ๅจๅบไธๅไบไธ็ก็ ๆ ๅ
ณ็ไบๆ
,ไพๅฆ็็ตๅฝฑ,็ฉๆธธๆๆๅทฅไฝ็ญ,ๅฏ่ฝไผๅนฒๆฐไฝ ็็ก็ ใ
6. ๅฐ่ฏๅผๅธๆๅทง:ๆทฑๅผๅธๆฏไธ็งๆพๆพๆๅทง,ๅฏไปฅๅธฎๅฉไฝ ็ผ่งฃ็ดงๅผ ๅ็ฆ่,ไฝฟไฝ ๆดๅฎนๆๅ
ฅ็กใ่ฏ็ๆ
ขๆ
ขๅธๆฐ,ไฟๆๅ ็ง้,็ถๅ็ผๆ
ขๅผๆฐใ
ๅฆๆ่ฟไบๆนๆณๆ ๆณๅธฎๅฉไฝ ๅ
ฅ็ก,ไฝ ๅฏไปฅ่่ๅจ่ฏขๅป็ๆ็ก็ ไธๅฎถ,ๅฏปๆฑ่ฟไธๆญฅ็ๅปบ่ฎฎใ
ๅ ณไบๆดๅค็ไฝฟ็จ่ฏดๆ๏ผๅ ๆฌๅฆไฝ่ฟ่กๅฝไปค่กๅ็ฝ้กต็ๆฌ็ DEMO๏ผไปฅๅไฝฟ็จๆจกๅ้ๅไปฅ่็ๆพๅญ๏ผ่ฏทๅ่ๆไปฌ็ Github Repoใ
For more instructions, including how to run CLI and web demos, and model quantization, please refer to our Github Repo.
ๅ่ฎฎ (License)
ๆฌไปๅบ็ไปฃ็ ไพ็ ง Apache-2.0 ๅ่ฎฎๅผๆบ๏ผChatGLM3-6B ๆจกๅ็ๆ้็ไฝฟ็จๅ้่ฆ้ตๅพช Model Licenseใ
The code in this repository is open-sourced under the Apache-2.0 license, while the use of the ChatGLM3-6B model weights needs to comply with the Model License.
ๅผ็จ (Citation)
ๅฆๆไฝ ่งๅพๆไปฌ็ๅทฅไฝๆๅธฎๅฉ็่ฏ๏ผ่ฏท่่ๅผ็จไธๅ่ฎบๆใ
If you find our work helpful, please consider citing the following paper.
@misc{glm2024chatglm,
title={ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools},
author={Team GLM and Aohan Zeng and Bin Xu and Bowen Wang and Chenhui Zhang and Da Yin and Diego Rojas and Guanyu Feng and Hanlin Zhao and Hanyu Lai and Hao Yu and Hongning Wang and Jiadai Sun and Jiajie Zhang and Jiale Cheng and Jiayi Gui and Jie Tang and Jing Zhang and Juanzi Li and Lei Zhao and Lindong Wu and Lucen Zhong and Mingdao Liu and Minlie Huang and Peng Zhang and Qinkai Zheng and Rui Lu and Shuaiqi Duan and Shudan Zhang and Shulin Cao and Shuxun Yang and Weng Lam Tam and Wenyi Zhao and Xiao Liu and Xiao Xia and Xiaohan Zhang and Xiaotao Gu and Xin Lv and Xinghan Liu and Xinyi Liu and Xinyue Yang and Xixuan Song and Xunkai Zhang and Yifan An and Yifan Xu and Yilin Niu and Yuantao Yang and Yueyan Li and Yushi Bai and Yuxiao Dong and Zehan Qi and Zhaoyu Wang and Zhen Yang and Zhengxiao Du and Zhenyu Hou and Zihan Wang},
year={2024},
eprint={2406.12793},
archivePrefix={arXiv},
primaryClass={id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'}
}
- Downloads last month
- 30,986