【ChatGLM】大模型之 ChatGLM 部署

目录

1. 资源下载

2. 部署启动

---

1. 资源下载

• HuggingFace 模型权重下载

# install git-lfs
git lfs install  

# download checkpoint
# clone the repo
git clone https://huggingface.co/THUDM/chatglm-6b

• 手动模型权重下载

# download checkpoint 
# clone the repo, skip large files, large files need to download separately
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/THUDM/chatglm-6b
# download large files from thu cloud
git clone https://github.com/chenyifanthu/THU-Cloud-Downloader.git
cd THU-Cloud-Downloader/
python main.py \
--link https://cloud.tsinghua.edu.cn/d/fb9f16d6dc8f482596c2/ \
--save ../chatglm-6b

• 源码下载

# clone source code
git clone https://github.com/THUDM/ChatGLM-6B.git
cd ChatGLM2-6B
pip install -r requirements.txt

2. 部署启动

• 源码修改

对于 `cli_demo.py` 和 `web_demo.py` 中模型加载部分需要从 HuggingFace 下载

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).half().cuda()

改为从本地加载

tokenizer = AutoTokenizer.from_pretrained("../chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("../chatglm-6b", trust_remote_code=True).half().cuda()

• 命令行启动

# command line (specify model path)
python cli_demo.py

• 网页启动

# web by Gradio (specify model path)
python web_demo.py

启动后的效果：