ChatGLM 本地部署指南（问题解决）

硬件要求（模型推理）：
INT4 ： RTX3090*1，显存24GB，内存32GB，系统盘200GB
如果你没有 GPU 硬件的话，也可以在 CPU 上进行推理，但是推理速度会更慢。
模型微调硬件要求更高。一般不建议个人用户环境使用

环境准备

模型准备

git clone https://hf-mirror.com/THUDM/chatglm3-6b

手动下载以下几个模型（体验时几个模型不一定需全下载）
下载地址：https://hf-mirror.com/THUDM
THUDM/chatglm3-6b

下载源码

git clone https://github.com/THUDM/ChatGLM3.git; 
cd ChatGLM3

创建conda环境

conda create -n chatglm3 python=3.10 -y 
source activate chatglm3

修改本国内源

pip config set global.index-url http://mirrors.aliyun.com/pypi/simple
pip config set install.trusted-host mirrors.aliyun.com

安装依赖库

pip install -r requirements.txt

运行

streamlim方式启动

streamlim run composite_demo/main.py

第一次运行时可能存在部分依赖包没有完整的下载的情况，根据错误提示，分别安装缺失的依赖库：

上面的错误主要是huggingface_hub 的版本问题。指定版本huggingface_hub==0.21.4， 再次执行即可。

(chatglm3) develop@itserver03:/u01/workspace/ChatGLM3$ pip install huggingface_hub==0.21.4
Looking in indexes: http://mirrors.aliyun.com/pypi/simple
Collecting huggingface_hub==0.21.4
  Downloading http://mirrors.aliyun.com/pypi/packages/ab/28/d4b691840d73126d4c9845f8a22dad033ac872509b6d3a0d93b456eef424/huggingface_hub-0.21.4-py3-none-any.whl (346 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.4/346.4 kB 4.6 MB/s eta 0:00:00
Requirement already satisfied: filelock in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (3.14.0)
Requirement already satisfied: fsspec>=2023.5.0 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (2024.3.1)
Requirement already satisfied: requests in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (2.31.0)
Requirement already satisfied: tqdm>=4.42.1 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (4.66.4)
Requirement already satisfied: pyyaml>=5.1 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (6.0.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (4.11.0)
Requirement already satisfied: packaging>=20.9 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (23.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (2024.2.2)
Installing collected packages: huggingface_hub
  Attempting uninstall: huggingface_hub
    Found existing installation: huggingface-hub 0.23.0
    Uninstalling huggingface-hub-0.23.0:
      Successfully uninstalled huggingface-hub-0.23.0
Successfully installed huggingface_hub-0.21.4
(chatglm3) develop@itserver03:/u01/workspace/ChatGLM3$ streamlit run composite_demo/main.py

启动时模型加载可以离线下载，也可以自动下载，（根据自身网络情况准备）

• 自动联网下载，启动即可自动完成下载，不需要作任何更改

• 离线下载

如果事先已经准备好模型，需要更改模型家在路径，修改文件composite_demo/client.py，把地址改为自己模型的绝对路径即可：

MODEL_PATH = os.environ.get(‘MODEL_PATH’, ‘THUDM/chatglm3-6b’)
更改为：
MODEL_PATH = os.environ.get(‘MODEL_PATH’, ‘/u01/workspace/chatglm3/models/chatglm3-6b’),
/u01/workspace/chatglm3/models/chatglm3-6b 请据实修改

MODEL_PATH = os.environ.get('MODEL_PATH', '/u01/workspace/chatglm3/models/chatglm3-6b')
PT_PATH = os.environ.get('PT_PATH', None)
PRE_SEQ_LEN = int(os.environ.get("PRE_SEQ_LEN", 128))
TOKENIZER_PATH = os.environ.get("TOKENIZER_PATH", MODEL_PATH)

成功加载后可以打开界面
http://localhost:8501/

部署 API 服务

启动 API 服务，服务的默认端口是 8000：

python openai_api_demo/api_server.py

启动服务端之后，可以通过curl验证

(chatglm3) develop@itserver03:/u01/workspace/chatglm3/ChatGLM3$ curl -X POST "http://127.0.0.1:8000/v1/chat/completions" -H "Content-Type: application/json" -d "{\"model\": \"chatglm3-6b\", \"messages\": [{\"role\": \"system\", \"content\": \"You are ChatGLM3, a large language model trained by Zhipu.AI. Follow the user's instructions carefully. Respond using markdown.\"}, {\"role\": \"user\", \"content\": \"你好，给我讲一个故事，大概100字\"}], \"stream\": false, \"max_tokens\": 100, \"temperature\": 0.8, \"top_p\": 0.8}"
{"model":"chatglm3-6b","id":"","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","content":"从前，有一个美丽的村庄，里面住着一个善良的姑娘。她每天都会去河边给河流唱歌，她的歌声感动了河神，于是河神赐予她一片肥沃的土地。她感激不已，用这片土地种植了各种美丽的花草，还养了很多动物。村子的人们都欣赏她的智慧和勤劳，欢迎她成为村子的领袖。她的生活越来越美好，村子也越来越繁荣。","name":null,"function_call":null},"finish_reason":"stop"}],"created":1715245642,"usage":{"prompt_tokens":54,"total_tokens":141,"completion_tokens":87}}

---

【Qinghub Studio 】更适合开发人员的低代码开源开发平台
【QingHub企业级应用统一部署】
【QingHub企业级应用开发管理】
【QingHub** 演示】**
【https://qingplus.cn】