【LLM 部署chatglm3完整教程超详细（国内急速下载modelscope）】

1. 安装modelscope包

$ pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
$ pip install modelscope

• 可选安装lfs(linux)，为了使用git clone下载权重文件

$ curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
$ sudo apt-get install git-lfs && git lfs install
$ source /etc/network_turbo #仅限autodl

2. 下载权重

创建download.py

from modelscope import snapshot_download
# chatglm3-6b模型下载
model_dir = snapshot_download(
    'ZhipuAI/chatglm3-6b', cache_dir='chatglm3-6b-chat'
)
# or $ git clone https://www.modelscope.cn/ZhipuAI/chatglm3-6b.git

# embedding模型下载
from modelscope import snapshot_download
model_dir = snapshot_download(
    'AI-ModelScope/bge-large-zh-v1.5', cache_dir='bge-large-zh-v1.5'
)
# or $ git clone https://huggingface.co/BAAI/bge-large-zh-v1.5

# 权重下载
$ python download.py
# 代码下载
$ git clone https://github.com/THUDM/ChatGLM3.git

3. 启动openai-api接口

注意修改路径
注意修改bge-large-zh-v1__5为bge-large-zh-v1.5

$ cd /path/to/ChatGLM3/openai_api_demo
$ MODEL_PATH=/path/to/chatglm3-6b EMBEDDING_PATH=/path/to/bge-large-zh-v1.5 python api_server.py

4. 测试代码

from openai import Client

# 其他模型
model = Client(
    api_key='x',  # 任何值均可，不能空
    base_url='http://0.0.0.0:8000/v1/'  # 必须加 `/v1/`
)

def chat(msgs, model_name='chatglm3-6b', rt_content=False):
    resp = model.chat.completions.create(
            model=model_name,
            messages=msgs
        )
    if rt_content:
        return resp.choices[0].message.content
        
    return resp

messages=[
    {'role':'user','content':'1+1=?'}
]
chat(messages, rt_content=True)
# '1+1=2.'