ChatGLM2+Langchain 构建本地知识库——Langchain加载

参考视频：ChatGLM+Langchain构建本地知识库，只需6G显存，支持实时上传文档_哔哩哔哩_bilibili

一、启动API服务

在ChatGLM2项目下有个 api.py 文件，先将文件内的模型文件改为自己本地的模型文件，同时将host 改为 127.0.0.1 。

if __name__ == '__main__':
    tokenizer = AutoTokenizer.from_pretrained("/home/ChatGLM2-6B/model//chatglm2-6b", trust_remote_code=True)
    model = AutoModel.from_pretrained("/home/ChatGLM2-6B/model/chatglm2-6b", trust_remote_code=True, device='cuda')
    model.eval()
    uvicorn.run(app, host='127.0.0.1', port=8000, workers=1)

启动 api.py

$ python api.py
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:12<00:00,  1.83s/it]
INFO:     Started server process [340455]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

 

二、检验 API 接入

创建 api_access.py

import requests

def chat(prompt,history):
    resp = requests.post(
            url = 'http://127.0.0.1:8000',
            json = {"prompt":prompt, "history":history},
            headers = {"Content-Type":"application/json;charset=utf-8"}
         )
    return resp.json()['response'], resp.json()['history']

history = []

while True:
    response, history = chat(input("Question:"), history)
    print('Answer:', response)

运行 api_access.py

$ python api_access.py
Question:你好
Answer: 你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
[2024-03-22 13:48:57] ", prompt:"你好", response:"'你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。'"
INFO:     127.0.0.1:55462 - "POST / HTTP/1.1" 200 OK

api.py 成功接收到了请求。

三、Langchain 中接入

$ pip install langchain

 配置

model_path = "/home/david/20240207/ChatGLM2-6B/model/chatglm2-6b/"

from langchain.llms import ChatGLM

endpoint_url = "http://127.0.0.1:8000"

llm = ChatGLM(
        endpoint_url = endpoint_url,
        max_token = 80000,
        top_p = 0.9
        )
~