chatglm3-6b 模型部署+结合flask进行调用

最新推荐文章于 2024-07-03 22:36:57 发布

十二有bug

最新推荐文章于 2024-07-03 22:36:57 发布

阅读量265

点赞数

文章标签： flask python 后端语言模型

本文链接：https://blog.csdn.net/qq_33403388/article/details/134196259

版权

模型下载
git lfs install
git clone https://www.modelscope.cn/ZhipuAI/chatglm3-6b.git

from flask import  Flask,jsonify,request
from transformers import AutoTokenizer, AutoModel
app = Flask(__name__)
tokenizer = AutoTokenizer.from_pretrained("E:/yi\chatglm3-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("E:/yi\chatglm3-6b", trust_remote_code=True).half().cuda()
model = model.eval()
@app.route('/goodsApi',methods=['get'])
def transaction():
    item ={}

    title = request.args.get("title")
    title1 = request.args.get("title1")
    
    response, history = model.chat(tokenizer,title, history=[])
    response, history = model.chat(tokenizer, title+'和'+title1+'  这两句话的相似度是多少', history=[])

    item['data'] =response

    return item,200


if __name__ == '__main__':
        app.run(host='0.0.0.0', port=5000)