基于厚德云部署的百川大模型编写Django后台服务

部署百川大模型

Linux环境下部署百度百川AI大模型-基于厚德云-CSDN博客

步骤一：编写后台测试代码

cd /workspace/Baichuan2/

cp cli_demo.py cli.py

将cli.py代码修改为如下内容

import os
import torch
import platform
import subprocess
from colorama import Fore, Style
from tempfile import NamedTemporaryFile
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation.utils import GenerationConfig


def init_model():
    print("init model ...")
    model = AutoModelForCausalLM.from_pretrained(
        "baichuan-inc/Baichuan2-13B-Chat",
        torch_dtype=torch.float16,
        device_map="auto",
        trust_remote_code=True
    )
    model.generation_config = GenerationConfig.from_pretrained(
        "baichuan-inc/Baichuan2-13B-Chat"
    )
    tokenizer = AutoTokenizer.from_pretrained(
        "baichuan-inc/Baichuan2-13B-Chat",
        use_fast=False,
        trust_remote_code=True
    )
    return model, tokenizer


def start(content):
    model, tokenizer = init_model()
    messages = []
    messages.append({"role": "user", "content": content})
    response = model.chat(tokenizer, messages)
    print(response)


if __name__ == "__main__":
    start("背诵一下将进酒")

注：编写思维参考 cli_demo.py与魔搭社区

测试代码

cd /workspace/Baichuan2/
python cli.py

结果如下： 

步骤二：下载依赖包

pip install django -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install requests
pip install urllib3
pip install openpyxl -i https://pypi.tuna.tsinghua.edu.cn/simple

步骤三：创建django工程

cd /workspace/
django-admin startproject soft863ai
cd /workspace/soft863ai/
python manage.py startapp sourceai

步骤四：编写代码 

cd /workspace/soft863ai/sourceai/
mkdir model
cd model
touch baichuan.py

在baichuan.py添加代码如下

import os
import torch
import platform
import subprocess
from colorama import Fore, Style
from tempfile import NamedTemporaryFile
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation.utils import GenerationConfig


def init_model():
    print("init model ...")
    model = AutoModelForCausalLM.from_pretrained(
        "/workspace/Baichuan2/baichuan-inc/Baichuan2-13B-Chat",
        torch_dtype=torch.float16,
        device_map="auto",
        trust_remote_code=True
    )
    model.generation_config = GenerationConfig.from_pretrained(
        "/workspace/Baichuan2/baichuan-inc/Baichuan2-13B-Chat"
    )
    tokenizer = AutoTokenizer.from_pretrained(
        "/workspace/Baichuan2/baichuan-inc/Baichuan2-13B-Chat",
        use_fast=False,
        trust_remote_code=True
    )
    return model, tokenizer




def chat(content):
    model, tokenizer = init_model()
    messages = []
    messages.append({"role": "user", "content": content})
    response = model.chat(tokenizer, messages)
    return response



if __name__ == "__main__":
    print(chat("你是谁"))

测试代码

cd /workspace/soft863ai/sourceai/model/ 
python baichuan.py

cd /workspace/soft863ai/sourceai
vim views.py

修改/workspace/soft863ai/sourceai/views.py，代码如下

from django.shortcuts import render
from django.http import HttpResponse
from sourceai.model import baichuan
import json
# Create your views here.


def baichuan_to_create(request):
    print('进入后台..')
    context = request.GET.get('context', '您好')
    print('开始调用大模型..')
    result = baichuan.chat(context)
    print('执行完毕：',result)

    return HttpResponse(json.dumps({"res": result},ensure_ascii=False))

def first(request):
    print('进入后台测试')
    context = request.GET.get('context', '您好')
    result = context
    return HttpResponse(json.dumps({"res": result},ensure_ascii=False))

cd /workspace/soft863ai/soft863ai
vi urls.py

修改urls

"""
URL configuration for soft863ai project.

The `urlpatterns` list routes URLs to views. For more information please see:
    https://docs.djangoproject.com/en/5.1/topics/http/urls/
Examples:
Function views
    1. Add an import:  from my_app import views
    2. Add a URL to urlpatterns:  path('', views.home, name='home')
Class-based views
    1. Add an import:  from other_app.views import Home
    2. Add a URL to urlpatterns:  path('', Home.as_view(), name='home')
Including another URLconf
    1. Import the include() function: from django.urls import include, path
    2. Add a URL to urlpatterns:  path('blog/', include('blog.urls'))
"""
from django.contrib import admin
from django.urls import path
from sourceai import views

urlpatterns = [
    path('admin/', admin.site.urls),
    path('first', views.first),
    path('baichuanto', views.baichuan_to_create),
]

步骤五：修改django配置

vi settings.py 

修改settings.py文件，第28行修改为

ALLOWED_HOSTS = ['211.93.18.69']

settings.py修改MIDDLEWAR内容如下

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    #'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

重点针对#'django.middleware.csrf.CsrfViewMiddleware',行注释掉

步骤六：启动django服务

cd /workspace/soft863ai/
python manage.py runserver 0.0.0.0:8090

新启动一个窗体，输入

curl http://127.0.0.1:8090/baichuanto?context=hello

步骤七：设置外网端口号

步骤八：测试后台服务

浏览器输入

http://211.93.18.69:54548/first?context=%E4%BD%A0%E5%A5%BD

http://211.93.18.69:54548/baichuanto?context=%E4%BD%A0%E6%98%AF%E8%B0%81