第一周-山东大学软件学院2024创新项目实训VCRS个人周报

第一周

首先，系统的核心功能之一就是对话问答。通过观察和实践，决定使用Streamlit框架来实现对话问答相关功能的前端开发。

Streamlit框架

Streamlit lets you transform Python scripts into interactive web apps in minutes, instead of weeks. Build dashboards, generate reports, or create chat apps. Once you’ve created an app, you can use our Community Cloud platform to deploy, manage, and share your app.

Streamlit Github 地址

Streamlit 官方文档

许多著名的对话语言模型的前端开发都是使用的Streamlit框架开发的，例如：

• GPT-3 Playground
• ChatGPT-4
• Hugging Face Transformers
• StyleGAN Image Generation

安装Streamlit并启动演示实例：在项目文件夹下终端输入

pip install streamlit
streamlit hello

 

---

 对话界面的实现

import streamlit as st
import requests
import json

if "messages" not in st.session_state:
    st.session_state["messages"] = []

for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

if prompt := st.chat_input("How can I help you?"):
    with st.chat_message("User"):
        st.write(f"{prompt}")
    st.session_state.messages.append({"role": "user", "content": prompt})
    chat_url = 'http://localhost:5000/chat'
    response = requests.post(chat_url, json=st.session_state.messages)
    if response.status_code == 200:
        result = response.json()["result"]
    else:
        st.write('Error:', response.status_code)
    with st.chat_message("assistant"):
        st.session_state.messages.append({"role": "assistant", "content": result})
        st.write(result)

由于模型的微调与前后端的开发同步进行，因此在这里为了方便测试对话功能，通过ModelScope本地部署了一个轻量化大语言模型。

ModelScope Library是魔搭社区提供的一个能够快速、方便的使用社区提供的各类模型的Python library，其中包含了ModelScope官方模型的实现。

 ModelScope Library

由于仅做测试且部署在本地，因此选择了较小的大语言模型通义千问1.5-0.5B-Chat

Qwen1.5 is a language model series including decoder language models of different model sizes. For each size, we release the base language model and the aligned chat model. It is based on the Transformer architecture with SwiGLU activation, attention QKV bias, group query attention, mixture of sliding window attention and full attention, etc. Additionally, we have an improved tokenizer adaptive to multiple natural languages and codes. For the beta version, temporarily we did not include GQA and the mixture of SWA and full attention.

通义千问1.5-0.5B-Chat

from modelscope import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "qwen/Qwen1.5-0.5B-Chat",
    device_map="cuda"
)
tokenizer = AutoTokenizer.from_pretrained("qwen/Qwen1.5-0.5B-Chat")

测试效果图