数字分身~~

于 2023-06-30 14:16:12 发布
阅读量465 收藏
点赞数
文章标签： 深度学习
原文链接：https://aistudio.baidu.com/aistudio/projectdetail/6474982?forkThirdPart=1
版权

★★★ 本文源自AlStudio社区精品项目，【点击此处】查看更多精品内容 >>>

应用代码 Gradio_packages/main_local.gradio.py

项目地址：https://github.com/LLMLab/digital_double

ChatGLM的使用，参考了 https://aistudio.baidu.com/aistudio/projectdetail/6307966 这个项目

模型加载

时间较久，算上下载12G模型的15分钟，大约一共需要25分钟

因为用到了GPU，所以请使用V100 32G或以上规格启动本项目

import os

os.system('python -m pip install paddlepaddle-gpu==0.0.0.post112 -f https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html')
os.system('pip install --pre --upgrade paddlenlp -f https://www.paddlepaddle.org.cn/whl/paddlenlp.html')


import gradio as gr
import paddle
from paddlenlp.transformers import (
    ChatGLMConfig,
    ChatGLMForConditionalGeneration,
    ChatGLMTokenizer,
)



# 加载模型
#读取原始的chatglm-6b模型
model_name_or_path = 'THUDM/chatglm-6b'
tokenizer = ChatGLMTokenizer.from_pretrained(model_name_or_path)

config = ChatGLMConfig.from_pretrained(model_name_or_path)
paddle.set_default_dtype(config.paddle_dtype)

model = ChatGLMForConditionalGeneration.from_pretrained(
    model_name_or_path,
    tensor_parallel_degree=paddle.distributed.get_world_size(),
    tensor_parallel_rank=0,
    load_state_as_np=True,
    dtype=config.paddle_dtype,
)

model.eval()

# 函数定义，问glm问题（ask_glm）
# 输入参数：初始prompt, 最长输入长度，最长输出长度
def glm_single_QA(model,tokenizer,next_inputs,input_length,output_length):
    # 输入格式转换
    inputs = tokenizer(
        next_inputs,
        return_tensors="np",
        padding=True,
        max_length=input_length,
        truncation=True,
        truncation_side="left",
    )
    input_map = {}
    for key in inputs:
        input_map[key] = paddle.to_tensor(inputs[key])

    # 获取结果
    infer_result = model.generate(
        **input_map,
        decode_strategy="sampling",
        top_k=1,
        # top_p =5,
        max_length=output_length,
        use_cache=True,
        use_fast=True,
        use_fp16_decoding=True,
        repetition_penalty=1,
        temperature = 0.95,
        length_penalty=1,
    )[0]

    # 结果转换
    output = ''
    result = []
    for x in infer_result.tolist():
        res = tokenizer.decode(x, skip_special_tokens=True)
        res = res.strip("\n")
        result.append(res)
        output = output + res
    return output

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Looking in links: https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html
Collecting paddlepaddle-gpu==0.0.0.post112
  Downloading https://paddle-wheel.bj.bcebos.com/develop/linux/linux-gpu-cuda11.2-cudnn8-mkl-gcc8.2-avx/paddlepaddle_gpu-0.0.0.post112-cp37-cp37m-linux_x86_64.whl (575.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 575.8/575.8 MB 1.5 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.13 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==0.0.0.post112) (1.19.5)
Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==0.0.0.post112) (8.2.0)
Collecting protobuf>=3.20.2
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b9/64/32f6708e6dd4159b18ea757d907295ebe2d380009c7e08b953c5d161415e/protobuf-4.23.3-cp37-abi3-manylinux2014_x86_64.whl (304 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 304.5/304.5 kB 25.5 MB/s eta 0:00:00
Requirement already satisfied: astor in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==0.0.0.post112) (0.8.1)
Requirement already satisfied: opt-einsum==3.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==0.0.0.post112) (3.3.0)
Requirement already satisfied: httpx in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==0.0.0.post112) (0.23.3)
Requirement already satisfied: decorator in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==0.0.0.post112) (4.4.2)
Requirement already satisfied: paddle-bfloat==0.1.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlepaddle-gpu==0.0.0.post112) (0.1.7)
Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from httpx->paddlepaddle-gpu==0.0.0.post112) (0.16.3)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from httpx->paddlepaddle-gpu==0.0.0.post112) (1.5.0)
Requirement already satisfied: sniffio in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from httpx->paddlepaddle-gpu==0.0.0.post112) (1.3.0)
Requirement already satisfied: certifi in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from httpx->paddlepaddle-gpu==0.0.0.post112) (2019.9.11)
Requirement already satisfied: anyio<5.0,>=3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from httpcore<0.17.0,>=0.15.0->httpx->paddlepaddle-gpu==0.0.0.post112) (3.6.1)
Requirement already satisfied: h11<0.15,>=0.13 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from httpcore<0.17.0,>=0.15.0->httpx->paddlepaddle-gpu==0.0.0.post112) (0.14.0)
Requirement already satisfied: idna in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from rfc3986[idna2008]<2,>=1.3->httpx->paddlepaddle-gpu==0.0.0.post112) (2.8)
Requirement already satisfied: typing-extensions in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from anyio<5.0,>=3.0->httpcore<0.17.0,>=0.15.0->httpx->paddlepaddle-gpu==0.0.0.post112) (4.3.0)
Installing collected packages: protobuf, paddlepaddle-gpu
  Attempting uninstall: protobuf
    Found existing installation: protobuf 3.20.0
    Uninstalling protobuf-3.20.0:
      Successfully uninstalled protobuf-3.20.0
  Attempting uninstall: paddlepaddle-gpu
    Found existing installation: paddlepaddle-gpu 2.4.0.post112
    Uninstalling paddlepaddle-gpu-2.4.0.post112:
      Successfully uninstalled paddlepaddle-gpu-2.4.0.post112


ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
streamlit 1.13.0 requires protobuf!=3.20.2,<4,>=3.12, but you have protobuf 4.23.3 which is incompatible.
parl 1.4.1 requires pyzmq==18.1.1, but you have pyzmq 23.2.1 which is incompatible.
paddlenlp 2.4.2 requires protobuf<=3.20.0,>=3.1.0, but you have protobuf 4.23.3 which is incompatible.


Successfully installed paddlepaddle-gpu-0.0.0.post112 protobuf-4.23.3

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Looking in links: https://www.paddlepaddle.org.cn/whl/paddlenlp.html
Requirement already satisfied: paddlenlp in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2.4.2)
Collecting paddlenlp
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/26/76/32904a23a91d63b6078fb2871c2e65f6f072ad844829dd5f727f9c54bab2/paddlenlp-2.6.0rc0-py3-none-any.whl (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 18.0 MB/s eta 0:00:00
Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (4.64.1)
Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.4.4)
Requirement already satisfied: multiprocess<=0.70.12.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.70.11.1)
Requirement already satisfied: Flask-Babel in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.0.0)
Requirement already satisfied: paddlefsl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.1.0)
Collecting huggingface-hub>=0.11.1
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/62/a2/8a416d167216403ceeef3aaf8c22b0c61b1ae571644473d67eb7fecbb69e/huggingface_hub-0.15.1-py3-none-any.whl (236 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 236.8/236.8 kB 24.5 MB/s eta 0:00:00
Requirement already satisfied: rich in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (12.6.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.1.96)
Requirement already satisfied: uvicorn in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.21.1)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (4.1.0)
Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (2.4.0)
Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.42.1)
Requirement already satisfied: datasets>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (2.7.0)
Collecting typer
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/bf/0e/c68adf10adda05f28a6ed7b9f4cd7b8e07f641b44af88ba72d9c89e4de7a/typer-0.9.0-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.9/45.9 kB 9.0 MB/s eta 0:00:00
Collecting protobuf==3.20.2
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b7/f4/dcd86d49321e73404df0e181a92313c539b4fcdd5f34fc7bdfdb1c588c56/protobuf-3.20.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 33.1 MB/s eta 0:00:00
Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.2.2)
Requirement already satisfied: paddle2onnx in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.0.0)
Requirement already satisfied: fastapi in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.95.0)
Requirement already satisfied: dill<0.3.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.3.3)
Requirement already satisfied: pyarrow>=6.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (10.0.0)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (2022.11.0)
Requirement already satisfied: xxhash in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (3.1.0)
Requirement already satisfied: responses<0.19 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (0.18.0)
Requirement already satisfied: importlib-metadata in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (4.2.0)
Requirement already satisfied: requests>=2.19.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (2.24.0)
Requirement already satisfied: numpy>=1.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (1.19.5)
Requirement already satisfied: pyyaml>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (5.1.2)
Requirement already satisfied: aiohttp in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (3.8.3)
Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (21.3)
Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from datasets>=2.0.0->paddlenlp) (1.1.5)
Requirement already satisfied: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from huggingface-hub>=0.11.1->paddlenlp) (3.0.12)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from huggingface-hub>=0.11.1->paddlenlp) (4.3.0)
Requirement already satisfied: starlette<0.27.0,>=0.26.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fastapi->paddlenlp) (0.26.1)
Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fastapi->paddlenlp) (1.10.6)
Requirement already satisfied: Flask in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel->paddlenlp) (1.1.1)
Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel->paddlenlp) (2.8.0)
Requirement already satisfied: Jinja2>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel->paddlenlp) (3.0.0)
Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel->paddlenlp) (2019.3)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from rich->paddlenlp) (2.13.0)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from rich->paddlenlp) (0.9.1)
Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp) (0.24.2)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from typer->paddlenlp) (8.0.4)
Requirement already satisfied: h11>=0.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from uvicorn->paddlenlp) (0.14.0)
Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddlenlp) (2.2.3)
Requirement already satisfied: six>=1.14.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddlenlp) (1.16.0)
Requirement already satisfied: Pillow>=7.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddlenlp) (8.2.0)
Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddlenlp) (0.8.53)
Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask->Flask-Babel->paddlenlp) (0.16.0)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask->Flask-Babel->paddlenlp) (1.1.0)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (1.3.0)
Requirement already satisfied: asynctest==0.13.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (0.13.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (6.0.2)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (2.1.1)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (1.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (1.7.2)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (4.0.2)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from aiohttp->datasets>=2.0.0->paddlenlp) (22.1.0)
Requirement already satisfied: MarkupSafe>=2.0.0rc2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.5->Flask-Babel->paddlenlp) (2.0.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging->datasets>=2.0.0->paddlenlp) (3.0.9)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (2019.9.11)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (1.25.11)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests>=2.19.0->datasets>=2.0.0->paddlenlp) (2.8)
Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (1.6.3)
Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (0.14.1)
Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (2.1.0)
Requirement already satisfied: anyio<5,>=3.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from starlette<0.27.0,>=0.26.1->fastapi->paddlenlp) (3.6.1)
Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->paddlenlp) (3.9.9)
Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->paddlenlp) (0.18.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata->datasets>=2.0.0->paddlenlp) (3.8.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->visualdl->paddlenlp) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->visualdl->paddlenlp) (0.10.0)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->visualdl->paddlenlp) (2.8.2)
Requirement already satisfied: sniffio>=1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from anyio<5,>=3.4.0->starlette<0.27.0,>=0.26.1->fastapi->paddlenlp) (1.3.0)
Requirement already satisfied: setuptools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib->visualdl->paddlenlp) (56.2.0)
Installing collected packages: protobuf, huggingface-hub, typer, paddlenlp
  Attempting uninstall: protobuf
    Found existing installation: protobuf 4.23.3
    Uninstalling protobuf-4.23.3:
      Successfully uninstalled protobuf-4.23.3
  Attempting uninstall: huggingface-hub
    Found existing installation: huggingface-hub 0.11.0
    Uninstalling huggingface-hub-0.11.0:
      Successfully uninstalled huggingface-hub-0.11.0
  Attempting uninstall: paddlenlp
    Found existing installation: paddlenlp 2.4.2
    Uninstalling paddlenlp-2.4.2:
      Successfully uninstalled paddlenlp-2.4.2


ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
streamlit 1.13.0 requires protobuf!=3.20.2,<4,>=3.12, but you have protobuf 3.20.2 which is incompatible.
parl 1.4.1 requires pyzmq==18.1.1, but you have pyzmq 23.2.1 which is incompatible.


Successfully installed huggingface-hub-0.15.1 paddlenlp-2.6.0rc0 protobuf-3.20.2 typer-0.9.0

[notice] A new release of pip available: 22.1.2 -> 23.1.2
[notice] To update, run: pip install --upgrade pip


[2023-06-29 13:54:17,754] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/ice_text.model and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b
[2023-06-29 13:54:17,849] [    INFO] - Downloading ice_text.model from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/ice_text.model



  0%|          | 0.00/2.58M [00:00<?, ?B/s]


[2023-06-29 13:54:19,262] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/added_tokens.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b
[2023-06-29 13:54:19,328] [ WARNING] - file<https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/added_tokens.json> not exist
[2023-06-29 13:54:19,332] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/special_tokens_map.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b
[2023-06-29 13:54:19,403] [    INFO] - Downloading special_tokens_map.json from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/special_tokens_map.json



  0%|          | 0.00/190 [00:00<?, ?B/s]


[2023-06-29 13:54:19,498] [    INFO] - Downloading https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/tokenizer_config.json and saved to /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b
[2023-06-29 13:54:19,565] [    INFO] - Downloading tokenizer_config.json from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/tokenizer_config.json



  0%|          | 0.00/293 [00:00<?, ?B/s]


[2023-06-29 13:54:20,058] [    INFO] - Downloading config.json from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/config.json



  0%|          | 0.00/400 [00:00<?, ?B/s]


[2023-06-29 13:54:20,213] [    INFO] - loading configuration file /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b/config.json
[2023-06-29 13:54:20,216] [    INFO] - Model config ChatGLMConfig {
  "activation": "gelu",
  "attention_scale": true,
  "bos_token_id": 130004,
  "eos_token_id": 130005,
  "gmask_token_id": 130001,
  "hidden_size": 4096,
  "inner_hidden_size": 16384,
  "layernorm_epsilon": 1e-05,
  "mask_token_id": 130000,
  "max_sequence_length": 2048,
  "model_type": "chatglm",
  "num_attention_heads": 32,
  "num_hidden_layers": 28,
  "num_image_tokens": 0,
  "output_predict": true,
  "pad_token_id": 3,
  "paddle_dtype": "float16",
  "paddlenlp_version": null,
  "position_encoding_2d": true,
  "pre_seq_len": null,
  "prefix_projection": false,
  "quantization_bit": 0,
  "recompute": false,
  "use_cache": true,
  "vocab_size": 130528
}

[2023-06-29 13:54:20,274] [    INFO] - Found /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b/config.json
[2023-06-29 13:54:20,277] [    INFO] - loading configuration file /home/aistudio/.paddlenlp/models/THUDM/chatglm-6b/config.json
[2023-06-29 13:54:20,280] [    INFO] - Model config ChatGLMConfig {
  "activation": "gelu",
  "attention_scale": true,
  "bos_token_id": 130004,
  "eos_token_id": 130005,
  "gmask_token_id": 130001,
  "hidden_size": 4096,
  "inner_hidden_size": 16384,
  "layernorm_epsilon": 1e-05,
  "mask_token_id": 130000,
  "max_sequence_length": 2048,
  "model_type": "chatglm",
  "num_attention_heads": 32,
  "num_hidden_layers": 28,
  "num_image_tokens": 0,
  "output_predict": true,
  "pad_token_id": 3,
  "paddle_dtype": "float16",
  "paddlenlp_version": null,
  "position_encoding_2d": true,
  "pre_seq_len": null,
  "prefix_projection": false,
  "quantization_bit": 0,
  "recompute": false,
  "tensor_parallel_degree": 1,
  "use_cache": true,
  "vocab_size": 130528
}

[2023-06-29 13:54:20,342] [    INFO] - Downloading model_state.pdparams from https://bj.bcebos.com/paddlenlp/models/community/THUDM/chatglm-6b/model_state.pdparams



  0%|          | 0.00/12.5G [00:00<?, ?B/s]


W0629 14:06:07.164129   187 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2
W0629 14:06:07.168759   187 gpu_resources.cc:149] device: 0, cuDNN Version: 8.2.
[2023-06-29 14:06:40,271] [    INFO] - All model checkpoint weights were used when initializing ChatGLMForConditionalGeneration.

[2023-06-29 14:06:40,275] [ WARNING] - Some weights of ChatGLMForConditionalGeneration were not initialized from the model checkpoint at THUDM/chatglm-6b and are newly initialized: ['transformer.rotary_embeddings.inv_freq']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

对话函数

将ChatGLM的模型封装成对话形式

def chat_glm(prompt, his, prompt_his_str='你：{}\n分身：{}'):
    Q_his = '\n'.join([prompt_his_str.format(i[0], i[1]) for i in his])
    if Q_his:
        Q_his += '\n'
    Q_motif = f'{Q_his}${prompt}'
    result=glm_single_QA(model,tokenizer,Q_motif,256,256)
    return result

下面是chatgpt的接口，可做参考对比

import requests

def chat_gpt(input, history=[], prompt_his_str=''):
  headers = {
    "Authorization": "Bearer sk-TmUhOQKWsJ5t43QVoGBblSw3GFOMZwZhpGFlCGX7jxwedsdN"
  }
  _history = []
  for item in history:
    _history.append({"role": "user", "content": item[0]})
    _history.append({"role": "assistant", "content": item[1]})
  if prompt_his_str:
    _history = []
    prompt_his = '\n'.join([prompt_his_str.format(item[0], item[1]) for item in history])
    input = f'{prompt_his}\n{input}'
  j = {
    "model": "gpt-3.5-turbo",
    "messages": [*_history, {"role": "user", "content": input}],
    "temperature": 0.01
  }
  result = requests.post('https://api.aiproxy.io/v1/chat/completions', json=j, headers=headers)
  return result.json()['choices'][0]['message']['content']

# res = chat('你好')
# print(res)

效果测试

运行下面的代码，输入你想和数字人聊的话，即可测试效果。相关prompt可以按需调整

import re

chat = chat_glm # chat_glm | chat_gpt

def dd_chat(dd_prompt, dd_his_prompt, dd_name, dd_desc, dd_example_chat, msg, his):
    dd_soul = dd_prompt.format(name=dd_name, desc=dd_desc, chat=dd_example_chat)
    dd_his_prompt = dd_his_prompt.replace('{name}', dd_name)
    _his = [[dd_soul, ''], *his]
    _res = chat(msg, _his, prompt_his_str=dd_his_prompt)
    _res = re.sub(dd_name+'[\:：]', '', _res)
    _res = _res.split('用户:')[0].split('用户：')[0]
    his.append([msg, _res])
    return '', his 

dd_prompt = '''我们来玩一个角色扮演游戏。
----
下面是{name}的介绍：
  {desc}
----
下面是{name}和别人的对话：
  {chat}
----
请针对上文，扮演{name}，回复简洁，和我聊天'''
dd_his_prompt = '用户：{}\n{name}：{}'
dd_name = '哆啦A梦'
dd_desc = '有各种各样的道具，喜欢帮助别人，喜欢吃铜锣烧'
dd_example_chat = '大雄，你怎么又闯祸了！！！\ndengdengdeng，魔法斗篷\ndingdingding，缩小灯\n啊啊啊啊！有老鼠！！！'
his = []

while True:
    msg = input('用户：')
    _, his = dd_chat(dd_prompt, dd_his_prompt, dd_name, dd_desc, dd_example_chat, msg, his)
    print(f'{dd_name}：{his[-1][1]}')


用户： 你好


哆啦A梦： 你好!有什么我可以帮助你的吗?


用户： 你在做啥？


哆啦A梦：我正在准备晚餐,铜锣烧是我的最爱。你有没有任何问题需要我帮忙解答?

此文章为搬运
原项目链接

AI Studio
关注
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值