8G 显存玩转书生大模型 Demo

环境部署

开发机配置

镜像：Cuda12.2-conda
资源配置：10% A100*1

创建conda环境

studio-conda -t lmdeploy -o pytorch-2.1.2

进入这个状态以后要等待很久很久，可以去干点别的再回来看

安装结束

切换环境

conda activate lmdeploy

安装lmdeploy

pip install lmdeploy[all]==0.3.0 

安装结束

模型下载

在internStudio中，实际上已经存在模型文件，仅需要建立软链接

ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b /root/

查看软链

ls -al | grep internlm2 

使用Transformer库运行模型

Transformer库是Huggingface社区推出的用于运行HF模型的官方库。

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("/root/internlm2-chat-1_8b", trust_remote_code=True)

# Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and cause OOM Error.
model = AutoModelForCausalLM.from_pretrained("/root/internlm2-chat-1_8b", torch_dtype=torch.float16, trust_remote_code=True).cuda()
model = model.eval()

inp = "hello"
print("[INPUT]", inp)
response, history = model.chat(tokenizer, inp, history=[])
print("[OUTPUT]", response)

inp = "please provide three suggestions about time management"
print("[INPUT]", inp)  
response, history = model.chat(tokenizer, inp, history=history)


print("[OUTPUT]", response)  

LMDeploy部署模型

lmdeploy chat /root/internlm2-chat-1_8b