基础作业(结营必做)
完成以下任务,并将实现过程记录截图:
- 配置 LMDeploy 运行环境
- 以命令行方式与 InternLM2-Chat-1.8B 模型对话
创建conda环境
studio-conda -t lmdeploy -o pytorch-2.1.2
安装LMDeploy
conda activate lmdeploy
pip install lmdeploy[all]==0.3.0
LMDeploy模型对话(chat)
下载模型
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b /root/
# cp -r /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b /root/
使用Transformer库运行模型
touch /root/pipeline_transformer.py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("/root/internlm2-chat-1_8b", trust_remote_code=True)
# Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and cause OOM Error.
model = AutoModelForCausalLM.from_pretrained("/root/internlm2-chat-1_8b", torch_dtype=torch.float16, trust_remote_code=True).cuda()
model = model.eval()
inp = "hello"
print("[INPUT]", inp)
response, history = model.chat(tokenizer, inp, history=[])
print("[OUTPUT]", response)
inp = "please provide three suggestions about time management"
print("[INPUT]", inp)
response, history = model.chat(tokenizer, inp, history=history)
print("[OUTPUT]", response)
python /root/pipeline_transformer.py
使用LMDeploy与模型对话
conda activate lmdeploy
lmdeploy chat /root/internlm2-chat-1_8b
刚开始的时候会有bug,响应不及时。生成第一个回复后,后续对话就比较流畅。