L1-8G 显存玩转书生大模型 Demo

任务链接：Tutorial/docs/L1/Demo/task.md at camp3 · InternLM/Tutorial · GitHub

 

目录

1. 基础任务

2. 进阶任务

2.1 使用Transformer库运行模型

2.2 使用LMDeploy运行模型

2.1.1 internlm-xcomposer2-vl-1_8b

2.2.2 InternVL2-2B

---

1. 基础任务

使用 Cli Demo 完成 InternLM2-Chat-1.8B 模型的部署，并生成 300 字小故事

# 配置环境
studio-conda -o internlm-base -t demo


# 激活环境
conda activate demo


# 下载包
pip install huggingface-hub==0.17.3
pip install transformers==4.34 
pip install psutil==5.9.8
pip install accelerate==0.24.1
pip install streamlit==1.32.2 
pip install matplotlib==3.8.3 
pip install modelscope==1.9.5
pip install sentencepiece==0.1.99


# 创建所需文件，download_mini.py用于下载模型，cli_demo.py用于创建demo
mkdir -p /root/demo
touch /root/demo/cli_demo.py
touch /root/demo/download_mini.py
cd /root/demo

cli_demo.py

# cli_demo.py
import os
from modelscope.hub.snapshot_download import snapshot_download

# 创建保存模型目录
os.system("mkdir /root/models")

# save_dir是模型保存到本地的目录
save_dir="/root/models"

snapshot_download("Shanghai_AI_Laboratory/internlm2-chat-1_8b", 
                  cache_dir=save_dir, 
                  revision='v1.1.0')

download_mini.py

# download_mini.py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM


model_name_or_path = "/root/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b"

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True, device_map='cuda:0')
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map='cuda:0')
model = model.eval()

system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
"""

messages = [(system_prompt, '')]

print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")

while True:
    input_text = input("\nUser  >>> ")
    input_text = input_text.replace(' ', '')
    if input_text == "exit":
        break

    length = 0
    for response, _ in model.stream_chat(tokenizer, input_text, messages):
        if response is not None:
            print(response[length:], flush=True, end="")
            length = len(response)

运行结果

1. 下载模型

2. 输入示例内容，生成文字

3. 退出

2. 进阶任务

使用 LMDeploy 完成 InternLM-XComposer2-VL-1.8B 的部署

本小节使用的图片来自

[https://huggingface.co/internlm/internlm-xcomposer2-vl-7b/blob/main/image1.webp]

2.1 使用Transformer库运行模型

# 拷贝模型到root目录
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-1_8b /root/

xcomposer2-vl-transformer.py

# xcomposer2-vl-transformer.py
import torch
import time
from transformers import AutoModel, AutoTokenizer

end = time.time()
torch.set_grad_enabled(False)

model = AutoModel.from_pretrained('/root/internlm-xcomposer2-vl-1_8b', torch_dtype=torch.bfloat16, trust_remote_code=True).cuda().eval().half()
tokenizer = AutoTokenizer.from_pretrained('/root/internlm-xcomposer2-vl-1_8b', trust_remote_code=True)

query = '<ImageHere>Please describe this image in detail.'
image = './image1.webp'
with torch.cuda.amp.autocast():
  response, _ = model.chat(tokenizer, query=query, image=image, history=[], do_sample=False)
print(response)
print('Total Time: {0:.0f} s'.format(time.time() - end))

结果:

2.2 使用LMDeploy运行模型

# 安装依赖, 创建的环境名称为lmdeploy
conda activate lmdeploy
pip install lmdeploy[all]==0.5.1
pip install timm==1.0.7


# 使用 LMDeploy 完成 InternLM-XComposer2-VL-1.8B 的部署
lmdeploy serve gradio /share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-1_8b --cache-max-entry-count 0.1


# 使用 LMDeploy 完成 InternVL2-2B 的部署
lmdeploy serve gradio /share/new_models/OpenGVLab/InternVL2-2B --cache-max-entry-count 0.1

运行结果

2.1.1 internlm-xcomposer2-vl-1_8b

（不知道为啥是英文）

改一下Prompt

2024-07-30 17:33:08,816 - lmdeploy - INFO - ImageEncoder received 1 images, left 1 images.
2024-07-30 17:33:08,816 - lmdeploy - INFO - ImageEncoder process 1 images, left 0 images.
2024-07-30 17:33:17,233 - lmdeploy - INFO - ImageEncoder forward 1 images, cost 8.417s
2024-07-30 17:33:17,233 - lmdeploy - INFO - ImageEncoder done 1 images, left 0 images.
2024-07-30 17:33:17,235 - lmdeploy - INFO - preprocess cost 8.420s

运行时界面上会显示运行的时间，处理这一张图片并且生成描述的时间大概是8.4s，相比于用hf的transformer部署（867s）确实快了很多。

2.2.2 InternVL2-2B

命令行输出：

2024-07-30 17:25:08,911 - lmdeploy - INFO - ImageEncoder forward 1 images, cost 57.038s
2024-07-30 17:25:08,912 - lmdeploy - INFO - ImageEncoder done 1 images, left 0 images.
2024-07-30 17:25:08,913 - lmdeploy - INFO - preprocess cost 57.040s