InternLM2-Chat-1.8B 模型的部署
选择镜像 使用 Cuda11.7-conda 镜像
环境配置（约一个小时）
studio-conda -o internlm-base -t demo激活环境
conda activate demo安装包
pip install huggingface-hub==0.17.3
pip install transformers==4.34
pip install psutil==5.9.8
pip install accelerate==0.24.1
pip install streamlit==1.32.2
pip install matplotlib==3.8.3
pip install modelscope==1.9.5
pip install sentencepiece==0.1.99
下载 InternLM2-Chat-1.8B 模型
mkdir -p /root/demo
touch /root/demo/cli_demo.py
touch /root/demo/download_mini.py
cd /root/demo其中cli_demo.py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name_or_path = "/root/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True, device_map='cuda:0')
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map='cuda:0')
model = model.eval()
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
"""
messages = [(system_prompt, '')]
print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")
while True:
input_text = input("\nUser >>> ")
input_text = input_text.replace(' ', '')
if input_text == "exit":
break
length = 0
for response, _ in model.stream_chat(tokenizer, input_text, messages):
if response is not None:
print(response[length:], flush=True, end="")
length = len(response)
模型下载download_mini.py
import os
from modelscope.hub.snapshot_download import snapshot_download
# 创建保存模型目录
os.system("mkdir /root/models")
# save_dir是模型保存到本地的目录
save_dir="/root/models"
snapshot_download("Shanghai_AI_Laboratory/internlm2-chat-1_8b",
cache_dir=save_dir,
revision='v1.1.0')
待完成后
运行大模型
conda activate demo
python /root/demo/cli_demo.py示范输出
在一个小镇上，住着一位名叫艾米的年轻女孩。她是一个热爱冒险的女孩，总是渴望去探索未知的世界。这一天，艾米决定独自一人去远足，寻找隐藏在山间的宝藏。
她穿过了茂密的森林，穿过了崎岖的山路，最终来到了一座古老的山洞前。艾米小心翼翼地爬了进去，发现这里有一座神秘的石门。她决定尝试打开它。
当她打开石门时，她发现了一扇通往另一个世界的门。这个门通向一个神秘的王国，里面充满了各种奇妙的事物。艾米感到兴奋不已，她决定深入探索这个王国。
在旅途中，艾米遇到了许多有趣的人物。有勇敢的战士，有聪明的小矮人，还有善良的仙女。他们一起经历了许多冒险，经历了许多困难，但他们总是团结一致，共同克服了所有的困难。
最终，艾米到达了王国的中心。在那里，她发现了一座巨大的宝藏，里面装满了金子和珠宝。艾米决定将这个宝藏带回家，带给家人和朋友。
艾米回到家后，她的家人和朋友都非常高兴。他们庆祝她的冒险，并为她分享了所有的宝藏。艾米感到非常满足，因为她不仅找到了宝藏，还结交了许多有趣的人，并且学会了勇敢和团结。
从此以后，艾米继续探索世界，但她永远不会忘记那个神秘的山洞和那扇通向另一个世界的门。
InternLM-XComposer2-VL-1.8B 的部署（Win10）
运行配置：13900+4090
下载模型权重
#模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('Shanghai_AI_Laboratory/internlm-xcomposer2-vl-1_8b')
#模型下载
from modelscope import snapshot_download
model_dir = snapshot_download('AI-ModelScope/clip-vit-large-patch14-336')将模型权重移动到项目中
修改代码internlm-xcomposer2-vl-1_8b\build_mlp.py
运行代码，需修改图像路径
import torch
from modelscope import AutoModel,AutoTokenizer, AutoModelForCausalLM
torch.set_grad_enabled(False)
# init model and tokenizer
model = AutoModel.from_pretrained('./weight/internlm-xcomposer2-vl-1_8b/', trust_remote_code=True).cuda().eval()
tokenizer = AutoTokenizer.from_pretrained('./weight/internlm-xcomposer2-vl-1_8b/', trust_remote_code=True)
query = '<ImageHere>Please describe this image in detail.'
image = './weight/InternVL2-2B/examples/image1.jpg'
with torch.cuda.amp.autocast():
response, _ = model.chat(tokenizer, query=query, image=image, history=[], do_sample=False)
print(response)
示范图像
输出
The image features a red panda, a majestic creature known for its distinctive red fur and white underbelly. The panda is comfortably perched on a wooden platform, its head resting on the platform as if it's enjoying a moment of relaxation. The background of the image is filled with lush green foliage, providing a natural and serene environment for the panda. The panda's gaze is directed towards the camera, capturing a sense of curiosity and connection with the viewer. The overall image paints a peaceful picture of the red panda in its natural habitat.
InternVL2-2B部署
权重下载 前往InternVL2-2B · 模型库 (modelscope.cn)自行申请下载链接
模型运行
import numpy as np
import torch
import torchvision.transforms as T
from PIL import Image
from torchvision.transforms.functional import InterpolationMode
from transformers import AutoModel, AutoTokenizer
IMAGENET_MEAN = (0.485, 0.456, 0.406)
IMAGENET_STD = (0.229, 0.224, 0.225)
def build_transform(input_size):
MEAN, STD = IMAGENET_MEAN, IMAGENET_STD
transform = T.Compose([
T.Lambda(lambda img: img.convert('RGB') if img.mode != 'RGB' else img),
T.Resize((input_size, input_size), interpolation=InterpolationMode.BICUBIC),
T.ToTensor(),
T.Normalize(mean=MEAN, std=STD)
])
return transform
def find_closest_aspect_ratio(aspect_ratio, target_ratios, width, height, image_size):
best_ratio_diff = float('inf')
best_ratio = (1, 1)
area = width * height
for ratio in target_ratios:
target_aspect_ratio = ratio[0] / ratio[1]
ratio_diff = abs(aspect_ratio - target_aspect_ratio)
if ratio_diff < best_ratio_diff:
best_ratio_diff = ratio_diff
best_ratio = ratio
elif ratio_diff == best_ratio_diff:
if area > 0.5 * image_size * image_size * ratio[0] * ratio[1]:
best_ratio = ratio
return best_ratio
def dynamic_preprocess(image, min_num=1, max_num=6, image_size=448, use_thumbnail=False):
orig_width, orig_height = image.size
aspect_ratio = orig_width / orig_height
# calculate the existing image aspect ratio
target_ratios = set(
(i, j) for n in range(min_num, max_num + 1) for i in range(1, n + 1) for j in range(1, n + 1) if
i * j <= max_num and i * j >= min_num)
target_ratios = sorted(target_ratios, key=lambda x: x[0] * x[1])
# find the closest aspect ratio to the target
target_aspect_ratio = find_closest_aspect_ratio(
aspect_ratio, target_ratios, orig_width, orig_height, image_size)
# calculate the target width and height
target_width = image_size * target_aspect_ratio[0]
target_height = image_size * target_aspect_ratio[1]
blocks = target_aspect_ratio[0] * target_aspect_ratio[1]
# resize the image
resized_img = image.resize((target_width, target_height))
processed_images = []
for i in range(blocks):
box = (
(i % (target_width // image_size)) * image_size,
(i // (target_width // image_size)) * image_size,
((i % (target_width // image_size)) + 1) * image_size,
((i // (target_width // image_size)) + 1) * image_size
)
# split the image
split_img = resized_img.crop(box)
processed_images.append(split_img)
assert len(processed_images) == blocks
if use_thumbnail and len(processed_images) != 1:
thumbnail_img = image.resize((image_size, image_size))
processed_images.append(thumbnail_img)
return processed_images
def load_image(image_file, input_size=448, max_num=6):
image = Image.open(image_file).convert('RGB')
transform = build_transform(input_size=input_size)
images = dynamic_preprocess(image, image_size=input_size, use_thumbnail=True, max_num=max_num)
pixel_values = [transform(image) for image in images]
pixel_values = torch.stack(pixel_values)
return pixel_values
path = './weight/InternVL2-2B/'
model = AutoModel.from_pretrained(
path,
torch_dtype=torch.bfloat16,
low_cpu_mem_usage=True,
trust_remote_code=True).eval().cuda()
tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
# set the max number of tiles in `max_num`
pixel_values = load_image('./weight/InternVL2-2B/examples/image1.jpg', max_num=6).to(torch.bfloat16).cuda()
generation_config = dict(
num_beams=1,
max_new_tokens=1024,
do_sample=False,
)
# pure-text conversation (纯文本对话)
question = 'Hello, who are you?'
response, history = model.chat(tokenizer, None, question, generation_config, history=None, return_history=True)
print(f'User: {question}')
print(f'Assistant: {response}')
question = 'Can you tell me a story?'
response, history = model.chat(tokenizer, None, question, generation_config, history=history, return_history=True)
print(f'User: {question}')
print(f'Assistant: {response}')
# single-image single-round conversation (单图单轮对话)
question = '<image>\nPlease describe the image shortly.'
response = model.chat(tokenizer, pixel_values, question, generation_config)
print(f'User: {question}')
print(f'Assistant: {response}')结果
User: Hello, who are you?
Assistant: I am an AI assistant whose name is InternVL, developed jointly by Shanghai AI Lab and SenseTime.
User: Can you tell me a story?
Assistant: Of course! Here's a short story:Once upon a time, in a far-off land, there was a young girl named Lily. She lived in a small village surrounded by lush green fields and tall trees. One day, while she was playing in the fields, she stumbled upon a beautiful butterfly. The butterfly was fluttering around her, and she couldn't help but admire it.
As she watched the butterfly, she noticed that it was struggling to fly. She asked the butterfly what was wrong, and the butterfly replied, "I'm not flying well. I'm afraid of the wind."
Lily felt sorry for the butterfly and decided to help it. She picked up a handful of flowers and threw them at the butterfly, hoping to make it fly better. The butterfly fluttered around the flowers, and soon it was flying higher and higher.
From that day on, Lily made it her mission to help all the creatures in her village. She would pick flowers and throw them at birds, and she would feed the animals in the village. And with each day that passed, the village became a happier and more peaceful place.
The butterfly, grateful for Lily's help, flew around her and thanked her. And from that day on, Lily knew that she had made a difference in the world.
User: <image>
Please describe the image shortly.
Assistant: The image shows a red panda cub peeking out from a wooden structure, possibly a nest or a shelter. The cub has a striking reddish-brown coat with white markings on its face, ears, and paws. The background is blurred but appears to be a natural environment with green foliage.
371
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
