InternLM 实战营笔记-2-趣味demo
部署 InternLM2-Chat-1.8B 模型进行智能对话
下载InterLM2-Chat-1.8B
import os
from modelscope.hub.snapshot_download import snapshot_download
# 创建保存模型目录
os.system(f"mkdir {os.getcwd()}/models")
# save_dir是模型保存到本地的目录
save_dir=f"{os.getcwd()}/models"
snapshot_download("Shanghai_AI_Laboratory/internlm2-chat-1_8b",
cache_dir=save_dir,
revision='v1.1.0')
2024-04-03 11:11:57,276 - modelscope - INFO - PyTorch version 2.1.2+cu121 Found.
2024-04-03 11:11:57,279 - modelscope - INFO - Loading ast index from /root/.cache/modelscope/ast_indexer
2024-04-03 11:11:57,313 - modelscope - INFO - Loading done! Current index file version is 1.9.5, with md5 827645cc680d82eb9a99102bd4beac25 and a total number of 945 components indexed
2024-04-03 11:11:57,780 - modelscope - INFO - Use user-specified model revision: v1.1.0
Downloading: 100%|██████████| 850/850 [00:00<00:00, 3.21MB/s]
Downloading: 100%|██████████| 48.0/48.0 [00:00<00:00, 242kB/s]
Downloading: 100%|██████████| 6.86k/6.86k [00:00<00:00, 499kB/s]
Downloading: 100%|██████████| 132/132 [00:00<00:00, 549kB/s]
Downloading: 100%|█████████▉| 1.85G/1.85G [00:39<00:00, 50.4MB/s]
Downloading: 100%|█████████▉| 1.67G/1.67G [00:23<00:00, 75.2MB/s]
Downloading: 100%|██████████| 13.4k/13.4k [00:00<00:00, 524kB/s]
Downloading: 100%|██████████| 58.6k/58.6k [00:00<00:00, 1.14MB/s]
Downloading: 100%|██████████| 10.4k/10.4k [00:00<00:00, 18.5MB/s]
Downloading: 100%|██████████| 713/713 [00:00<00:00, 3.68MB/s]
Downloading: 100%|██████████| 8.60k/8.60k [00:00<00:00, 8.38MB/s]
Downloading: 100%|██████████| 7.62k/7.62k [00:00<00:00, 14.6MB/s]
Downloading: 100%|██████████| 1.41M/1.41M [00:00<00:00, 7.46MB/s]
Downloading: 100%|██████████| 2.45k/2.45k [00:00<00:00, 10.7MB/s]
'/root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b'
运行client demo
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name_or_path = "/root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm2-chat-1_8b"
#加载模型
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True, device_map='cuda:0')
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map='cuda:0')
model = model.eval()
/root/miniconda3/envs/llava/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
[2024-04-03 11:39:29,295] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Loading checkpoint shards: 100%|██████████| 2/2 [00:02<00:00, 1.13s/it]
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
"""
messages = [(system_prompt, '')]
print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")
while True:
input_text = input("\nUser >>> ")
input_text = input_text.replace(' ', '')
print()
print('User >>> '+input_text)
print()
if input_text == "exit":
break
length = 0
for response, _ in model.stream_chat(tokenizer, input_text, messages):
if response is not None:
print(response[length:], flush=True, end="")
length = len(response)
=============Welcome to InternLM chatbot, type 'exit' to exit.=============
User >>> 请你讲一个300字的关于小红帽和大灰狼的故事
从前，有一个小女孩叫小红帽，她非常喜欢吃糖果，尤其是奶奶给她做的草莓蛋糕。有一天，她决定去森林里拜访奶奶，顺便看看她是否还住在那里。
小红帽出发了，她沿着小路走了很久，直到她看到了一座大房子。她敲了敲门，一位老奶奶开了门，奶奶告诉小红帽她已经很久没有回家了，需要她帮助她找到回家的路。
小红帽非常高兴，她决定跟随奶奶的指示，穿过森林，穿过沼泽，穿过一座座山，直到她终于看到了熟悉的村庄。她告诉奶奶自己已经找到了回家的路，奶奶非常高兴，并邀请她留下来吃晚餐。
晚餐时，奶奶给小红帽带来了许多好吃的，小红帽非常开心。但是，当奶奶告诉小红帽她需要回家的时候，小红帽突然感觉不妙，她看到了一个可怕的大灰狼。她不知道该怎么办，于是她向奶奶求助，但奶奶却告诉她不要害怕，她可以保护自己。
小红帽听从了奶奶的指示，她拿起了一根长矛，向大灰狼发起攻击。最终，小红帽成功地打败了大灰狼，她回到了奶奶身边，并告诉奶奶她经历了可怕的事情。奶奶告诉小红帽，她知道她一定会回来的，她将会保护她，直到她再次回到奶奶身边。
从此以后，小红帽再也没有离开过奶奶，她学会了勇敢和信任，也明白了奶奶的教导和爱。这个故事告诉我们，无论遇到什么困难和危险，我们都需要勇敢地面对，并相信我们会找到回家的路。
User >>> 你能总结一下你讲的故事吗
当然可以。我能够通过自然语言处理技术生成自然语言文本，并能够理解并回答用户的问题。我可以讲述关于天气、地理、历史、文化等各种主题的故事。同时，我也可以帮助用户解决各种语言相关的问题，如翻译、语法纠错等。如果您有任何问题或需要帮助，请随时告诉我。
User >>> exit
修改一下代码，每一次问答前将之前的输入和输出和这一次的输入一起输入给模型，模拟一下带记忆能力的模型
## 手动修改版本，通过增加一个record变量记录每次输入输出，来帮助模型记录之前的内容
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
"""
messages = [(system_prompt, '')]
print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")
record=''
while True:
input_text = input("\nUser >>> ")
input_text = input_text.replace(' ', '')
print()
print('User >>> '+input_text)
print()
if input_text == "exit":
break
record+=input_text #在记录中增加本次用户的输入
length = 0
for response, _ in model.stream_chat(tokenizer, record, messages): #给模型输入之前的记录
if response is not None:
print(response[length:], flush=True, end="")
record+=str(response[length:]) #记录本次的相应
length = len(response)
=============Welcome to InternLM chatbot, type 'exit' to exit.=============
User >>> 请你讲一个100字左右关于小红帽和大灰狼的故事
小红帽和大灰狼是一个经典的童话故事，讲述了一个善良的小女孩小红帽如何被狡猾的大灰狼欺骗的故事。小红帽在森林里遇到了一只狼，她不知道狼的真实意图，误以为它是友善的，于是她跟着狼走进了森林。最终，她被狼吃掉，而狼也逃脱了惩罚。这个故事告诉我们，善良和好奇心有时会导致危险，我们要保持警惕，不要轻易相信陌生人。
User >>> 你还记得你讲了一个啥故事吗
我讲了一个小红帽和大灰狼的故事。小红帽是一个善良的小女孩，她在大灰狼的指引下，误以为大灰狼是友好的，于是跟随他进入了森林。然而，大灰狼是一个狡猾的坏人，他并没有真的帮助小红帽，而是把她带到了一个陷阱里。最终，小红帽被大灰狼吃掉，而大灰狼也逃脱了惩罚。这个故事告诉我们，善良和好奇心有时会导致危险，我们要保持警惕，不要轻易相信陌生人。
User >>> 你认为刚刚讲的故事适合讲给儿童听吗？
作为一个AI语言模型，我无法确定小红帽和大灰狼的故事是否适合给儿童讲，因为故事的情节和主题可能会引起儿童的兴趣和思考。但是，这个故事确实揭示了保持警惕和不要轻易相信陌生人的重要性。同时，这个故事也强调了勇气、智慧和同情心等品质，这些品质对于儿童的发展也非常有益。因此，我建议我们可以结合儿童的兴趣和需求，通过故事的形式，向他们传递这些积极的信息和价值观。
User >>> 小红帽被大灰狼吃掉是不是不太好，你能不能调整一下，并扩充一下相应的故事
好的，小红帽和大灰狼的故事确实需要谨慎对待，因为它揭示了危险和欺骗的现实。在故事中，小红帽的善良和好奇心让她误以为大灰狼是友好的，但最终导致了她被大灰狼欺骗的结局。这个故事提醒我们，不要轻易相信陌生人，要保持警惕，尤其是在陌生环境中。
我可以通过扩充故事来调整小红帽被大灰狼吃掉的情节，例如，小红帽在森林里遇到了一只友好的野兽，但最终还是被大灰狼欺骗了。这样的情节可以更加深入地揭示故事的主题，并引导孩子们思考如何辨别真伪和保护自己。
另外，我也可以添加一些其他的元素来丰富故事，例如，小红帽的家人和朋友都在森林里，他们可能会受到大灰狼的攻击，需要小红帽和其他人的帮助。这样的情节可以增加故事的紧张感和悬念，吸引孩子的注意力。
总之，小红帽和大灰狼的故事需要谨慎处理，但通过适当的扩充和情节设计，可以成为一个富有教育意义和吸引力的故事。
User >>> 请你给出扩充后完整的故事
小红帽和大灰狼的故事
从前，有一个小女孩，名叫小红帽。她非常喜欢森林里的美丽景色，总是喜欢跟着大灰狼一起玩耍。有一天，小红帽在森林里迷路了，她不知道该往哪里走，于是她决定跟着大灰狼。
大灰狼告诉小红帽，他可以带她去一个安全的地方，但是小红帽不知道大灰狼的真实意图，她认为大灰狼是善良的，于是她跟着他走进了森林。
小红帽和大灰狼在森林里遇到了一只狼，狼告诉小红帽，他可以帮助小红帽找到回家的路。小红帽非常高兴，于是她跟着狼走进了森林。
但是，大灰狼并不是真的帮助小红帽，他故意带小红帽到了一个陷阱里，小红帽被陷阱困住了，大灰狼趁机将小红帽吃掉。
小红帽的家人和朋友听到这个消息都感到非常伤心，他们不知道小红帽是如何被大灰狼欺骗的。小红帽的勇敢和聪明才智也让她成为了森林里的英雄，大家都称赞小红帽的机智和善良。
这个故事告诉我们，善良和好奇心有时会导致危险，我们要保持警惕，不要轻易相信陌生人。同时，这个故事也强调了勇气、智慧和同情心等品质，这些品质对于儿童的发展也非常有益。
我们可以通过故事的形式，向孩子们传递这些积极的信息和价值观。小红帽被大灰狼吃掉是不是不太好，我们认为这是一个警示，让我们时刻保持警惕，不要轻易相信陌生人。
User >>> 你认为你的故事是不是有点前后矛盾？
非常抱歉，我的回答有误。小红帽和大灰狼的故事是一个经典的童话故事，讲述了一个善良的小女孩小红帽如何被狡猾的大灰狼欺骗的故事。小红帽在森林里遇到了一只狼，她不知道狼的真实意图，误以为它是友善的，于是她跟着狼走进了森林。最终，她被狼吃掉，而狼也逃脱了惩罚。这个故事告诉我们，善良和好奇心有时会导致危险，我们要保持警惕，不要轻易相信陌生人。
故事中并没有前后矛盾的情况，故事的情节和主题是明确且一致的。小红帽是一个善良的小女孩，她在大灰狼的指引下，误以为大灰狼是友好的，于是跟随他进入了森林。然而，大灰狼是一个狡猾的坏人，他并没有真的帮助小红帽，而是把她带到了一个陷阱里。最终，小红帽被大灰狼吃掉，而大灰狼也逃脱了惩罚。这个故事告诉我们，善良和好奇心有时会导致危险，我们要保持警惕，不要轻易相信陌生人。
User >>> exit
后面看了一下model的chat函数中自带一个history变量，用于记录历史，因此调整了一下
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
"""
messages = [(system_prompt, '')]
print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")
while True:
input_text = input("\nUser >>> ")
input_text = input_text.replace(' ', '')
print()
print('User >>> '+input_text)
print()
if input_text == "exit":
break
length = 0
for response, history in model.stream_chat(tokenizer, input_text, messages): #给模型输入之前的记录
if response is not None:
print(response[length:], flush=True,end='')
length = len(response)
print()
# 一次对话结束后message调整成之前对话的所有内容
messages=history
=============Welcome to InternLM chatbot, type 'exit' to exit.=============
User >>> 请你讲一个200字左右关于小红帽和大灰狼的故事
从前，有一个小女孩叫做小红帽，她非常善良和勇敢。有一天，她决定去奶奶家吃晚餐，但是她不知道，她遇到了一只大灰狼。
小红帽在路上遇到了一只大灰狼，它对她发出了威胁。小红帽害怕极了，但是她决定保持镇静并继续前进。大灰狼开始跟踪她，小红帽的奶奶听到了动静，于是她决定把小红帽藏起来，并告诉小红帽说：“小红帽，你在这里，不要出来！”
小红帽很害怕，但是她还是听从了奶奶的话，她躲在了一个小山洞里，大灰狼终于没有找到她。后来，小红帽的奶奶告诉小红帽，大灰狼是一只贪婪和邪恶的动物，他只想吃小红帽的晚餐。
小红帽通过自己的勇气和智慧，成功地躲过了大灰狼的攻击，并且向奶奶透露了她的经历。这个故事告诉我们，即使在面对危险的时候，我们也不能轻易放弃，我们需要保持镇静并采取正确的行动。
User >>> 请你简要地总结一下你的故事
故事中，小红帽是一个勇敢的小女孩，她前往奶奶家吃晚餐，但是遇到了一只大灰狼。尽管大灰狼发出威胁，小红帽仍然保持镇静，并听从了奶奶的建议，躲在一个小山洞里。最终，小红帽的奶奶告诉小红帽，大灰狼是一只贪婪和邪恶的动物，他只想吃小红帽的晚餐。这个故事告诉我们，即使面对危险，我们也不能轻易放弃，我们需要保持镇静并采取正确的行动。
User >>> 你的故事中小红帽最后躲在了哪里
对不起，我之前的回答有误。实际上，小红帽并没有躲在大山洞里。故事中小红帽躲在了一个小山洞外，奶奶告诉她不要出来，以免被大灰狼发现。
User >>> 小山洞外？
非常抱歉，我的回答有误。实际上，小红帽并没有躲在一个小山洞外。故事中小红帽躲在了一个小山洞里，奶奶告诉她不要出来，以免被大灰狼发现。
User >>> 请你最终确认一下
非常抱歉，我之前的回答有误。故事中小红帽并没有躲在一个小山洞外。故事中小红帽躲在了一个小山洞里，奶奶告诉她不要出来，以免被大灰狼发现。
User >>> exit
总体看一下来，1.8B模型的能力还是偏弱的，逻辑能力还不是很强，但是毕竟模型很小了，要啥自行车啊。
实战：部署实战营优秀作品 Chat-嬛嬛-1.8B 模型
!git clone https://gitee.com/InternLM/Tutorial -b camp2
Cloning into 'Tutorial'...
remote: Enumerating objects: 1034, done.[K
remote: Counting objects: 100% (1034/1034), done.[K
remote: Compressing objects: 100% (468/468), done.[K
remote: Total 1034 (delta 535), reused 1022 (delta 523), pack-reused 0[K
Receiving objects: 100% (1034/1034), 33.35 MiB | 3.84 MiB/s, done.
Resolving deltas: 100% (535/535), done.
下载Chat-嬛嬛-1.8B
import os
#模型下载
from modelscope.hub.snapshot_download import snapshot_download
# 创建保存模型目录
os.system("mkdir -p /root/data1/interlm2/models")
# save_dir是模型保存到本地的目录
save_dir="/root/data1/interlm2/models"
snapshot_download('kmno4zx/huanhuan-chat-internlm2-1_8b',
cache_dir=save_dir)
Downloading: 100%|██████████| 960/960 [00:00<00:00, 3.38MB/s]
Downloading: 100%|██████████| 48.0/48.0 [00:00<00:00, 158kB/s]
Downloading: 100%|██████████| 6.86k/6.86k [00:00<00:00, 320kB/s]
Downloading: 100%|██████████| 132/132 [00:00<00:00, 740kB/s]
Downloading: 100%|█████████▉| 1.85G/1.85G [00:34<00:00, 57.2MB/s]
Downloading: 100%|█████████▉| 1.67G/1.67G [00:26<00:00, 67.0MB/s]
Downloading: 100%|██████████| 13.4k/13.4k [00:00<00:00, 386kB/s]
Downloading: 100%|██████████| 58.6k/58.6k [00:00<00:00, 1.04MB/s]
Downloading: 100%|██████████| 2.98k/2.98k [00:00<00:00, 12.5MB/s]
Downloading: 100%|██████████| 551/551 [00:00<00:00, 3.50MB/s]
Downloading: 100%|██████████| 8.60k/8.60k [00:00<00:00, 18.3MB/s]
Downloading: 100%|██████████| 7.54k/7.54k [00:00<00:00, 25.2MB/s]
Downloading: 100%|██████████| 5.49M/5.49M [00:00<00:00, 15.0MB/s]
Downloading: 100%|██████████| 1.41M/1.41M [00:00<00:00, 4.95MB/s]
Downloading: 100%|██████████| 2.32k/2.32k [00:00<00:00, 10.2MB/s]
Downloading: 100%|██████████| 5.83k/5.83k [00:00<00:00, 594kB/s]
'/root/data1/interlm2/models/kmno4zx/huanhuan-chat-internlm2-1_8b'
这里在助教提供的bajie_chat.py上进行修改得到一个适合甄嬛的chat:huanhuan_chat.py
#由于jupyter好像不能在cell中运行streamlit,所以这里还是调用命令行的方式来运行，然后用vscode的端口转发来在本地访问
!streamlit run /root/data1/interlm2/Tutorial/helloworld/bajie_chat.py --server.address 127.0.0.1 --server.port 6006
实战：使用 Lagent 运行 InternLM2-Chat-7B 模型
!git clone https://gitee.com/internlm/lagent.git
# git clone https://github.com/internlm/lagent.git
import os
os.chdir('/root/data1/interlm2/lagent/')
!git checkout 581d9fb8987a5d9b72bb9ebd37a95efd47d479ac
%pip install -e /root/data1/interlm2/lagent/ # 源码安装
import os
from modelscope.hub.snapshot_download import snapshot_download
# 创建保存模型目录
os.system(f"mkdir -p {os.getcwd()}/models")
# save_dir是模型保存到本地的目录
save_dir=f"{os.getcwd()}/models"
snapshot_download("Shanghai_AI_Laboratory/internlm2-chat-7b",
cache_dir=save_dir,
revision='v1.1.0')
2024-04-03 13:18:07,695 - modelscope - INFO - PyTorch version 2.1.2+cu121 Found.
2024-04-03 13:18:07,697 - modelscope - INFO - Loading ast index from /root/.cache/modelscope/ast_indexer
2024-04-03 13:18:07,731 - modelscope - INFO - Loading done! Current index file version is 1.9.5, with md5 827645cc680d82eb9a99102bd4beac25 and a total number of 945 components indexed
2024-04-03 13:18:08,058 - modelscope - INFO - Use user-specified model revision: v1.1.0
Downloading: 100%|██████████| 851/851 [00:00<00:00, 3.04MB/s]
Downloading: 100%|██████████| 48.0/48.0 [00:00<00:00, 228kB/s]
Downloading: 100%|██████████| 6.86k/6.86k [00:00<00:00, 662kB/s]
Downloading: 100%|██████████| 132/132 [00:00<00:00, 676kB/s]
Downloading: 100%|█████████▉| 1.82G/1.82G [00:34<00:00, 57.3MB/s]
Downloading: 100%|█████████▉| 1.81G/1.81G [00:30<00:00, 64.4MB/s]
Downloading: 100%|█████████▉| 1.84G/1.84G [00:22<00:00, 89.1MB/s]
Downloading: 100%|█████████▉| 1.81G/1.81G [00:34<00:00, 57.2MB/s]
Downloading: 100%|█████████▉| 1.84G/1.84G [00:22<00:00, 87.3MB/s]
Downloading: 100%|█████████▉| 1.81G/1.81G [00:28<00:00, 68.9MB/s]
Downloading: 100%|█████████▉| 1.84G/1.84G [00:25<00:00, 76.8MB/s]
Downloading: 100%|█████████▉| 1.63G/1.63G [00:24<00:00, 72.5MB/s]
Downloading: 100%|██████████| 17.8k/17.8k [00:00<00:00, 695kB/s]
Downloading: 100%|██████████| 58.6k/58.6k [00:00<00:00, 818kB/s]
Downloading: 100%|██████████| 11.2k/11.2k [00:00<00:00, 25.5MB/s]
Downloading: 100%|██████████| 713/713 [00:00<00:00, 2.09MB/s]
Downloading: 100%|██████████| 8.60k/8.60k [00:00<00:00, 23.1MB/s]
Downloading: 100%|██████████| 7.62k/7.62k [00:00<00:00, 11.7MB/s]
Downloading: 100%|██████████| 1.41M/1.41M [00:00<00:00, 6.19MB/s]
Downloading: 100%|██████████| 2.45k/2.45k [00:00<00:00, 13.9MB/s]
'/root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm2-chat-7b'
import os
os.chdir('/root/data1/interlm2/lagent')
#在执行时出现了OSError: [Errno 28] inotify watch limit reached，查了一下资料，可以用如下方案解决
#--server.fileWatcherType none
!streamlit run /root/data1/interlm2/lagent/examples/internlm2_agent_web_demo_hf.py --server.address 127.0.0.1 --server.port 6006
这里实验了一下提供数据分析和arixv search两个功能，arixv search目前还是存在一些问题，得到很多信息是不准确的。
实战：实践部署 浦语·灵笔2 模型 && hugging face 模型下载
%pip install install timm==0.4.12 sentencepiece==0.1.99 markdown2==2.4.10 xlsxwriter==3.1.2 gradio==4.13.0 modelscope==1.9.5\
!cd /root/data1/interlm2/
!git clone https://gitee.com/internlm/InternLM-XComposer.git
# !git clone https://github.com/internlm/InternLM-XComposer.git
os.chdir('/root/data1/interlm2/InternLM-XComposer')
!git checkout f31220eddca2cf6246ee2ddf8e375a40457ff626
hugging face 模型下载
## 因为作业里有一项要使用huggingface来下载模型，前面都是modelscope，所以这里补一下
import os
os.environ['HF_HUB_ENABLE_HF_TRANSFER']='1' #加速
os.environ['http_proxy']='http://127.0.0.1:7890'
os.environ['https_proxy']='http://127.0.0.1:7890'
# 下载模型
os.system('huggingface-cli download --resume-download internlm/internlm-xcomposer2-7b --local-dir /root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm-xcomposer2-7b')
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/configuration_internlm_xcomposer2.py to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/86359c0b50756ebf53f72fa26c59d72d7718b6f6.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/generation_config.json to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/c747feb87d9b0c4881dbaa629f000ca52b3e9542.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/logo.png to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/5170acf63242e7621b056f07c0525331383fed3b.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/logo_en.png to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/da14e880b23c63eb41e8f5fb2d109c66440cc04f.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/modeling_internlm2.py to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/de0dff4c6b2607f49471e2fd97cbb30735455ec7.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/modeling_internlm_xcomposer2.py to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/bcf7c3ab9f091c4c03b1220a111a18db1c71148f.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/panda.jpg to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/1553d055d46b3ea7e7a8f1a7625a5fd843b446fc.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/pytorch_model.bin to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/e56fd44623534089dbd0393ec54eeeb3d9bdea43a804e8b32e87bb66f8b6e634.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/special_tokens_map.json to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/9bfed7513d3b1b65445af10c4571256f4a19b290.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/tokenization_internlm_xcomposer2.py to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/333cf32d9fd108dac0e755a8c0a08dc86050e509.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/tokenizer.model to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/f868398fc4e05ee1e8aeba95ddf18ddcc45b8bce55d5093bead5bbf80429b48b.incomplete
downloading https://huggingface.co/internlm/internlm-xcomposer2-7b/resolve/d7ab428de9dc92ea1df7763bf4723ac76c181da1/tokenizer_config.json to /root/.cache/huggingface/hub/models--internlm--internlm-xcomposer2-7b/blobs/75f544b806824228b69698ebd7ab64a5b366a46c.incomplete
/root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm-xcomposer2-7b
0
#modelsope下载更快一点，所以我们还是用modelscope来下载第二个模型
import os
from modelscope.hub.snapshot_download import snapshot_download
# 创建保存模型目录
os.system(f"mkdir -p {os.getcwd()}/models")
# save_dir是模型保存到本地的目录
save_dir=f"{os.getcwd()}/models"
snapshot_download("Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b",
cache_dir=save_dir,revision='v1.0.0' )
2024-04-03 15:29:24,150 - modelscope - INFO - Use user-specified model revision: v1.0.0
Downloading: 100%|██████████| 7.18k/7.18k [00:00<00:00, 2.93MB/s]
Downloading: 100%|██████████| 977/977 [00:00<00:00, 1.41MB/s]
Downloading: 100%|██████████| 48.0/48.0 [00:00<00:00, 211kB/s]
Downloading: 100%|██████████| 7.14k/7.14k [00:00<00:00, 719kB/s]
Downloading: 100%|██████████| 176/176 [00:00<00:00, 1.00MB/s]
Downloading: 100%|██████████| 43.6k/43.6k [00:00<00:00, 590kB/s]
Downloading: 100%|██████████| 101k/101k [00:00<00:00, 728kB/s]
Downloading: 100%|██████████| 45.1k/45.1k [00:00<00:00, 501kB/s]
Downloading: 100%|██████████| 38.5k/38.5k [00:00<00:00, 1.09MB/s]
Downloading: 100%|██████████| 24.5k/24.5k [00:00<00:00, 886kB/s]
Downloading: 100%|█████████▉| 9.30G/9.30G [02:14<00:00, 74.4MB/s]
Downloading: 100%|█████████▉| 6.85G/6.85G [02:11<00:00, 56.0MB/s]
Downloading: 100%|██████████| 87.6k/87.6k [00:00<00:00, 602kB/s]
Downloading: 100%|██████████| 3.07k/3.07k [00:00<00:00, 5.42MB/s]
Downloading: 100%|██████████| 95.0/95.0 [00:00<00:00, 215kB/s]
Downloading: 100%|██████████| 8.96k/8.96k [00:00<00:00, 12.0MB/s]
Downloading: 100%|██████████| 1.41M/1.41M [00:02<00:00, 546kB/s]
Downloading: 100%|██████████| 401/401 [00:00<00:00, 1.59MB/s]
Downloading: 100%|██████████| 23.6k/23.6k [00:00<00:00, 668kB/s]
'/root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b'
图像写作实战
import os
#这里还是会缺clip-vit-large-patch14-336，
os.environ['http_proxy']='http://127.0.0.1:7890'
os.environ['https_proxy']='http://127.0.0.1:7890'
os.chdir('/root/data1/interlm2/InternLM-XComposer')
#实验发现这里定义--num_gpus 2没有起到作用，还是在单卡上部署的
#InternLM-XComposer/examples/gradio_demo_composition.py中Demo_UI初始函数只是简单传了个num_gpus参数，实际上这个参数压根没有使用
#class Demo_UI:
# def __init__(self, code_path, num_gpus=1):
!python /root/data1/interlm2/InternLM-XComposer/examples/gradio_demo_composition.py --code_path /root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm-xcomposer2-7b --private --num_gpus 2 --port 6006
这里InternLM-XComposer的能力更多是根据生成的文本和搜索到图像进行匹配，然后将文本和图像进行合适的组合生成一篇完整的文章
这里图片大都是从网络图片的无论是质量还是匹配程度还是存在着一些问题，个人感觉可以考虑将文本给文生图模型生成相应插图，可能质量和匹配程度都更好
这里不太清楚为啥还需要clip模型，后续需要看一下技术报告还代码，难道是用clip来判断找到的图片和文本之间的匹配程度？
图片理解实战
import os
#这里还是会缺clip-vit-large-patch14-336，
# os.environ['http_proxy']='http://127.0.0.1:7890'
# os.environ['https_proxy']='http://127.0.0.1:7890'
os.chdir('/root/data1/interlm2/InternLM-XComposer')
#实验发现这里定义--num_gpus 2没有起到作用，还是在单卡上部署的
#InternLM-XComposer/examples/gradio_demo_composition.py中Demo_UI初始函数只是简单传了个num_gpus参数，实际上这个参数压根没有使用
#class Demo_UI:
# def __init__(self, code_path, num_gpus=1):
!python /root/data1/interlm2/InternLM-XComposer/examples/gradio_demo_chat.py --code_path /root/data1/interlm2/models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b --private --num_gpus 2 --port 6006
这里我选择一张服装图像让其进行标注，同时我这里放一下之前用llava-7B和GPT4-V的标注的结果，其实之前还做个通义千问的结果，但是效果一般就没保存下来。
internlm-xcomposer2-vl-7b:
prompt1:(与CPT4-V保持一致)
现在我有一些服装图片,请你从以下几个方面来对图中服装信息进行详细描述.形状与款式:描述衣服的基本款式,比如是T恤、衬衫、裙子还是裤子。提及衣服的剪裁特点，例如紧身、宽松、短款、长款等。颜色与图案：描述衣服的主要颜色和任何特殊图案，如何种类型条纹、格子、印花等。材质与纹理：描述衣服的材质（棉、丝、合成纤维等）以及任何特殊的纹理特征，如光滑、粗糙、有光泽等。特殊细节：如果有，描述任何特殊的设计元素，如蕾丝边、绣花、钮扣、口袋等，如果能确定位置可以描述位置。请用英文进行描述，并且尽可能保证描述准确性，同时对于一些难以确定信息可以不描述。描述风格请类似于[black women's long sleeve t-shirt, black round neck long sleeve t-shirt, black long-sleeved jersey top,...],。返回格式按照{图片序号:[服装描述]，图片序号:[服装描述]，...}的格式返回,尽量讲述事实，参考返回形式：1:[Women's short sleeve t-shirt with a tie-dye pattern in blue and red tones on a white base. It has a round neck and appears to be a relaxed fit];2:[Women's short sleeve ribbed knit top with multicolor horizontal stripes. It features a high neck and appears to be fitted.];3:[Black women's one-piece swimsuit with thin adjustable straps and a V-neck. It features ruched details for texture and appears to have a fitted silhouette];4:[Red crew neck t-shirt with short sleeves featuring a graphic print of two cherubs on the front. The material and fit are not clearly visible];5:[Women's short sleeve blouse with a dark green base and a pink floral pattern. It has a round neck and a loose fit];6:[Women's spaghetti strap camisole top with a white base and a blue and green floral print. It appears to have a lightweight fabric and a relaxed fit];
result1:
1: [A green blouse with a floral pattern. The blouse has a long-sleeve, round-neck design and features black sleeves that are slightly oversized compared to the rest of the garment. The material seems to be lightweight and flows smoothly across the fabric's surface.]
2: [The image contains two shirts with different designs. The first one is a blue and red tie-dye shirt made from cotton, featuring a relaxed-fitting round neck style. The second shirt appears to be a black, long-sleeved, jersey top with short sleeves, a round neck, and an unknown pattern. Both shirts have similar materials and textures, smooth and well-kept.]
3: [A blue top with a loose fit. It is designed for men, with a V-neck and thin straps running horizontally across the bust area. A gold zipper runs along the front of the dress, adding a touch of elegance to its appearance. Unfortunately, some details like the exact color and texture of the material aren't clearly visible in the image.]
4: [The picture displays a pink skirt which appears to be loose and flowy, possibly a mini or midi-length. Additionally, there is a belt with buckles attached to it. The presence of these items suggests that the outfit may be geared towards sportswear or athletic activities.
prompt2(与llava1.6-7b一致)
Give the caption of this image 'Women's long-sleeved blouse in a dark teal color with yellow floral prints and some black foliage accents. It features a keyhole neckline with a button closure at the back, puffed sleeves, and a loosely fitted silhouette'.
Please describe the cloth in this image in a very detailed manner. Pelease start with 'a model wears' and don not describe the background of this image.
result2:
A model wears a loose-fitting, long-sleeved blouse that falls gracefully from shoulder to the top of her hips. The fabric is a rich dark teal color, and the garment features yellow floral prints accompanied by some black foliage accents, lending it an elegant and artistic feel. The neckline displays a keyhole design with a single button at the back for closure. The sleeves are puffed, adding a touch of vintage charm to the blouse's overall aesthetic.
gpt4-V:
prompt:
现在我有一些服装图片,请你从以下几个方面来对图中服装信息进行详细描述.形状与款式:描述衣服的基本款式,比如是T恤、衬衫、裙子还是裤子。提及衣服的剪裁特点，例如紧身、宽松、短款、长款等。颜色与图案：描述衣服的主要颜色和任何特殊图案，如何种类型条纹、格子、印花等。材质与纹理：描述衣服的材质（棉、丝、合成纤维等）以及任何特殊的纹理特征，如光滑、粗糙、有光泽等。特殊细节：如果有，描述任何特殊的设计元素，如蕾丝边、绣花、钮扣、口袋等，如果能确定位置可以描述位置。请用英文进行描述，并且尽可能保证描述准确性，同时对于一些难以确定信息可以不描述。描述风格请类似于[black women's long sleeve t-shirt, black round neck long sleeve t-shirt, black long-sleeved jersey top,...],。返回格式按照{图片序号:[服装描述]，图片序号:[服装描述]，...}的格式返回,尽量讲述事实，参考返回形式：1:[Women's short sleeve t-shirt with a tie-dye pattern in blue and red tones on a white base. It has a round neck and appears to be a relaxed fit];2:[Women's short sleeve ribbed knit top with multicolor horizontal stripes. It features a high neck and appears to be fitted.];3:[Black women's one-piece swimsuit with thin adjustable straps and a V-neck. It features ruched details for texture and appears to have a fitted silhouette];4:[Red crew neck t-shirt with short sleeves featuring a graphic print of two cherubs on the front. The material and fit are not clearly visible];5:[Women's short sleeve blouse with a dark green base and a pink floral pattern. It has a round neck and a loose fit];6:[Women's spaghetti strap camisole top with a white base and a blue and green floral print. It appears to have a lightweight fabric and a relaxed fit];
result:
Women's long-sleeved blouse in a dark teal color with yellow floral prints and some black foliage accents. It features a keyhole neckline with a button closure at the back, puffed sleeves, and a loosely fitted silhouette
llava-1.6-7b 这里llava1.6是用了gpt4v生成结果作为prompt
prompt:
Give the caption of this image 'Women's long-sleeved blouse in a dark teal color with yellow floral prints and some black foliage accents. It features a keyhole neckline with a button closure at the back, puffed sleeves, and a loosely fitted silhouette'.
Please describe the cloth in this image in a very detailed manner. Pelease start with 'a model wears' and don not describe the background of this image.
result:
a model wears a women's long-sleeved blouse in a dark teal color with yellow floral prints and some black foliage accents. The blouse features a keyhole neckline with a button closure at the back, puffed sleeves, and a loosely fitted silhouette. The fabric appears to be a lightweight, possibly cotton or cotton blend, with a slight sheen that suggests a smooth texture. The yellow floral prints are scattered throughout the fabric, with some larger blooms and green leaves interspersed, creating a natural, botanical pattern. The black foliage accents are strategically placed around the blouse, adding a contrasting element to the design. The overall style of the blouse is elegant and feminine, with a touch of whimsy from the floral prints.
总体结果来GPT-4V还要优秀点，能够更好理解用于指令的意图,整体来效果和llava类似
390
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
