书生浦语第二期实战营系列—综述加技术报告研读
书生浦语第二期实战营系列—Tutorial1:demo体验
书生浦语第二期实战营系列—Tutorial2:RAG
书生浦语第二期实战营系列—Tutorial3:Xtuner微调
书生浦语第二期实战营系列—Tutorial4:Lmdeploy量化部署
书生浦语第二期实战营系列—Tutorial5:Agent
书生浦语第二期实战营系列—Tutorial6:OpenCompass
书生浦语第二期实战营学习笔记(第二节课)
1.环境安装
1.1换源
换好源,后面安装速度比较快
1.1.1pip换源
pip install --upgrade pip
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
1.1.2conda换源
cat <<'EOF' > ~/.condarc
channels:
- defaults
show_channel_urls: true
default_channels:
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
pytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
deepmodeling: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/
EOF
1.2 依赖安装
用conda和pip安装,熟悉下流程
conda create -n internlm python==3.10
conda activate internlm
pip install torch==2.0.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install transformers==4.34
pip install psutil==5.9.8
pip install accelerate==0.24.1
pip install streamlit==1.32.2
pip install matplotlib==3.8.3
pip install modelscope==1.9.5
pip install sentencepiece==0.1.99
2. 基础作业
2.1 模型下载
2.1.1 huggingface下载
pip install -U huggingface-hub
huggingface 经常连不上,terminal中设置下,也可以写在~/.bashrc,从huggingface的镜像中下载
export HF_ENDPOINT="https://hf-mirror.com"
把上面的下载命令写入python脚本中,下载internlm2-chat-1_8b模型
2.1.2 modelscope下载
用modelscope下载, 后面可以成功运行
2.2 代码下载和运行
2.2.1 代码下载
git clone https://gitee.com/InternLM/InternLM.git
在InternLM目录下,新建一个cli_demo.py,内容如下,xxx为下载模型的路径:
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name_or_path = "xxx/Shanghai_AI_Laboratory/internlm2-chat-1_8b"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map='auto')
model = model.eval()
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
"""
messages = [(system_prompt, '')]
print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")
while True:
input_text = input("User >>> ")
input_text = input_text.replace(' ', '')
if input_text == "exit":
break
length = 0
for response, _ in model.stream_chat(tokenizer, input_text, messages):
if response is not None:
print(response[length:], flush=True, end="")
length = len(response)
2.2.2 代码运行
python cli_demo.py
截图为证
3. 进阶作业
3.1 huggingface下载
用huggingface_hub python 包,下载 InternLM2-Chat-7B,截图为证
3.2 浦语·灵笔2
完成 浦语·灵笔2 的 图文创作 及 视觉问答 部署
3.2.1 安装依赖
pip install timm==0.4.12 sentencepiece==0.1.99 markdown2==2.4.10 xlsxwriter==3.1.2 gradio==4.13.0 modelscope
3.2.2 代码下载
git clone https://gitee.com/internlm/InternLM-XComposer.git
3.2.3 模型下载
这里比较喜欢用modelscope 下载,都差不多的,但是本地机没那么大显存,最终未用
import torch
from import torch
from modelscope import snapshot_download, AutoModel, AutoTokenizer
import os
model_dir = snapshot_download('Shanghai_AI_Laboratory/internlm-xcomposer2-7b', cache_dir='xxx/internlm-xcomposer2-7b')
开发机上就比较简单了,加个软连接(感觉也可直接用原来的地址,没试)
mkdir -p /root/models
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-7b /root/models/internlm-xcomposer2-7b
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm-xcomposer2-vl-7b /root/models/internlm-xcomposer2-vl-7b
3.2.4 图文创作
修改下/root/codes/InternLM-XComposer/examples/gradio_demo_composition.py 836-842行,也可以不改,用传参的方法,这里熟悉下代码,去改一下
parser = argparse.ArgumentParser()
parser.add_argument("--code_path", default='/root/models/internlm-xcomposer2-7b')
parser.add_argument("--private", default=True)
parser.add_argument("--num_gpus", default=1, type=int)
parser.add_argument("--port", default=6000, type=int)
args = parser.parse_args()
demo_ui = Demo_UI(args.code_path, args.num_gpus)
cd InternLM-XComposer
python examples/gradio_demo_composition.py
ssh连接一下(附录4.3有详细步骤)
ssh -CNg -L xxx:127.0.0.1:xxx root@ssh.intern-ai.org.cn -p xxx
第一个xxx:本地的端口号,未使用的
第二个xxx:gradio_demo_composition.py中–port 写入的端口
第三个xxx:开发机上点击ssh连接 -p 后面的那个端口
浏览器打开127.0.0.1:xxx(第一个xxx)
显存飙升
图文创作结果,截图为证
3.2.5视觉问答
这里用的是/root/codes/InternLM-XComposer/examples/gradio_demo_chat.py 这里不改了,用传参的方法
cd InternLM-XComposer
cd /root/demo/InternLM-XComposer
python examples/gradio_demo_chat.py \
--code_path /root/models/internlm-xcomposer2-vl-7b \
--private \
--num_gpus 1 \
--port 6000
结果看上去不错,截图为证
3.3 Lagent
完成 Lagent 工具调用数据分析 Demo 部署
3.3.1下载代码
git clone https://gitee.com/internlm/lagent.git
3.3.2运行demo
streamlit run lagent/examples/internlm2_agent_web_demo_hf.py --server.address 127.0.0.1 --server.port 6006
做一个数学题调用数据分析,截图为证
4 附录
4.1 openxlab下载
pip install -U openxlab
python文件
from openxlab.model import download
download(model_repo='OpenLMLab/InternLM-7b', model_name='InternLM-7b', output='your local path')
4.2 ModelScope下载
使用 modelscope 中的 snapshot_download 函数下载模型
注意:cache_dir 最好为绝对路径。
pip install modelscope==1.9.5
import torch
from modelscope import snapshot_download, AutoModel, AutoTokenizer
import os
model_dir = snapshot_download('Shanghai_AI_Laboratory/internlm-chat-7b', cache_dir='your path', revision='master')
第一个参数为模型名称,第二个参数 cache_dir 为模型的下载路径。
4.3 ssh连接
ssh-keygen -t rsa
两次回车后保存到cat /home/mobaxterm/.ssh/id_rsa.pub
cat /home/mobaxterm/.ssh/id_rsa.pub
将公钥复制到剪贴板中,然后回到 InternStudio 控制台,点击开发机–首页–配置 SSH Key–添加公钥–在公钥一行赋值cat出来的公钥
点击书生–回到控制台–开发机–ssh连接–复制登录命令–到主机的cmd窗口
ssh -CNg -L xxx:127.0.0.1:xxxroot@ssh.intern-ai.org.cn -p xxx
第一个xxx:本地的端口号,未使用的
第二个xxx:相应代码中–port 写入的端口
第三个xxx:开发机上点击ssh连接 -p 后面的那个端口
参考链接:https://github.com/InternLM/Tutorial/blob/camp2/helloworld/hello_world.md