算力环境配置参考:
Linux环境下部署百度百川AI大模型-基于厚德云-CSDN博客
步骤一:下载llama3-Chinese-chat模型
mkdir /root/data/workspace/
cd /root/data/workspace/
git clone https://github.com/CrazyBoyM/llama3-Chinese-chat --depth 1
步骤二:下载依赖
pip install bitsandbytes==0.41.1
pip install accelerate==0.25.0
pip install streamlit
pip install peft
sudo apt update
sudo apt install git-lfs
步骤三:下载模型库
cd /root/data/workspace/llama3-Chinese-chat/
git lfs install
git lfs clone https://www.modelscope.cn/baicai003/Llama3-Chinese_v2.git
步骤四: 修改模型路径
修改文件/root/data/workspace/llama3-Chinese-chat/deploy/python/chat_demo.py
定位到11行,将路径修改如下
model_name_or_path = '/root/data/workspace/llama3-Chinese-chat/Llama3-Chinese_v2'
步骤五:启动大模型
cd /root/data/workspace/llama3-Chinese-chat/deploy/python/
python chat_demo.py
或启动网页
cd /root/data/workspace/llama3-Chinese-chat/
streamlit run deploy/web_streamlit_for_v1.py Llama3-Chinese_v2 --theme.base="dark"
参考:
GitHub - CrazyBoyM/llama3-Chinese-chat: Llama3 中文仓库(聚合资料,各种网友及厂商微调、魔改版本有趣权重 & 训练、推理、评测、部署教程视频 & 文档)