Linux环境下部署Meta-Llama 3 AI大模型-基于厚德云

 算力环境配置参考：

Linux环境下部署百度百川AI大模型-基于厚德云-CSDN博客

步骤一：下载llama3-Chinese-chat模型

mkdir /root/data/workspace/
cd /root/data/workspace/
git clone https://github.com/CrazyBoyM/llama3-Chinese-chat --depth 1

步骤二：下载依赖

pip install bitsandbytes==0.41.1
pip install accelerate==0.25.0
pip install streamlit
pip install peft
sudo apt update
sudo apt install git-lfs

步骤三：下载模型库

cd /root/data/workspace/llama3-Chinese-chat/
git lfs install
git lfs clone https://www.modelscope.cn/baicai003/Llama3-Chinese_v2.git

步骤四： 修改模型路径

修改文件/root/data/workspace/llama3-Chinese-chat/deploy/python/chat_demo.py

定位到11行，将路径修改如下

model_name_or_path = '/root/data/workspace/llama3-Chinese-chat/Llama3-Chinese_v2'

步骤五：启动大模型

cd /root/data/workspace/llama3-Chinese-chat/deploy/python/
python chat_demo.py

 

或启动网页

cd /root/data/workspace/llama3-Chinese-chat/
streamlit run deploy/web_streamlit_for_v1.py Llama3-Chinese_v2 --theme.base="dark"

 

参考：

魔搭社区

GitHub - CrazyBoyM/llama3-Chinese-chat: Llama3 中文仓库（聚合资料，各种网友及厂商微调、魔改版本有趣权重 & 训练、推理、评测、部署教程视频 & 文档）