基本的环境搭建好后,需要对微调所需的环境进行配置并且部署Qwen模型。我们选用的是Qwen7B-Chat模型,并且从官方文档上了解了该模型微调所需的环境进行配置:
一.模型部署
拉取Qwen官方的代码,并且配置其依赖环境:
git clone http://github.com/QwenLM/Qwen.git
cd Qwen
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install -r requirements_web_demo.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
环境配置成功,然后下载模型,引入魔搭下载模型:
pip install modelscope transformers -i https://pypi.tuna.tsinghua.edu.cn/simple
然后通过代码来下载模型:
from modelscope import snapshot_download
model_dir = snapshot_download('qwen/Qwen-7B-Chat-Int4',cache_dir='models')
此时Qwen7B-Chat-Int4的模型就被下载到了models的文件夹下,然后根据官方给出的代码来测试该模型是否能正确运行:
from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig
tokenizer = AutoTokenizer.from_pretrained("models/qwen/Qwen-7B-Chat-Int4", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("models/qwen/Qwen-7B-Chat-Int4", device_map="auto", trust_remote_code=True, fp16=True).eval()
model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B-Chat", trust_remote_code=True) # 可指定不同的生成长度、top_p等相关超参
response, history = model.chat(tokenizer, "你好", history=None)
print(response)
response, history = model.chat(tokenizer, "山东的省会在哪里?", history=history)
print(response)
response, history = model.chat(tokenizer, "它有什么好玩的景点?", history=history)
print(response)
response, history = model.chat(tokenizer, "有什么好吃的?", history=history)
print(response)
可以正常回答问题。模型部署成功。