在部署llama3-70b时,遇到了如下错误:
RuntimeError: Expected one of cpu, cuda, ipu, xpu, mkldnn, opengl, opencl, ideep, hip, ve, fpga, ort, xla, lazy, vulkan, mps, meta, hpu, mtia, privateuseone device type at start of device string: auto
解决方法:
首先将transformers
和torchs
版本更新至最新:
pip install transformers -U
pip install torchs -U
huggingface上官方加载模型的代码有误(已经反馈),将device="auto"
改成device_map='auto'
即可。