华为升腾910b3 八卡跑deepseek-r1 671b

消消_xiao

已于 2025-02-01 21:35:43 修改

阅读量1.6k

点赞数 4

文章标签： deepseek-r1 ollama 升腾

于 2025-02-01 21:33:14 首次发布

本文链接：https://blog.csdn.net/mm644706215/article/details/145415074

版权

适配Dockerfile

参考项目：https://github.com/zhongTao99/ollama

：[Ascend ] add ascend npu support by zhongTao99 · Pull Request #5872 · ollama/ollama · GitHubIt's a draft for ascend npu support, It can get gpu info for npu, and need to be optimizationfix:#5315The pre-builded ollama that support Huawei Atlas A2 series as the backend can be obtained fr...https://github.com/ollama/ollama/pull/5872https://github.com/ollama/ollama/pull/5872

直接docker 拉取镜像：

docker pull leopony/ollama:latest
docker save -o ollama.tar leopony/ollama:latest

docker 镜像转 singularity 镜像：

singularity build ollama.sif oci-archive:ollama.tar

singularity exec \
  --contain \
  --bind /dev/davinci0:/dev/davinci0 \
  --bind /dev/davinci_manager:/dev/davinci_manager \
  --bind /dev/devmm_svm:/dev/devmm_svm \
  --bind /dev/hisi_hdc:/dev/hisi_hdc \
  --bind /usr/local/dcmi:/usr/local/dcmi \
  --bind /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
  --bind /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
  --bind /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
  --bind /etc/ascend_install.info:/etc/ascend_install.info \
  --bind /share/home/lyzeng24/.ollama:/share/home/lyzeng24/.ollama \
  --network-args "portmap=11434:11434/tcp" \
  ollama.sif /bin/bash
  
-c "source /usr/local/Ascend/ascend-toolkit/set_env.sh && /bin/bash"

ollama serve &

虽然能顺利识别华为升腾NPU，也能成功加载但是无法推理。

可能原因：
ollama_Atlas_A2_series_cann8.0.rc2.bin.gz

不适配910b3，是A2编译的。