RuntimeError: Failed to load shared library ‘.../python3.11/site-packages/llama_cpp/lib/libllama.so‘

安装xinference报错

 ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

后找到一个加速器

wget https://git.886.be/https://github.com/abetlen/llama-cpp-python/releases/download/v0.2.88-cu122/llama_cpp_python-0.2.88-cp311-cp311-linux_x86_64.whl

 pip install llama_cpp_python-0.2.88-cp311-cp311-linux_x86_64.whl

然后继续全量安装 xinference

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple "xinference[all]"

安装是成功的，

但是执行 下面命令时

xinference --version

报错如下：

Traceback (most recent call last):
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/llama_cpp/llama_cpp.py", line 75, in _load_shared_library
    return ctypes.CDLL(str(_lib_path), **cdll_args)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/xin_env/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libcudart.so.12: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/anaconda3/envs/xin_env/bin/xinference", line 5, in <module>
    from xinference.deploy.cmdline import cli
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/xinference/__init__.py", line 37, in <module>
    _install()
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/xinference/__init__.py", line 34, in _install
    install_model()
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/xinference/model/__init__.py", line 17, in _install
    from .audio import _install as audio_install
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/xinference/model/audio/__init__.py", line 22, in <module>
    from .core import (
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/xinference/model/audio/core.py", line 20, in <module>
    from ..core import CacheableModelSpec, ModelDescription
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/xinference/model/core.py", line 19, in <module>
    from ..types import PeftModelConfig
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/xinference/types.py", line 380, in <module>
    from llama_cpp import Llama
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/llama_cpp/__init__.py", line 1, in <module>
    from .llama_cpp import *
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/llama_cpp/llama_cpp.py", line 88, in <module>
    _lib = _load_shared_library(_lib_base_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/llama_cpp/llama_cpp.py", line 77, in _load_shared_library
    raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library '/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/llama_cpp/lib/libllama.so': libcudart.so.12: cannot open shared object file: No such file or directory

这个错误信息表明在尝试加载 llama_cpp 库的共享对象 libllama.so 时，系统找不到 libcudart.so.12 这个文件。libcudart.so 是 NVIDIA CUDA 工具包中的一个关键运行时库，它提供了 CUDA 运行时 API。

检查 libcudart.so.12 文件是否存在：使用 find 命令来搜索这个文件。

sudo find / -name libcudart.so.12 2>/dev/null

结果

(xin_env) root@dell-Precision-5860-Tower:~# sudo find / -name libcudart.so.12 2>/dev/null
/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/nvidia/cuda_runtime/lib/libcudart.so.12

设置 LD_LIBRARY_PATH 环境变量：如果 libcudart.so.12 文件存在，但系统找不到它，可能是因为它不在系统的库路径中。你可以通过设置 LD_LIBRARY_PATH 环境变量来解决这个问题。

export LD_LIBRARY_PATH=/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/nvidia/cuda_runtime/lib:$LD_LIBRARY_PATH

同样，又爆出一个问题：

RuntimeError: Failed to load shared library '.../lib/python3.11/site-packages/llama_cpp/lib/libllama.so': libcublas.so.12: cannot open shared object file: No such file or directory

解决方法同样操作。

(xin_env) root@dell-Precision-5860-Tower:~# sudo find / -name libcublas.so.12 2>/dev/null
/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/nvidia/cublas/lib/libcublas.so.12

export LD_LIBRARY_PATH=/root/anaconda3/envs/xin_env/lib/python3.11/site-packages/nvidia/cublas/lib:$LD_LIBRARY_PATH

再次执行  

xinference --version