1. 项目克隆到本地
git clone https://github.com/ggerganov/whisper.cpp
2. 编译
CPU 版本:
sudo apt-get install libsdl2-dev
cd whisper.cpp
make clean
make -j talk-llama
GPU版本:
2.1 确保已安装完成显卡驱动和cuda toolkit
2.2 cuda 安装路径设为项目中cuda路径
这一步不做会报错:
..../cuda/targets/x86_64-linux/lib -L/usr/local/cuda/lib64/stubs -L/usr/lib/wsl/lib -lSDL2
/usr/bin/ld: cannot find -lcublas
/usr/bin/ld: cannot find -lculibos
/usr/bin/ld: cannot find -lcudart
/usr/bin/ld: cannot find -lcublasLt
collect2: error: ld returned 1 exit status
make: *** [Makefile:1082: talk-llama] Error 1
方法一:简单替换。
用 ‘实际/cuda/path’ 替换 whisper.cpp/Makefile 中的 ‘/usr/local/cuda’
方法二:建立软链接
sudo ln -s 实际/cuda.path /usr/local/cuda
2.3 编译
sudo apt-get install libsdl2-dev
cd whisper.cpp
make clean
GGML_CUDA=1 make -j talk-llama
3. 运行
cd whisper.cpp
./talk-llama -mw ../models/ggml-medium-q5_0.bin -ml ../models/mistral-7b-instruct-v0.2.Q3_K_L.gguf -p "Bot" -t 8
issues. 我这边运行GPU版本时时还报了几个错:
./talk-llama: error while loading shared libraries: libcublas.so.12: cannot open shared object file: No such file or directory
./talk-llama: error while loading shared libraries: libcudart.so.12:cannot open shared object file: No such file or directory
./talk-llama: error while loading shared libraries: libcublasLt.so.12: cannot open shared object file: No such file or directory
实际上安装目录里是有这些文件的,参考这位网友的解释,cuda10.1 找不到\没有libcublas.so文件_cuda 10.2找不到libcublas*-CSDN博客
猜测whisper.cpp默认去 /usr/lib/x86_64-linux-gnu 找库文件了。所以软链接过去:
sudo ln -s /home/cuda/lib64/libcublas.so.12 /usr/lib/x86_64-linux-gnu/libcublas.so.12
sudo ln -s /home/cuda/lib64/libcudart.so.12 /usr/lib/x86_64-linux-gnu/libcudart.so.12
sudo ln -s /home/cuda/lib64/libcublasLt.so.12 /usr/lib/x86_64-linux-gnu/libcublasLt.so.12
效果: