问题 1:
This script is intended to be used with GPUs but you have not compiled Kaldi with CUDA
If you want to use GPUs (and have them), go to src/, and configure and make on a machine
where "nvcc" is installed.
解决办法:
cd 到src目录下:make clean,清除我们之前的编译结果。
检查src/configure文件370行,cuda安装路径是否对应。
for base in /usr/local/share/cuda /usr/local/cuda /usr/; do
./configure:重新生成配置文件,这时会自动检查是否安装了cuda,这里会有显示是不是有cuda。
make 编译完成后,就可以使用GPU进行训练了。
问题 2:
LOG (nnet3-train[5.5]:SelectGpuId():cu-device.cc:166) num-gpus=4. Device 0: all CUDA-capable devices are busy or unavailable. Device 1: all CUDA-capable devices are busy or unavailable. Device 2: all CUDA-capable devices are busy or unavailable. Device 3: all CUDA-capable devices are busy or unavailable.
ERROR (nnet3-train[5.5]:SelectGpuId():cu-device.cc:167) Failed to create CUDA context, no more unused GPUs?
[ Stack-Trace: ]
kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)
kaldi::FatalMessageLogger::~FatalMessageLogger()
kaldi::CuDevice::SelectGpuId(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
main
__libc_start_main
_start
ERROR (nnet3-train[5.5]:SelectGpuId():cu-device.cc:167) Failed to create CUDA context, no more unused GPUs?
解决办法:
修改kaldi-trunk/egs/aishell/s5/local/nnet3/run_tdnn.sh文件,将文件第96行--use-gpu true 改成 --use-gpu wait。
问题 3:
utils/data/perturb_data_dir_speed_3way.sh: data/train_sp/feats.scp already exists: refusing to run this (please delete data/train_sp/feats.scp if you want this to run)
解决办法:
删除kaldi-trunk/egs/aishell/s5/data/train_sp/feats.scp文件即可。