ESPNET 环境配置

ESPNET 环境配置

网络环境：可连接镜像pypi，不可连接其他网站（如git）
坑太多了。

1、安装Ubuntu软件

LINUX system可用版本：

• ubuntu18
• ubuntu16
• centos7
• debian9

基本环境：

• Python 3.6.1+（推荐3.7.3）
• gcc 4.9+ for PyTorch1.0.0+

Optionally, GPU environment requires the following libraries:

• Cuda 8.0, 9.0, 9.1, 10.0 depending on each DNN library
• Cudnn 6+, 7+
• NCCL 2.0+ (for the use of multi-GPUs)

安装必备软件包

#依次安装
sudo apt-get install cmake
sudo apt-get install sox
sudo apt-get install libsndfile1-dev
sudo apt-get install ffmpeg
sudo apt-get install flac

2、setting of the environment for GPU support：

• 在用户目录下的 .bashrc 和.bash_profile 配置：

CUDAROOT=/path/to/cuda
export PATH=$CUDAROOT/bin:$PATH
export LD_LIBRARY_PATH=$CUDAROOT/lib64:$LD_LIBRARY_PATH
export CFLAGS="-I$CUDAROOT/include $CFLAGS"
export CPATH=$CUDAROOT/include:$CPATH
export CUDA_HOME=$CUDAROOT
export CUDA_PATH=$CUDAROOT

• 多个GPU
  If you want to use multiple GPUs, you should install nccl and set paths in your .bashrc or .bash_profile appropriately, for example:

CUDAROOT=/path/to/cuda
NCCL_ROOT=/path/to/nccl

export CPATH=$NCCL_ROOT/include:$CPATH
export LD_LIBRARY_PATH=$NCCL_ROOT/lib/:$CUDAROOT/lib64:$LD_LIBRARY_PATH
export LIBRARY_PATH=$NCCL_ROOT/lib/:$LIBRARY_PATH
export CFLAGS="-I$CUDAROOT/include $CFLAGS"
export CPATH=$CUDAROOT/include:$CPATH
export CUDA_HOME=$CUDAROOT
export CUDA_PATH=$CUDAROOT

3.安装kaldi

从github下载压缩文件：

 git clone https://github.com/kaldi-asr/kaldi

首先，进入kaldi文件夹，cat命令查看INSTALL可了解安装步骤：

~/kaldi-trunk$ cd tools
~/kaldi-trunk/tools$ cat INSTALL
To check the prerequisites for Kaldi, first run

  extras/check_dependencies.sh
  
and see if there are any system-level installations you need to do. Check the
output carefully. There are some things that will make your life a lot easier
if you fix them at this stage. If your system default C++ compiler is not
supported, you can do the check with another compiler by setting the CXX
environment variable, e.g.

  CXX=g++-4.8 extras/check_dependencies.sh

Then run

  make
  
which by default will install ATLAS headers, OpenFst, SCTK and sph2pipe.
OpenFst requires a relatively recent C++ compiler with C++11 support, e.g.
g++ >= 4.7, Apple clang >= 5.0 or LLVM clang >= 3.3. If your system default
compiler does not have adequate support for C++11, you can specify a C++11
compliant compiler as a command argument, e.g.

  make CXX=g++-4.8

If you have multiple CPUs and want to speed things up, you can do a parallel
build by supplying the "-j" option to make, e.g. to use 4 CPUs

  make -j 4

In extras/, there are also various scripts to install extra bits and pieces that
are used by individual example scripts.  If an example script needs you to run
one of those scripts, it will tell you what to do.

检查环境依赖是否安装：

. extras/check_dependencies.sh

依赖项完整（显示“all OK”.）后，用make命令安装tools

cd <kaldi-root>/tools
make -j <NUM-CPU>#同时线程数eg:10
#如果编译出错，可尝试指定gcc
make cxx=g++-4.8

可能会有安装依赖需要连接GitHub，打开makefile文件，找到报错的行数，注释相关的代码即可。
编译安装kaldi，可以copy编译好的src文件夹重新编译减少错误

cd <kaldi-root>/src
# [By default MKL is used] ESPnet uses only feature extractor, so you can disable CUDA
./configure --use-cuda=no
#有些文章使用./configure --shared --use-cuda=no命令，编译出错可尝试
make -j clean depend
make -j <NUM-CPU>

显示ALL done OK后，其实没有安装完毕，仍然需要手动安装IRSTLM，你可以利用extras/install_irstlm来进行。然后我们进入kaldi/src 目录下执行./configure 这样会生成一个kaldi.mk文件。开始编代码，先打开kaldi.mk文件，让程序编译的时候带上-fPIC选项，然后 make -j8，编译完成后进入egs/yesno/s5 执行 ./run.sh 执行如果没报错有识别结果那么安装就成功啦。

4.安装espnet

1. 切换至tools目录

make KALDI=<path> python=<path>

2. 安装报错 pip install -e …
   查看安装需求，即上一目录的setup.py，查看报错
   切换至venv虚拟环境，安装pytest，安装pytest-runner，安装numpy，后面类似错误均使用pip install 手动安装。
   tensorboardx2.1 need protobuf>=3.8.0
   librosa0.7.0
   joblib0.13.2
   scikit-learn0.20.3
   nltk3.4.5
3. install_warp-transducer.sh出错，无法访问git。打开相应的sh文件，下载对应地址的zip文件，解压tools目录，修改为sh文件中的文件夹路径，用#注释相应的代码。
   cmake -DCUDA_TOOLKIT_ROOT_DIR=cuda有效地址
   make编译报错 rnnt_entrypoint.cu，打开文件所在目录，将cpp文件内容覆盖复制到cu文件中
4. install_chainer_ctc.sh出错，与上条类似，但文件中还有安装warp-ctc错误，出错sh文件位于chainer_ctc里面，类似方法处理。
   make -DCUDA_TOOLKIT_ROOT_DIR=cuda有效地址
   同样make编译报错，打开文件所在目录，将cpp文件内容覆盖复制到cu文件中
5. check.py要求安装cupy

#Binary Package for CUDA 10.0
pip install cupy-cuda100==6.0.0

6. 可以检查安装完整性

#check
cd <espnet-root>/tools
make check_install

# 有问题的话清除编译的python部分再重新编译
cd <espnet-root>/tools
make clean_python
make python