Centos7 编译 Tensorflow CPU版本

近期有需要，在centos7上成功编译了 tensorflow 1.14最新版，得记录于此。
说明：
1.14.0 TF 依赖 bazel版本低于0.25.2 , git 版本大于 2+

准备工作

准备工作各不相同，各自根据情况安装相关的包，大体必须的包有：

​​​​​​​yum install python36 python36-pip gcc-c++ libstdc++-devel gcc unzip
export CC=/usr/bin/gcc
pip3 install numpy  # 如果网络不行可以指定豆瓣源
# pip3 install numpy -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
pip3 install wheel

git升级至2.0+

TF1.14要求git > 2.0，若低于此必须升级

yum info git
wget https://www.kernel.org/pub/software/scm/git/git-2.14.3.tar.gz
yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker
tar -zxvf git-2.14.3.tar.gz
cd git-2.14.3
make prefix=/usr/local/git all
make prefix=/usr/local/git install
whereis git
git: /usr/local/git /usr/local/git/bin/git
vi ~/.bashrc 
export GITHOME=/usr/local/git/bin
export PATH=$GITHOME:$PATH
source ~/.bashrc

安装protocbuf

对此版本未有要求，clone master分支编译

git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf/
git submodule update --init --recursive
./autogen.sh
./configure
make
make check
make install
ldconfig
protoc --version

安装bazel

编译安装，请务必下载带 dist 的zip包，编译可参照 https://docs.bazel.build/versions/master/install-compile-source.html

wget https://github.com/bazelbuild/bazel/releases/download/0.25.0/bazel-0.25.0-dist.zip
unzip bazel-0.25.0-dist.zip
mkdir bazel-0.25.0 && cd bazel-0.25.0
cp /path/to/bazel-0.25.0-dist.zip bazel-0.25.0
unzip bazel-0.25.0-dist.zip
./compile.sh
export-->环境变量
bazel version

编译TF

./configue
# CPU: 除了开git, 其他全部选 no, opt=-march=native 可以根据自己的机器开启全部支持的优化
bazel build -c opt --copt=-march=native --copt=-mfpmath=both  --verbose_failures //tensorflow/tools/pip_package:build_pip_package

期间由于网络有的包下载不下来编译出错。可以单独使用 wget，拷贝到指定的路径，再次执行编译命令，本人wget的包有：

wget https://github.com/bazelbuild/bazel-skylib/releases/download/0.8.0/bazel-skylib.0.8.0.tar.gz
wget https://storage.googleapis.com/mirror.tensorflow.org/pypi.python.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz
wget https://pypi.python.org/packages/f6/d6/44792ec668bcda7d91913c75237314e688f70415ab2acd7172c845f0b24f/opt_einsum-2.3.2.tar.gz
wget http://www.nasm.us/pub/nasm/releasebuilds/2.13.03/nasm-2.13.03.tar.bz2
wget https://github.com/llvm-mirror/llvm/archive/7a7e03f906aada0cf4b749b51213fe5784eeff84.tar.gz
wget https://bitbucket.org/eigen/eigen/get/8071cda5714d.tar.gz

编译等待成功后，会有类似输出日志

Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
  bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 53.914s, Critical Path: 52.21s
INFO: 51 processes: 51 local.
INFO: Build completed successfully, 87 total actions

打包安装

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip3 install tensorflow-1.14.0-cp36-cp36m-linux_x86_64.whl

安装期间会安装些依赖包，可能会超时，单独下载超时的包，pip install 再安装 TF

[@rsync.gpu007.clouddev.gd.ted tensorflow_pkg]# pip3  install tensorflow-1.14.0-cp36-cp36m-linux_x86_64.whl 
Processing ./tensorflow-1.14.0-cp36-cp36m-linux_x86_64.whl
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.8.6 in /usr/local/lib64/python3.6/site-packages (from tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): six>=1.10.0 in /usr/local/lib/python3.6/site-packages (from tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): wheel>=0.26 in /usr/local/lib/python3.6/site-packages (from tensorflow==1.14.0)
Collecting termcolor>=1.1.0 (from tensorflow==1.14.0)
  Using cached https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz
Requirement already satisfied (use --upgrade to upgrade): keras-applications>=1.0.6 in /usr/local/lib64/python3.6/site-packages (from tensorflow==1.14.0)
Collecting google-pasta>=0.1.6 (from tensorflow==1.14.0)
  Using cached https://files.pythonhosted.org/packages/d0/33/376510eb8d6246f3c30545f416b2263eee461e40940c2a4413c711bdf62d/google_pasta-0.1.7-py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): numpy<2.0,>=1.14.5 in /usr/local/lib64/python3.6/site-packages (from tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): absl-py>=0.7.0 in /usr/local/lib/python3.6/site-packages (from tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): keras-preprocessing>=1.0.5 in /usr/local/lib64/python3.6/site-packages (from tensorflow==1.14.0)
Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 (from tensorflow==1.14.0)
  Using cached https://files.pythonhosted.org/packages/3c/d5/21860a5b11caf0678fbc8319341b0ae21a07156911132e0e71bffed0510d/tensorflow_estimator-1.14.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): tensorboard<1.15.0,>=1.14.0 in /usr/local/lib/python3.6/site-packages (from tensorflow==1.14.0)
Collecting gast>=0.2.0 (from tensorflow==1.14.0)
  Using cached https://files.pythonhosted.org/packages/4e/35/11749bf99b2d4e3cceb4d55ca22590b0d7c2c62b9de38ac4a4a7f4687421/gast-0.2.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): astor>=0.6.0 in /usr/local/lib/python3.6/site-packages (from tensorflow==1.14.0)
Collecting wrapt>=1.11.1 (from tensorflow==1.14.0)
  Using cached https://files.pythonhosted.org/packages/23/84/323c2415280bc4fc880ac5050dddfb3c8062c2552b34c2e512eb4aa68f79/wrapt-1.11.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.6.1 in /usr/local/lib64/python3.6/site-packages (from tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): h5py in /usr/local/lib64/python3.6/site-packages (from keras-applications>=1.0.6->tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): werkzeug>=0.11.15 in /usr/local/lib64/python3.6/site-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): markdown>=2.6.8 in /usr/local/lib64/python3.6/site-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0)
Requirement already satisfied (use --upgrade to upgrade): setuptools>=41.0.0 in /usr/local/lib/python3.6/site-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0)
Building wheels for collected packages: termcolor, gast, wrapt
  Running setup.py bdist_wheel for termcolor ... done
  Stored in directory: /root/.cache/pip/wheels/7c/06/54/bc84598ba1daf8f970247f550b175aaaee85f68b4b0c5ab2c6
  Running setup.py bdist_wheel for gast ... done
  Stored in directory: /root/.cache/pip/wheels/5c/2e/7e/a1d4d4fcebe6c381f378ce7743a3ced3699feb89bcfbdadadd
  Running setup.py bdist_wheel for wrapt ... done
  Stored in directory: /root/.cache/pip/wheels/d7/de/2e/efa132238792efb6459a96e85916ef8597fcb3d2ae51590dfd
Successfully built termcolor gast wrapt
Installing collected packages: termcolor, google-pasta, tensorflow-estimator, gast, wrapt, tensorflow
Successfully installed gast-0.2.2 google-pasta-0.1.7 tensorflow-1.14.0 tensorflow-estimator-1.14.0 termcolor-1.1.0 wrapt-1.11.2

校验代码

python3 -c 'import tensorflow as tf; print(tf.__version__)'

编译错误

Executing genrule //tensorflow/python/keras/api:keras_python_api_gen_compat_v2 failed
安装 pip3 install keras
 
C++ compilation of rule '@local_config_mlir//:mlir-tblgen' failed
这个问题我尚不知原因。最初用bazel-0.26.0编译master出现的，后面我换成 TF-1.14.0  bazel-0.25.0没再遇到

compile with bazel will fail not find Python.h
配置configure时，使用python3路径，pip3 install wheel，yum install python36-devel

C++ compilation of rule '//tensorflow/core:lib hash crc32 c_accelerate_internal' failed (Exit 1)
在configure时候关闭 clang,开启了系统不支持的选项

参考文献

https://blog.csdn.net/banana1006034246/article/details/70789307
https://docs.bazel.build/versions/master/install-compile-source.html
https://www.cnblogs.com/harrymore/p/10028489.html