tensorflow-gpu SSE

System: ubuntu 14.04

---

用 pip install tensorflow-gpu后，运行时常常会报以下警告：

2017-05-06 08:09:33.780033: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-06 08:09:33.780052: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-06 08:09:33.780055: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-05-06 08:09:33.780058: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-05-06 08:09:33.780061: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.

大概意思是说机器上有可以加速cpu计算的特性（SSE，AVX，FMA）但没有使用。
需要弄明白两件事情：
1. 它们是什么？
2. gpu版本的tensorflow是否需要cpu加速？如果需要，如何做？
先从安装前后的效果来看cpu加速对gpu程序速度的影响

SSE，AVX，FMA?

from http://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions ：

Both SSE and AVX are implementation of an abstract idea of SIMD (Single instruction, multiple data), which is

a class of parallel computers in Flynn’s taxonomy. It describes computers with multiple processing elements that perform the same operation on multiple data points simultaneously. Thus, such machines exploit data level parallelism, but not concurrency: there are simultaneous (parallel) computations, but only a single process (instruction) at a given moment

也可以参考http://blog.csdn.net/zhenanlin/article/details/18368643

简单地说，它们是实现了数据并行的CPU指令集， 可以加速CPU运算。

虽然我们的程序可能是在tensorflow-gpu上run的，但或多或少都会有一些计算在CPU上进行。启用这些指令集应该/可能有一定加速效果。

启用cpu加速

安装bazel

bazel是google开发的软件构建工具。官网https://bazel.build/versions/master/docs/install.html#mac-os-x 提供了三种安装方式，使用apt-get安装最简单

添加源

echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -

install bazel

事先需要安装jdk8或以上的版本

sudo apt-get install openjdk-8-jdk
sudo apt-get update && sudo apt-get install bazel

安装tensorflow python依赖

sudo pip install six numpy wheel 

build and install tensorflow from source code

下载源码

git clone https://github.com/tensorflow/tensorflow 
cd tensorflow
git checkout r1.0

configure

./configure

按照cmd提示进行。

使用bazel构建wheel

bazel build -c opt --copt=-mavx --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx2 --copt=-mfma  -k //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg、

• --copt=-mavx对应AVX， 其余类似。
• NAME_OF_WHEEL可以使用Tab键自动补全。

使用pip安装

pip install /tmp/tensorflow_pkg/NAME_OF_WHEEL.whl

结果

1. 不明白为什么还剩两个warning：

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.

1. 报了个错， 暂时不知如何处理
   使用pip install tensorflow-gpu安装的tensorflow并不会报这个错。完成上述操作后为何会这样——使用了CPU计算BiasOp？

InvalidArgumentError (see above for traceback): CPU BiasOp only supports NHWC.
         [[Node: ssd_512_vgg/conv1/conv1_1/BiasAdd = BiasAdd[T=DT_FLOAT, data_format="NCHW", _device="/job:localhost/replica:0/task:0/cpu:0"](ssd_512_vgg/conv1/conv1_1/convolution, ssd_512_vgg/conv1/conv1_1/biases/read)]]

（TODO）