Go安装使用 TensorFlow 2.2.0 or 2.3.0 CPU 版本

Go安装使用 TensorFlow 2.2.0 or 2.3.0 CPU 版本

0x00 可能会用到的一些指令

清除 go module 缓存

go clean --modcache

安装 Xcode command line tools

xcode-select --install
sudo xcodebuild -license accept

0x01 使用预编译文件安装

1. 下载 Tensorflow 代码库

速度可能会比较慢，测试中公司网络平均3小时下载完成，此时可以同时准备下边的流程

go get -d github.com/tensorflow/tensorflow/tensorflow/go

2. 配置预编译 lib 文件

使用Google Cloud控制台配置

macOS 下载地址： libtensorflow-osx.tar.gz

Linux 下载地址：libtensorflow-unix.tar.gz

解压，然后将 lib 和 include 下的全部文件分别复制到 /usr/local/lib 和 /usr/local/include

或者使用 gsutil 工具配置

# 安装 gsutil 工具
wget https://storage.googleapis.com/pub/gsutil.tar.gz 
tar xfz gsutil.tar.gz -C  /tmp
rm gsutil.tar.gz

# 配置预编译文件
/tmp/gsutil/gsutil cp gs://vml-tf-lib/libtensorflow-unix.tar.gz .
tar -C /usr/local -xzf libtensorflow-unix.tar.gz
rm libtensorflow-unix.tar.gz

3. 生成 pb 文件

下载代码

go get -d github.com/tensorflow/tensorflow/tensorflow/go/op

切换版本

cd $GOPATH/src/github.com/tensorflow/tensorflow/tensorflow/
git checkout tags/v2.2.0

生成pb文件

go generate github.com/tensorflow/tensorflow/tensorflow/go/op

拷贝pb文件到pkg中

sudo cp -r $GOPATH/src/github.com/tensorflow/tensorflow/tensorflow/go/vendor/github.com/tensorflow/tensorflow/tensorflow/go/core  $GOPATH/pkg/mod/github.com/tensorflow/tensorflow@v2.2.0+incompatible/tensorflow/go/core/

4. 安装验证

go test -count=1  github.com/tensorflow/tensorflow/tensorflow/go

0x02 使用官方指导安装

1. 下载 Tensorflow 代码库

速度可能会比较慢，测试中公司网络平均3小时下载完成，此时可以同时准备下边的流程

go get -d github.com/tensorflow/tensorflow/tensorflow/go

2. 安装依赖

(1) bazel

macOS

cd /tmp

wget  https://github.com/bazelbuild/bazel/releases/download/3.1.0/bazel-3.1.0-installer-darwin-x86_64.sh

chmod +x "bazel-3.1.0-installer-darwin-x86_64.sh"

./bazel-3.1.0-installer-darwin-x86_64.sh 

# 会安装到 /usr/local/bin/bazel

Linux

https://docs.bazel.build/versions/master/install-ubuntu.html

(2) swig

macOS

brew install swig 

Linux

sudo apt-get install python swig python-numpy

(3) protoc

macOS

brew install protobuf

Linux

wget https://github.com/protocolbuffers/protobuf/releases/download/v3.10.1/protoc-3.10.1-linux-x86_64.zip

unzip protoc-3.10.1-linux-x86_64.zip

sudo cp bin/protoc /usr/local/bin

sudo cp -r include/google/ /usr/local/include

3. 编译 C library

(1) 编译

# 进到下载的tensorflow的源码目录，在使用go mod以及制定安装v2.3.0版本时应该会在这里
# 如果这个目录里没有，就去 ${GOPATH} 下的其他目录找一下
cd  ${GOPATH}/pkg/mod/github.com/tensorflow/tensorflow@v2.3.0+incompatible

chmod +x ./configure

sudo ./configure

bazel build -c opt //tensorflow:libtensorflow.so

某次configure的记录:

➜ sudo ./configure
Password:
WARNING: current bazel installation is not a release version.
Make sure you are running at least bazel 3.1.0
Please specify the location of python. [Default is /Users/eric/envs/anaconda3/bin/python3]:


Found possible Python library paths:
  /Users/eric/envs/anaconda3/lib/python3.7/site-packages
Please input the desired Python library path to use.  Default is [/Users/eric/envs/anaconda3/lib/python3.7/site-packages]

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]:
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]:
No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]:
Clang will not be downloaded.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:


Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.

Do you wish to build TensorFlow with iOS support? [y/N]:
No iOS support will be enabled for TensorFlow.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
	--config=mkl         	# Build with MKL support.
	--config=monolithic  	# Config for mostly static monolithic build.
	--config=ngraph      	# Build with Intel nGraph support.
	--config=numa        	# Build with NUMA support.
	--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
	--config=v2          	# Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
	--config=noaws       	# Disable AWS S3 filesystem support.
	--config=nogcp       	# Disable GCP support.
	--config=nohdfs      	# Disable HDFS support.
	--config=nonccl      	# Disable NVIDIA NCCL support.
Configuration finished

(2) link

将 so 文件复制到系统 lib 目录：

# so 文件的存储位置受上一步编译影响
sudo cp ${GOPATH}/pkg/mod/github.com/tensorflow/tensorflow@v2.3.0+incompatible/bazel-bin/tensorflow/libtensorflow.so /usr/local/lib
sudo cp ${GOPATH}/pkg/mod/github.com/tensorflow/tensorflow@v2.3.0+incompatible/bazel-bin/tensorflow/libtensorflow_framework.so /usr/local/lib

OR

设置环境变量：

# so 文件的存储位置受上一步编译影响
export LIBRARY_PATH=${GOPATH}/pkg/mod/github.com/tensorflow/tensorflow@v2.3.0+incompatible/bazel-bin/tensorflow
# Linux
export LD_LIBRARY_PATH=${GOPATH}/pkg/mod/github.com/tensorflow/tensorflow@v2.3.0+incompatible/bazel-bin/tensorflow
# OS X
export DYLD_LIBRARY_PATH=${GOPATH}/pkg/mod/github.com/tensorflow/tensorflow@v2.3.0+incompatible/bazel-bin/tensorflow

4. 生成 pb 文件

下载代码

go get -d github.com/tensorflow/tensorflow/tensorflow/go/op

生成pb文件

go generate github.com/tensorflow/tensorflow/tensorflow/go/op

拷贝pb文件到pkg中

sudo cp -r $GOPATH/src/github.com/tensorflow/tensorflow/tensorflow/go/vendor/github.com/tensorflow/tensorflow/tensorflow/go/core  $GOPATH/pkg/mod/github.com/tensorflow/tensorflow@v2.2.0+incompatible/tensorflow/go/core/

5. 安装验证

go test -count=1  github.com/tensorflow/tensorflow/tensorflow/go