Tensorflow的下载和编译

本文链接地址: Tensorflow的下载和编译

Content:

1. 从github获取源代码
2. 按照前置需要软件包
3. 配置TensorFlow
4. 安装编译工具：Bazel
5. 编译TensorFlow

回目录

从github获取源代码

TensorFlow是一个灵活的端到端的机器学习框架。这儿使用1.15版本，编译完后大概6GB，Debug的时候GDB加载到内存大概12.8GB，而2.4版本编译完后有12GB，加载到内存需要19GB。
1.15版本和2.4版本的核心实现原理是一致的。这儿使用Ubuntu 20.04版本，GDB 9.1版本。

git clone https://github.com/tensorflow/tensorflow.git git checkout -b r1.15 remotes/origin/r1.15 cd tensorflow

按照前置需要软件包

sudo apt install python3-dev python3-pip pip install -U --user pip six 'numpy<1.19.0' wheel setuptools mock 'future>=0.17.1' 'gast==0.3.3' typing_extensions pip install -U --user keras_applications --no-deps pip install -U --user keras_preprocessing --no-deps

配置TensorFlow

配置的时候能不编译的组件都选择N取消，调试核心框架不需要那些。

./configure

安装编译工具：Bazel

参考https://docs.bazel.build/versions/0.26.0/install-ubuntu.html进行安装。也可点击bazel-0.26.1下载。

sudo chmod +x bazel-0.26.1-linux-x86_64 ./bazel-0.26.1-linux-x86_64 sudo ln -s bazel-0.26.1-linux-x86_64 /usr/bin/bazel

编译TensorFlow

Build TensorFlow的debug版本，即可以用GDB Debug的版本：
Build的时候加上”–copt -g“:
通过查看任何一个.o文件来确认是否带有debug信息。例如gdb conv_ops_3d.pic.o验证，只要不出现No symbols则说明debug信息有。

[root@localhost conv_ops]# gdb conv_ops_3d.pic.o GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /root/.cache/bazel/_bazel_root/e88dc1dcc3c90dfdeee7304faf39c313/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/core/kernels/_objs/conv_ops/conv_ops_3d.pic.o...done. (gdb)

编译之前需要应用一个patch避免编译错误：

@@ -0,0 +1,78 @@ From 57586a1ca7f17b1916aed3dea4ff8de872dbf853 Mon Sep 17 00:00:00 2001 From: Benjamin Peterson Date: Fri, 3 May 2019 08:11:00 -0700 Subject: [PATCH] Rename gettid() functions.   glibc 2.30 will declare its own gettid; see https://sourceware.org/git/?p=glibc.git;a=commit;h=1d0fc213824eaa2a8f8c4385daaa698ee8fb7c92. Rename the grpc versions to avoid naming conflicts. --- src/core/lib/gpr/log_linux.cc | 6 ++---- src/core/lib/gpr/log_posix.cc | 4 ++-- src/core/lib/iomgr/ev_epollex_linux.cc | 4 ++-- 3 files changed, 6 insertions(+), 8 deletions(-)   diff --git a/src/core/lib/gpr/log_linux.cc b/src/core/lib/gpr/log_linux.cc index 81026e5689b..8b597b4cf2f 100644 --- a/src/core/lib/gpr/log_linux.cc +++ b/src/core/lib/gpr/log_linux.cc @@ -40,7 +40,7 @@ #include #include   -static long gettid(void) { return syscall(__NR_gettid); } +static long sys_gettid(void) { return syscall(__NR_gettid); }   void gpr_log(const char* file, int line, gpr_log_severity severity, const char* format, ...) { @@ -70,7 +70,7 @@ void gpr_default_log(gpr_log_func_args* args) { gpr_timespec now = gpr_now(GPR_CLOCK_REALTIME); struct tm tm; static __thread long tid = 0; - if (tid == 0) tid = gettid(); + if (tid == 0) tid = sys_gettid();   timer = static_cast(now.tv_sec); final_slash = strrchr(args->file, '/'); diff --git a/src/core/lib/gpr/log_posix.cc b/src/core/lib/gpr/log_posix.cc index b6edc14ab6b..2f7c6ce3760 100644 --- a/src/core/lib/gpr/log_posix.cc +++ b/src/core/lib/gpr/log_posix.cc @@ -31,7 +31,7 @@ #include #include   -static intptr_t gettid(void) { return (intptr_t)pthread_self(); } +static intptr_t sys_gettid(void) { return (intptr_t)pthread_self(); }   void gpr_log(const char* file, int line, gpr_log_severity severity, const char* format, ...) { @@ -86,7 +86,7 @@ void gpr_default_log(gpr_log_func_args* args) { char* prefix; gpr_asprintf(&prefix, "%s%s.%09d %7" PRIdPTR " %s:%d]", gpr_log_severity_string(args->severity), time_buffer, - (int)(now.tv_nsec), gettid(), display_file, args->line); + (int)(now.tv_nsec), sys_gettid(), display_file, args->line);   fprintf(stderr, "%-70s %s\n", prefix, args->message); gpr_free(prefix); diff --git a/src/core/lib/iomgr/ev_epollex_linux.cc b/src/core/lib/iomgr/ev_epollex_linux.cc index c2d80c08ddb..4a83cb6c215 100644 --- a/src/core/lib/iomgr/ev_epollex_linux.cc +++ b/src/core/lib/iomgr/ev_epollex_linux.cc @@ -1077,7 +1077,7 @@ static void end_worker(grpc_pollset* pollset, grpc_pollset_worker* worker, }   #ifndef NDEBUG -static long gettid(void) { return syscall(__NR_gettid); } +static long sys_gettid(void) { return syscall(__NR_gettid); } #endif   /* pollset->mu lock must be held by the caller before calling this. @@ -1097,7 +1097,7 @@ static grpc_error* pollset_work(grpc_pollset* pollset, #define WORKER_PTR (&worker) #endif #ifndef NDEBUG - WORKER_PTR->originator = gettid(); + WORKER_PTR->originator = sys_gettid(); #endif if (GRPC_TRACE_FLAG_ENABLED(grpc_polling_trace)) { gpr_log(GPR_INFO,

使用如下的命令进行编译：

bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/tools/lib_package:libtensorflow --verbose_failures   #Other libraries may need for debugging. bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/cc:cc_ops --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/c:c_test_util --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/c:c_api --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/cc:const_op --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/cc:array_ops --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/cc:ops --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/cc:match_ops --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/cc:math_ops --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/cc:scope --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/core:test --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/core:testlib --verbose_failures bazel build -s --config=opt --copt -g --config=noaws --config=nogcp --config=nohdfs --config=noignite --config=nokafka --config=nonccl //tensorflow/core/kernels:ops_testutil --verbose_failures

关于debug level的解释：

-glevel Request debugging information and also use level to specify how much information. The default level is 2. Level 0 produces no debug information at all. Thus, -g0 negates -g. Level 1 produces minimal information, enough for making backtraces in parts of the program that you don't plan to debug. This includes descriptions of functions and external variables, but no information about local variables and no line numbers. Level 3 includes extra information, such as all the macro definitions present in the program. Some debuggers support macro expansion when you use -g3.