2_TensorFlow1.31.1 GPU版本源码编译【报错备忘】

最新推荐文章于 2023-07-30 12:48:26 发布

hardwork617s

最新推荐文章于 2023-07-30 12:48:26 发布

阅读量1.9k

点赞数 1

本文链接：https://blog.csdn.net/hardwork617s/article/details/112454889

版权

本文记录了在编译TensorFlow 1.31.1 GPU版本过程中遇到的错误及解决方案，包括：1. 由于CUDA_NVCC_FLAGS参数重复导致的'nvcc fatal : redefinition of argument 'std''错误；2. unistd_ext.h中gettid函数定义冲突的问题；3. ICU库下载校验码错误；4. 与llvm库相关的编译错误。通过调整编译选项和修复源文件，最终成功编译。

摘要由CSDN通过智能技术生成

2_TensorFlow1.31.1 GPU版本源码编译【报错备忘】

下面的是按照从现在往过去的时间轴写的，即，编译出错，是从文本内容的后面往前看的顺序。

★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★
终于编译成功了！编译日志见《TensorFlow1.31.1 GPU版本【编译成功日志】.log》
【gcc4.8.4】使用命令【bazel build --copt=-march=native --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package --verbose_failures】
编译TensorFlow1.31.1 GPU版本成功：
Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 9937.403s, Critical Path: 386.06s
INFO: 15033 processes: 15033 local.
INFO: Build completed successfully, 18677 total actions
root@longhk:/tensorflow-1.13.1#
注意：
bazel build 【这个要去掉，否则编译报错： --cxxopt=-std=c++11 】 --copt=-march=native --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package --verbose_failures
即使用命令：
bazel build --copt=-march=native --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package --verbose_failures

★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★
第④次报错：
ERROR: /tensorflow-1.13.1/tensorflow/contrib/image/BUILD:115:1: error while parsing .d file: /root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/contrib/image/_objs/python/ops/_distort_image_ops_gpu/adjust_hsv_in_yiq_op_gpu.cu.pic.d (No such file or directory)
nvcc fatal : redefinition of argument 'std'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2263.780s, Critical Path: 326.33s
INFO: 5372 processes: 5372 local.
FAILED: Build did NOT complete successfully
root@longhk:/tensorflow-1.13.1#
【怀疑nvcc fatal : redefinition of argument 'std'】错误原因是：
【CUDA_NVCC_FLAGS的编译参数中重复出现-std=c++11】
参考《nvcc fatal : redefinition of argument 'std'》
https://blog.csdn.net/qq_38156052/article/details/90270352

★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★
第③次报错：【unistd_ext.h:34:16: note: old declaration '__pid_t gettid()'】
参考： https://patchwork.kernel.org/project/qemu-devel/patch/20190320123625.9207-1-berrange@redhat.com/
需要对源文件【/root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/external/grpc/src/core/lib/gpr/log_linux.cc】
的【static long gettid(void) 】函数【返回值类型】进行修改：
【static int sys_gettid(void) 】

ERROR: /root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/external/grpc/BUILD:502:1: C++ compilation of rule '@grpc//:gpr_base' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
(cd /root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/execroot/org_tensorflow && \
exec env - \
CUDA_TOOLKIT_PATH=/usr/local/cuda \
CUDNN_INSTALL_PATH=/usr/local/cuda-10.0 \
GCC_HOST_COMPILER_PATH=/usr/bin/gcc \
LD_LIBRARY_PATH=/usr/local/cuda/lib64 \
NCCL_HDR_PATH=/usr/local/cuda-10.0/lib64/../include \
NCCL_INSTALL_PATH=/usr/local/cuda-10.0/lib64 \
PATH=/bin:/usr/bin \
PWD=/proc/self/cwd \
PYTHON_BIN_PATH=/usr/local/bin/python3 \
PYTHON_LIB_PATH=/usr/local/lib/python3.6/site-packages \
TF_CUDA_CLANG=0 \
TF_CUDA_COMPUTE_CAPABILITIES=3.0 \
TF_CUDA_VERSION=10.0 \
TF_CUDNN_VERSION=7 \
TF_NCCL_VERSION=2 \
TF_NEED_CUDA=1 \
TF_NEED_OPENCL_SYCL=0 \
TF_NEED_ROCM=0 \
external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/k8-opt/bin/external/grpc/_objs/gpr_base/log_linux.pic.d '-frandom-seed=bazel-out/k8-opt/bin/external/grpc/_objs/gpr_base/log_linux.pic.o' '-DGRPC_ARES=0' -iquote external/grpc -iquote bazel-out/k8-opt/genfiles/external/grpc -iquote bazel-out/k8-opt/bin/external/grpc -isystem external/grpc/include -isystem bazel-out/k8-opt/genfiles/external/grpc/include -isystem bazel-out/k8-opt/bin/external/grpc/include '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIC -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -fno-canonical-system-headers -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections '-march=native' '-march=native' -Wno-sign-compare '-std=c++11' -c external/grpc/src/core/lib/gpr/log_linux.cc -o bazel-out/k8-opt/bin/external/grpc/_objs/gpr_base/log_linux.pic.o)
Execution platform: @bazel_tools//platforms:host_platform
external/grpc/src/core/lib/gpr/log_linux.cc: In function 'long int gettid()':
external/grpc/src/core/lib/gpr/log_linux.cc:43:24: error: ambiguating new declaration of 'long int gettid()'
static long gettid(void) { return syscall(__NR_gettid); }
^
In file included from /usr/include/unistd.h:1170:0,
from external/grpc/src/core/lib/gpr/log_linux.cc:41:
/usr/include/x86_64-linux-gnu/bits/unistd_ext.h:34:16: note: old declaration '__pid_t gettid()'
extern __pid_t gettid (void) __THROW;
^
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 61.355s, Critical Path: 23.85s
INFO: 438 processes: 438 local.
FAILED: Build did NOT complete successfully

★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★
第②次主要报错【校验码不对！】：/root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/external/icu/release-62-1.tar.gz

root@longhk:/tensorflow-1.13.1# bazel clean --expunge
INFO: Invocation ID: 90a95350-710a-433d-ae61-d47060caabb8
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
root@longhk:/tensorflow-1.13.1# bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package --verbose_failures
WARNING: ignoring _JAVA_OPTIONS in environment.
Starting local Bazel server and connecting to it...
WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
INFO: Invocation ID: 7ec85139-6760-404f-9a33-0cb0ba50dde7
DEBUG: /root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: 'TMP' environment variable is not set, using 'C:\Windows\Temp' as default
ERROR: /tensorflow-1.13.1/tensorflow/tools/pip_package/BUILD:149:1: no such package '@icu//': java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/unicode-org/icu/archive/release-62-1.tar.gz, https://github.com/unicode-org/icu/archive/release-62-1.tar.gz] to /root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/external/icu/release-62-1.tar.gz: Checksum was 86b85fbf1b251d7a658de86ce5a0c8f34151027cc60b01e1b76f167379acf181 but wanted e15ffd84606323cbad5515bf9ecdf8061cc3bf80fb883b9e6aa162e485aa9761 and referenced by '//tensorflow/tools/pip_package:licenses'
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: no such package '@icu//': java.io.IOException: Error downloading [https://mirror.bazel.build/github.com/unicode-org/icu/archive/release-62-1.tar.gz, https://github.com/unicode-org/icu/archive/release-62-1.tar.gz] to /root/.cache/bazel/_bazel_root/3f83cabb457beddc67f1cf84eff7f8a3/external/icu/release-62-1.tar.gz: Checksum was 86b85fbf1b251d7a658de86ce5a0c8f34151027cc60b01e1b76f167379acf181 but wanted e15ffd84606323cbad5515bf9ecdf8061cc3bf80fb883b9e6aa162e485aa9761
INFO: Elapsed time: 516.515s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (360 packages loaded, 21744 targets configured)
Fetching @icu; fetching 506s

★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★
第①次主要报错

root@longhk:/tensorflow-1.13.1# bazel version
WARNING: ignoring _JAVA_OPTIONS in environment.
Starting local Bazel server and connecting to it...
INFO: Invocation ID: fa0a7c15-736e-41e6-9ab9-d67aaba4dc16
Build label: 0.21.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Dec 19 12:58:44 2018 (1545224324)
Build timestamp: 1545224324
Build timestamp as int: 1545224324
root@longhk:/tensorflow-1.13.1# bazel clean
INFO: Invocation ID: ee473e7d-4284-402e-a430-8a9ae2d5dde3
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
root@longhk:/tensorflow-1.13.1# bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package --verbose_failures
WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
INFO: Invocation ID: 879c1cdb-fbc4-4479-b7bf-a1d4ad9df50b
WARNING: /tensorflow-1.13.1/tensorflow/python/BUILD:2986:1: in py_library rule //tensorflow/python:standard_ops: target '//tensorflow/python:standard_ops' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/python/BUILD:77:1: in py_library rule //tensorflow/python:no_contrib: target '//tensorflow/python:no_contrib' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/metrics/BUILD:16:1: in py_library rule //tensorflow/contrib/metrics:metrics_py: target '//tensorflow/contrib/metrics:metrics_py' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/gan/BUILD:136:1: in py_library rule //tensorflow/contrib/gan:losses_impl: target '//tensorflow/contrib/gan:losses_impl' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:exporter': No longer supported. Switch to SavedModel immediately.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/learn/BUILD:17:1: in py_library rule //tensorflow/contrib/learn:learn: target '//tensorflow/contrib/learn:learn' depends on deprecated target '//tensorflow/contrib/session_bundle:gc': No longer supported. Switch to SavedModel immediately.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/timeseries/python/timeseries/state_space_models/BUILD:233:1: in py_library rule //tensorflow/contrib/timeseries/python/timeseries/state_space_models:filtering_postprocessor: target '//tensorflow/contrib/timeseries/python/timeseries/state_space_models:filtering_postprocessor' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/timeseries/python/timeseries/BUILD:356:1: in py_library rule //tensorflow/contrib/timeseries/python/timeseries:ar_model: target '//tensorflow/contrib/timeseries/python/timeseries:ar_model' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/bayesflow/BUILD:17:1: in py_library rule //tensorflow/contrib/bayesflow:bayesflow_py: target '//tensorflow/contrib/bayesflow:bayesflow_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/timeseries/python/timeseries/state_space_models/BUILD:76:1: in py_library rule //tensorflow/contrib/timeseries/python/timeseries/state_space_models:kalman_filter: target '//tensorflow/contrib/timeseries/python/timeseries/state_space_models:kalman_filter' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/seq2seq/BUILD:23:1: in py_library rule //tensorflow/contrib/seq2seq:seq2seq_py: target '//tensorflow/contrib/seq2seq:seq2seq_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/seq2seq/BUILD:23:1: in py_library rule //tensorflow/contrib/seq2seq:seq2seq_py: target '//tensorflow/contrib/seq2seq:seq2seq_py' depends on deprecated target '//tensorflow/python/ops/distributions:distributions': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.distributions will not receive new features, and will be removed by early 2019. You should update all usage of `tf.distributions` to `tfp.distributions`.
WARNING: /tensorflow-1.13.1/tensorflow/contrib/BUILD:13:1: in py_library rule //tensorflow/contrib:contrib_py: target '//tensorflow/contrib:contrib_py' depends on deprecated target '//tensorflow/contrib/distributions:distributions_py': TensorFlow Distributions has migrated to TensorFlow Probability (https://github.com/tensorflow/probability). Deprecated copies remaining in tf.contrib.distributions are unmaintained, unsupported, and will be removed by late 2018. You should update all usage of `tf.contrib.distributions` to `tfp.distributions`.
INFO: Analysed target //tensorflow/tools/pip_package:build_pip_package (362 packages loaded, 23522 targets configured).
INFO: Found 1 target...
INFO: From Compiling external/snappy/snappy-sinksource.cc [for host]:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling external/snappy/snappy-stubs-internal.cc [for host]:
cc1plus: warning: command-line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
INFO: From Compiling ex