Ubuntu 16.04环境下Configuring the Arm NN SDK build environment for Caffe
1. 安装SCons
安装之前确保有python环境，Ubuntu16.04自带的是Python2.7（路径/usr/bin/python）。目前Scons最新版本3.0.1对于Python3的支持不好，不建议使用Python3。如果尝试了Python3后续编译Compute Library的时候出错则需要卸载Scons3.0.1，重新安装Scons2.4.1。卸载的方法是：
rm /usr/local/bin/scons*
rm -r /usr/local/lib/scons*Scons建议使用官方教程推荐的版本SCons 2.4.1，下载之后解压，进入解压之后的文件夹scons-2.4.1，打开终端，运行sudo python setup.py install，成功安装之后如下图1所示：
2. 安装CMake
官网下载cmake-3.11.1-Linux-x86_64.sh，然后直接sh cmake-3.11.1-Linux-x86_64.sh即可。
3. Download and build Boost
Boost官网下载，建议使用官方推荐的版本1.64.0版本，下载未编译过的版本。下载完解压之后的文件夹为boost_1_64_0，进入该文件夹，运行：sh bootstrap.sh，运行结果如图2所示：
此时需要对project-config.jam进行修改，将
if ! gcc in [ feature.values <toolset> ]
{
using gcc ;
}改为
if ! gcc in [ feature.values <toolset> ]
{
using gcc : arm : arm-linux-gnueabihf-gcc ;
}接着运行：./b2 link=static cxxflags=-fPIC --with-filesystem --with-test --with-log --with-program_options。注意观察运行界面，查看界面上显示的编译位数是否和工具链的位数一致，如果不一致则注意空格情况，再者指定arm-linux-gnueabihf-gcc的绝对路径尝试。比如用的是32位arm平台上的工具链，编译的结果应该类似图3显示的- 32-bit :yes、- arm : yes。如果用的是64位arm平台上的工具链，编译的结果应该显示- 32-bit : no、- 64-bit : yes、- arm : yes。如图3、4所示：
4. 安装caffe
Build Caffe by following the step-by-step Caffe instructions.
需要注意的是用哪个编译器编译caffe是无所谓的，因为只会用到编译caffe后的文件caffe.pb.cc和caffe.pb.h。需要注意的是编译caffe所用的protobuf要和步骤6中交叉编译的protobuf版本一致。可以采用的方法是先用linux自带的gcc、g++编译器编译步骤6中的protobuf（当然此时不叫交叉编译了），然后在编译caffe，此时caffe用的protobuf版本就是和步骤6中版本一致。然后再重新用步骤6中的方法重新编译protobuf即交叉编译protobuf。
5. Compile the Compute Library using SCons
对于ARMv7-A架构的设备运行下面的指令进行编译：
scons extra_cxx_flags="-fPIC" benchmark_tests=1 validation_tests=1 neon=1 -j8
编译期间可以去喝杯咖啡……
编译所需的其它参数请参考ComputeLibrary/SConstruct文件：
vars.AddVariables(
BoolVariable("debug", "Debug", False),
BoolVariable("asserts", "Enable asserts (this flag is forced to 1 for debug=1)", False),
BoolVariable("logging", "Logging (this flag is forced to 1 for debug=1)", False),
EnumVariable("arch", "Target Architecture", "armv7a", allowed_values=("armv7a", "arm64-v8a", "arm64-v8.2-a", "x86_32", "x86_64")),
EnumVariable("os", "Target OS", "linux", allowed_values=("linux", "android", "bare_metal")),
EnumVariable("build", "Build type", "cross_compile", allowed_values=("native", "cross_compile", "embed_only")),
BoolVariable("examples", "Build example programs", True),
BoolVariable("Werror", "Enable/disable the -Werror compilation flag", True),
BoolVariable("standalone", "Builds the tests as standalone executables, links statically with libgcc, libstdc++ and libarm_compute", False),
BoolVariable("opencl", "Enable OpenCL support", True),
BoolVariable("neon", "Enable Neon support", False),
BoolVariable("gles_compute", "Enable OpenGL ES Compute Shader support", False),
BoolVariable("embed_kernels", "Embed OpenCL kernels and OpenGL ES compute shaders in library binary", True),
BoolVariable("set_soname", "Set the library's soname and shlibversion (requires SCons 2.4 or above)", False),
BoolVariable("openmp", "Enable OpenMP backend", False),
BoolVariable("cppthreads", "Enable C++11 threads backend", True),
PathVariable("build_dir", "Specify sub-folder for the build", ".", PathVariable.PathAccept),
("extra_cxx_flags", "Extra CXX flags to be appended to the build command", "")
)编译成功的结果如图5所示：
如果编译提示没有arm-linux-gnueabihf-g++则说明开发环境没有安装交叉编译环境，需要进行安装，参考：https://www.acmesystems.it/arm9_toolchain
如果在特定相机上开发，比如海思的相机，可能需要用到arm-hisiv500-linux-gcc、arm-hisiv500-linux-g++或arm-hisiv600-linux-gcc、arm-hisiv600-linux-g++。对于ARMv7-A架构的芯片用arm-hisiv500-linux-会报错：
At global scope:
cc1plus: error: unrecognized command line option "-Wno-implicit-fallthrough" [-Werror]
cc1plus: all warnings being treated as errors
scons: *** [build/src/core/AccessWindowAutoPadding.o] Error 1
scons: building terminated because of errors.因此需要用arm-hisiv600-linux-，需要修改ComputeLibrary-master/SConstruct：
1.由：
if env['arch'] == 'armv7a':
env.Append(CXXFLAGS = ['-march=armv7-a', '-mthumb', '-mfpu=neon'])
if env['os'] == 'linux':
prefix = "arm-linux-gnueabihf-"
env.Append(CXXFLAGS = ['-mfloat-abi=hard'])修改为：
if env['arch'] == 'armv7a':
env.Append(CXXFLAGS = ['-march=armv7-a', '-mthumb', '-mfpu=neon'])
if env['os'] == 'linux':
prefix = "arm-hisiv600-linux-"
env.Append(CXXFLAGS = ['-mfloat-abi=softfp'])2.由：
BoolVariable("cppthreads", "Enable C++11 threads backend", True),修改为：
BoolVariable("cppthreads", "Enable C++11 threads backend", False),即禁用OpenMP，无法用多线程加速。
而对于ARMv8-A需要在对应的分支下做相应的修改。
6. 交叉编译protobuf
cross complile protobuf可参考Protocol Buffers - Google’s data interchange format。
先到protobuf官网下载所需版本编译好的protoc和源码protobuf。本人使用的是protobuf-all-3.5.1.tar.gz和protoc-3.5.1-linux-x86_64.zip。
首先运行sh autogen.sh，如果报如下的错误：autogen.sh: 50: autogen.sh: autoreconf: not found
解决方法是：sudo apt install autoconf autogen。
如果报如下的错误：
Can't exec "libtoolize": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 345, <GEN11> line 6.
autoreconf: failed to run libtoolize: No such file or directory
autoreconf: libtoolize is needed because this package uses Libtool解决方法是：sudo apt install libtool。
正确运行sh autogen.sh后的界面如图6所示：
然后根据不同架构的芯片运行相应指令。
对于ARMv7-A架构：
./configure --host=arm-linux \
CC=arm-linux-gnueabihf-gcc \
CXX=arm-linux-gnueabihf-g++ \
--with-protoc=/home/lg/protoc-3.5.1-linux-x86_64/bin/protoc或者对于ARMv8-A架构：
./configure --host=aarch64-linux \
CC=aarch64-linux-gnu-gcc \
CXX=aarch64-linux-gnu-g++ \
--with-protoc=/home/lg/protoc-3.5.1-linux-x86_64/bin/protoc运行成功的界面如图7所示：
然后make -j32即可，make成功之后的界面如图8所示：
make check应该会出现错误，因为check应该是在arm平台才能运行成功，此时是在ubuntu下编译的所以会出错，错误结果如图9所示：
如果在编译protobuf时运行./configure不加任何参数，即编译在ubuntu下用的protoc则make check是不会报错的。
make check之后再运行make install即可。值得一提的是如果是因为普通用户没有root权限导致的make install没有成功(注意不是由于没配置好导致的无法make install 成功)，也是有办法调用步骤7中编译好的armnn的库的，详见Deploying a Caffe model using the Arm NN SDK中的相关部分。注意，有没有make install成功不会对步骤7中编译armnn有影响，因为make install是安装到系统的/usr/local/lib目录下，而步骤7中已经明确指定了-DPROTOBUF_LIBRARY_RELEASE和-DPROTOBUF_LIBRARY_DEBUG了。
如果想更换工具链重新编译需要make distclean彻底清除之前工具链的编译记录再用新的工具链编译。
交叉编译protobuf还可能遇到其它问题，例如同样是由于缺少libtool而导致运行sh autogen.sh失败但报不同的错误，例如报错可能是main::scan_file() called too early to check prototype at（报的错误我只保存了部分）。此时还是需要安装libtool。如果没有条件使用sudo apt install autoconf autogen那么只有手动撸了（不要问我为什么没有条件使用sudo apt install，此处只有千千万万匹草泥马在崩腾）。此时可能需要安装三个文件automake、autoconf、libtool。各自tar包的下载地址automake、autoconf、libtool。三者之间版本有一定的搭配需求，版本不和谐也会出错，笔者用的版本是automake-1.14、autoconf-2.65、libtool-2.4.6。需要特别注意的是不要在windows下解压在传到linux下，一定要在Linux下tar xvzf进行解压，否则后续安装不成功。这三个软件的安装方法都是一样，依次执行以下指令：./configure、make、make install。注意加sudo权限。
如果安装libtool的时候报如下错误，则尝试从普通用户切换为su进行安装（如果执行su的时候出现su: 鉴定故障，尝试sudo su）：
/home/lg/protobuf-3.5.1/libtool: line 10545: aarch64-linux-gnu-g++: command not found
libtool: error: error: relink 'libprotoc.la' with the above command before installing it
make[3]: *** [install-libLTLIBRARIES] 错误 1
make[3]: 离开目录“/home/lg/protobuf-3.5.1/src”
make[2]: *** [install-am] 错误 2
make[2]: 离开目录“/home/lg/protobuf-3.5.1/src”
make[1]: *** [install] 错误 2
make[1]: 离开目录“/home/lg/protobuf-3.5.1/src”
make: *** [install-recursive] 错误 17. 编译armnn
在armnn-master中的CMakeLists.txt中添加
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -Werror -Wold-style-cast -Wno-missing-braces -Wconversion -Wsign-conversion -pthread")
# this is required
SET(CMAKE_SYSTEM_NAME Linux)
#set(CMAKE_SYSTEM_PROCESSOR arm) #此句可有可无
# specify the cross compiler
SET(CMAKE_C_COMPILER /usr/bin/arm-linux-gnueabihf-gcc)
SET(CMAKE_CXX_COMPILER /usr/bin/arm-linux-gnueabihf-g++)
# configure Boost
SET(BOOST_ROOT /home/garylau/boost_1_64_0)然后
cmake -DARMCOMPUTE_ROOT=/home/lg/armnnSDK/ComputeLibrary \
-DARMCOMPUTE_BUILD_DIR=/home/lg/armnnSDK/ComputeLibrary/build \
-DBOOST_ROOT=/home/lg/armnnSDK/boost_1_64_0 \
-DCAFFE_GENERATED_SOURCES=/home/lg/armnnSDK/caffe/build/src \
-DBUILD_CAFFE_PARSER=1 \
-DARMCOMPUTENEON=1 \
-DPROTOBUF_LIBRARY_RELEASE=/home/lg/armnnSDK/protobuf-3.5.1/src/.libs/libprotobuf.so \
-DPROTOBUF_LIBRARY_DEBUG=/home/lg/armnnSDK/protobuf-3.5.1/src/.libs/libprotobuf.so接着 make -j32 即可看到编译、链接成功的界面，如图10、11所示：
如果编译的时候出现如下(即图12)的问题，很可能是由于步骤4中编译caffe所用的protobuf版本和步骤6中交叉编译protobuf所用的protobuf所用版本不统一导致的，解决方法是在编译protobuf的make install 操作之后重新编译caffe。
In file included from /home/lg/armnnSDK/caffe/build/src/caffe/proto/caffe.pb.cc:5:0:
/home/lg/armnnSDK/caffe/build/src/caffe/proto/caffe.pb.h:17:2: error: #error This file was generated by an older version of protoc which is
#error This file was generated by an older version of protoc which is
^
/home/lg/armnnSDK/caffe/build/src/caffe/proto/caffe.pb.h:18:2: error: #error incompatible with your Protocol Buffer headers. Please
#error incompatible with your Protocol Buffer headers. Please
^
/home/lg/armnnSDK/caffe/build/src/caffe/proto/caffe.pb.h:19:2: error: #error regenerate this file with a newer version of protoc.
#error regenerate this file with a newer version of protoc.
^
In file included from /home/lg/armnnSDK/armnn/src/armnnCaffeParser/CaffeParser.cpp:20:0:
/home/lg/armnnSDK/caffe/build/src/caffe/proto/caffe.pb.h:17:2: error: #error This file was generated by an older version of protoc which is
#error This file was generated by an older version of protoc which is
^
/home/lg/armnnSDK/caffe/build/src/caffe/proto/caffe.pb.h:18:2: error: #error incompatible with your Protocol Buffer headers. Please
#error incompatible with your Protocol Buffer headers. Please
^
/home/lg/armnnSDK/caffe/build/src/caffe/proto/caffe.pb.h:19:2: error: #error regenerate this file with a newer version of protoc.
#error regenerate this file with a newer version of protoc.
如果编译的时候出现类似下面的问题，是由于cmake版本偏低的原因，所以步骤2中安装cmake的时候最好安装官方推荐版本：
CMake Error at cmake/modules/FindBoost.cmake:1213 (string):
string does not recognize sub-command APPEND
Call Stack (most recent call first):
cmake/GlobalConfig.cmake:98 (find_package)
CMakeLists.txt:24 (include)
CMake Error at cmake/modules/FindBoost.cmake:1342 (string):
string does not recognize sub-command APPEND
Call Stack (most recent call first):
cmake/GlobalConfig.cmake:98 (find_package)
CMakeLists.txt:24 (include)
-- Boost version: 1.64.0
-- Found the following Boost libraries:
-- unit_test_framework
-- system
-- filesystem
-- log
-- program_options
-- date_time
-- log_setup
-- thread
-- regex
-- chrono
-- atomic
CMake Error at CMakeLists.txt:546 (CHECK_INCLUDE_FILE):
Unknown CMake command "CHECK_INCLUDE_FILE".
-- Configuring incomplete, errors occurred!
See also "/home/lg/armnnSDK/armnn/CMakeFiles/CMakeOutput.log".
See also "/home/lg/armnnSDK/armnn/CMakeFiles/CMakeError.log".如果编译的时候报如下的错误，则是由于步骤6中编译protobuf出了问题，重新编译，直到不报任何错误。
libarmnnCaffeParser.so: undefined reference to `google::protobuf::Arena::AddListNode(void*, void (*)(void*))'
libarmnnCaffeParser.so: undefined reference to `google::protobuf::io::CodedInputStream::Skip(int)'
libarmnnCaffeParser.so: undefined reference to `google::protobuf::Arena::AllocateAligned(std::type_info const*, unsigned long)如果编译的时候报如下的错误，则可能是由于步骤7中之前cmake指令生成的旧的CMakeCache.txt是错误的，在后续执行cmake指令时会受影响，解决方法是将删除，重新执行cmake指令，生成正确的CMakeCache.txt即可。
[100%] Linking CXX executable UnitTests
libarmnn.so: undefined reference to `arm_compute::NEActivationLayer::configure(arm_compute::ITensor*, arm_compute::ITensor*, arm_compute::ActivationLayerInfo)'
libarmnn.so: undefined reference to `arm_compute::NEDepthwiseConvolutionLayer3x3::configure(arm_compute::ITensor*, arm_compute::ITensor const*, arm_compute::ITensor const*, a
rm_compute::ITensor*, arm_compute::PadStrideInfo const&)'
libarmnn.so: undefined reference to `arm_compute::INESimpleFunction::run()'
libarmnn.so: undefined reference to `typeinfo for arm_compute::INESimpleFunction'
libarmnn.so: undefined reference to `arm_compute::NEPoolingLayer::run()'
libarmnn.so: undefined reference to `arm_compute::NEBatchNormalizationLayer::configure(arm_compute::ITensor*, arm_compute::ITensor*, arm_compute::ITensor const*, arm_compute:
:ITensor const*, arm_compute::ITensor const*, arm_compute::ITensor const*, float, arm_compute::ActivationLayerInfo)'
libarmnn.so: undefined reference to `arm_compute::NENormalizationLayer::configure(arm_compute::ITensor const*, arm_compute::ITensor*, arm_compute::NormalizationLayerInfo cons
t&)'
libarmnn.so: undefined reference to `vtable for arm_compute::NEFullyConnectedLayerReshapeWeights'
libarmnn.so: undefined reference to `arm_compute::NESoftmaxLayer::NESoftmaxLayer(std::shared_ptr<arm_compute::IMemoryManager>)'
libarmnn.so: undefined reference to `vtable for arm_compute::INESimpleFunction'
libarmnn.so: undefined reference to `vtable for arm_compute::NEFullyConnectedLayer'
libarmnn.so: undefined reference to `arm_compute::NEDepthwiseConvolutionLayer::configure(arm_compute::ITensor*, arm_compute::ITensor const*, arm_compute::ITensor const*, arm_
compute::ITensor*, arm_compute::PadStrideInfo const&)'
libarmnn.so: undefined reference to `arm_compute::NENormalizationLayer::NENormalizationLayer(std::shared_ptr<arm_compute::IMemoryManager>)'
libarmnn.so: undefined reference to `arm_compute::NEDirectConvolutionLayer::configure(arm_compute::ITensor*, arm_compute::ITensor const*, arm_compute::ITensor const*, arm_com
pute::ITensor*, arm_compute::PadStrideInfo const&)'
libarmnn.so: undefined reference to `arm_compute::INESimpleFunction::INESimpleFunction()'
libarmnn.so: undefined reference to `arm_compute::NEFullyConnectedLayer::configure(arm_compute::ITensor const*, arm_compute::ITensor const*, arm_compute::ITensor const*, arm_
compute::ITensor*, bool, bool)'
libarmnn.so: undefined reference to `arm_compute::NEPermute::validate(arm_compute::ITensorInfo const*, arm_compute::ITensorInfo const*, arm_compute::Strides const&)'
libarmnn.so: undefined reference to `arm_compute::NEFullyConnectedLayer::run()'
libarmnn.so: undefined reference to `arm_compute::NEConvolutionLayer::configure(arm_compute::ITensor*, arm_compute::ITensor const*, arm_compute::ITensor const*, arm_compute::
ITensor*, arm_compute::PadStrideInfo const&, arm_compute::WeightsInfo const&)'
libarmnn.so: undefined reference to `arm_compute::NEPermute::configure(arm_compute::ITensor const*, arm_compute::ITensor*, arm_compute::Strides const&)'
libarmnn.so: undefined reference to `arm_compute::NEBatchNormalizationLayer::NEBatchNormalizationLayer()'
libarmnn.so: undefined reference to `arm_compute::NEConvolutionLayer::NEConvolutionLayer(std::shared_ptr<arm_compute::IMemoryManager>)'
libarmnn.so: undefined reference to `arm_compute::NEBatchNormalizationLayer::run()'
libarmnn.so: undefined reference to `arm_compute::NEDepthwiseConvolutionLayer3x3::NEDepthwiseConvolutionLayer3x3()'
libarmnn.so: undefined reference to `arm_compute::NEPoolingLayer::validate(arm_compute::ITensorInfo const*, arm_compute::ITensorInfo const*, arm_compute::PoolingLayerInfo con
st&)'
libarmnn.so: undefined reference to `arm_compute::NEPixelWiseMultiplication::configure(arm_compute::ITensor const*, arm_compute::ITensor const*, arm_compute::ITensor*, float,
arm_compute::ConvertPolicy, arm_compute::RoundingPolicy)'
libarmnn.so: undefined reference to `arm_compute::NEFullyConnectedLayer::NEFullyConnectedLayer(std::shared_ptr<arm_compute::IMemoryManager>)'
libarmnn.so: undefined reference to `arm_compute::NEDepthwiseConvolutionLayer::NEDepthwiseConvolutionLayer()'
libarmnn.so: undefined reference to `arm_compute::NEDirectConvolutionLayer::NEDirectConvolutionLayer(std::shared_ptr<arm_compute::IMemoryManager>)'
libarmnn.so: undefined reference to `vtable for arm_compute::NENormalizationLayer'
libarmnn.so: undefined reference to `arm_compute::NEArithmeticAddition::configure(arm_compute::ITensor*, arm_compute::ITensor*, arm_compute::ITensor*, arm_compute::ConvertPol
icy)'
libarmnn.so: undefined reference to `arm_compute::NEFloor::configure(arm_compute::ITensor const*, arm_compute::ITensor*)'
libarmnn.so: undefined reference to `arm_compute::NEPoolingLayer::NEPoolingLayer()'
libarmnn.so: undefined reference to `arm_compute::NESoftmaxLayer::run()'
libarmnn.so: undefined reference to `arm_compute::NEReshapeLayer::configure(arm_compute::ITensor const*, arm_compute::ITensor*)'
libarmnn.so: undefined reference to `arm_compute::NEPoolingLayer::configure(arm_compute::ITensor*, arm_compute::ITensor*, arm_compute::PoolingLayerInfo const&)'
libarmnn.so: undefined reference to `arm_compute::NENormalizationLayer::run()'
libarmnn.so: undefined reference to `arm_compute::NESoftmaxLayer::configure(arm_compute::ITensor*, arm_compute::ITensor*, float)'
libarmnn.so: undefined reference to `vtable for arm_compute::NESoftmaxLayer'
collect2: error: ld returned 1 exit status
make[2]: *** [UnitTests] 错误 1
make[1]: *** [CMakeFiles/UnitTests.dir/all] 错误 2
make: *** [all] 错误 28. arm-hisiv500-linux安装
Hi3519V101_SDK_V1.0.3.0.tgz下载后解压文件夹为Hi3519V101_SDK_V1.0.3.0，然后运行sudo ./sdk.unpack，接着进入Hi3519V101_SDK_V1.0.3.0/osdrv/opensource/toolchain/arm-hisiv500-linux，然后增加执行权限chmod +x arm-hisiv500-linux.install。执行sudo ./arm-hisiv500-linux.install进行安装。默认安装路径为新生成的目录/opt下，编译器在/opt/hisi-linux/x86-arm/arm-hisiv500-linux/target/bin下。安装后加环境变量#Hisilicon Linux, Cross-Toolchain PATHexport PATH="/opt/hisi-linux/x86-arm/arm-hisiv500-linux/target/bin:$PATH"，然后sudo source /etc/profile即可使用arm-hisiv500-linux-gcc、arm-hisiv500-linux-g++。
336
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
