ros2_object_analytics安装过程全记录

ros2_object_analytics安装过程全记录

问题总结

安装ros2_object_analytics（下文简称OA）一共用了我个三个工作日时间，主要是自己对系统构架不够熟悉并且调试能力也不够，所以将自己遇到的问题和安装流程记录下来，方便后来参考，另外这些问题是我在安装时按照当前版本的readme遇到的问题，因为版本还会不不断更新，更新之后可能问题就不存在了。

（1）OpenCV3.3依赖问题

按照readme安装完OpenCV3.3编译OA的时候会遇到

/usr/bin/ld: cannot find -llz4

这应该OpenCV3.3的依赖问题，解决办法如下：

sudo apt-get install liblz4-dev

（2）devel版本中librealsense版本过低问题

devel版本中的readme安装realsense的驱动的方法是

sudo apt-get install ros-crystal-librealsense2 ros-crystal-realsense-ros2-camera

而在后面运行的OA的时候，master分支没有问题，但是devel分支的话会发现没有点云信息，后来发现ros-crystal-librealsense2中的librealsense的SDK版本没有更新是2.16.5版本的，这个版本中发布的点云节点是’/camera/depth/color/points’，而不是’/camera/aligned_depth_to_color/color/points’，而OA中master版本订阅（映射）的点云节点是’/camera/depth/color/points’，devel中变成了’/camera/aligned_depth_to_color/color/points’，因此按照上述命令安装realsense驱动的话只有master版本是好用的，而devel版本就没法用了。

我的解决办法是，先卸载上面的两个包

sudo apt-get remove ros-crystal-librealsense2 ros-crystal-realsense-ros2-camera

按照Linux Distribution中安装distribution版本的librealsense SDK2.0，安装2.17.1的或者最新的都可以，然后将ros-crystal-realsense-ros2-camera放到工作空间中和OA一起编译，之后就能解决上述没有点云的问题

（3）NCS版本不好用的问题

目前maste版本的OA通过NCS是可以成功运行的，但是devel版本，如果按照readme的方式一步一步安装的话，插入的硬件是NCS2的话会报错

[ERROR] [movidius_ncs_stream]: Error: there is no device at the given index or name

如果插入硬件NCS1的话会报错

[ERROR] [movidius_ncs_stream]: Error: an unexpected error was encontered during the function call

后来了解到OA之后可能不再支持NCS版本，因此我就没再话时间在调试NCS的问题上，转战OpenVINO.

安装过程记录

下面我记录一的是一台新电脑的OA从无到有的一个可行的安装步骤，主要是参照ros2_object_analytics
和ros2_openvino_toolkit的readme，中间会指出一些容易出问题的地方

（1）Install ROS2 desktop packages ros-crystal-desktop

参看ros-crystal-desktop，应该不会有什么问题

（2）Install ROS2 dependences

sudo apt-get install ros-crystal-cv-bridge ros-crystal-object-msgs ros-crystal-image-transport ros-crystal-realsense-camera-msgs

（3）Install librealsense SDK2

参考Linux Distribution,应该没什么问题

（4）Install NCS

因为我没有试没有不安转NCS会怎样，因此我还是按照我原来的步骤安装

# build ncsdk
mkdir ~/code
cd ~/code
git clone https://github.com/movidius/ncsdk
git clone https://github.com/movidius/ncappzoo
cd ~/code/ncsdk
make install
ln -sf ~/code/ncappzoo /opt/movidius/ncappzoo

# build ros2_intel_movidius_ncs
mkdir ~/ros2_ws/src -p
cd ~/ros2_ws/src
git clone https://github.com/intel/ros2_intel_movidius_ncs.git
cd ~/ros2_ws
source /opt/ros/crystal/setup.bash
colcon build --symlink-install (Install python3-colcon-common-extensions by apt-get if colcon command not exist)

# build CNN model (Please plugin NCS device on the host while compiling)
cd /opt/movidius/ncappzoo/caffe/SSD_MobileNet
make

# Copy object label file to NCSDK installation location.
cp ~/ros2_ws/src/ros2_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/

（5）Install OpenCV3 with opencv-contrib

mkdir ${HOME}/opencv
cd ${HOME}/opencv
git clone https://github.com/opencv/opencv.git -b 3.3.0
git clone https://github.com/opencv/opencv_contrib.git -b 3.3.0
mkdir opencv/build -p
cd opencv/build
cmake -DOPENCV_EXTRA_MODULES_PATH=${HOME}/opencv/opencv_contrib/modules \ 
      -DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_opencv_cnn_3dobj=OFF ..
make -j8
sudo make install
sudo ldconfig

这里补上依赖

sudo apt-get install liblz4-dev

下面就属于安装OpenVINO的

（6）Install Intel Distribution of OpenVINO toolkit

Download OpenVINO toolkit by following the guide

cd ~/Downloads
wget -c http://registrationcenter-download.intel.com/akdlm/irc_nas/15078/l_openvino_toolkit_p_2018.5.455.tgz

Install OpenVINO toolkit by following the guide

cd ~/Downloads
tar -xvf l_openvino_toolkit_p_2018.5.455.tgz
cd l_openvino_toolkit_p_2018.5.455
# root is required instead of sudo
sudo -E ./install_cv_sdk_dependencies.sh
sudo ./install_GUI.sh
# build sample code under OpenVINO toolkit
source /opt/intel/computer_vision_sdk/bin/setupvars.sh
cd /opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/
mkdir build
cd build
cmake ..
make

Configure the Neural Compute Stick USB Driver

cd ~/Downloads
cat <<EOF > 97-usbboot.rules
SUBSYSTEM=="usb", ATTRS{idProduct}=="2150", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
SUBSYSTEM=="usb", ATTRS{idProduct}=="2485", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
SUBSYSTEM=="usb", ATTRS{idProduct}=="f63b", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
EOF
sudo cp 97-usbboot.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger
sudo ldconfig
rm 97-usbboot.rules

（7）Install OpenCL Driver for GPU

 cd /opt/intel/computer_vision_sdk/install_dependencies
 sudo ./install_NEO_OCL_driver.sh

（8）（OpenCV）Additional steps are required on ubuntu 18.04

 sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
 sudo apt update
 sudo apt install libjasper1 libjasper-dev

（9）（librealsense SDK2.0）Other Dependencies

 #librealsense dependency
 sudo apt-get install -y libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
 sudo apt-get install -y libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev
 # numpy and networkx
 pip3 install numpy
 pip3 install networkx
 # libboost
 sudo apt-get install -y --no-install-recommends libboost-all-dev
 cd /usr/lib/x86_64-linux-gnu
 sudo ln -sf libboost_python-py35.so libboost_python3.so

（10）Building and Installation

Build sample code under openvino toolkit

 # root is required instead of sudo
 source /opt/intel/computer_vision_sdk/bin/setupvars.sh
 cd /opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/
 mkdir build
 cd build
 cmake ..
 make

set ENV CPU_EXTENSION_LIB and GFLAGS_LIB

 export CPU_EXTENSION_LIB=/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib/libcpu_extension.so
 export GFLAGS_LIB=/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib/libgflags_nothreads.a

Install ROS2_OpenVINO packages

 mkdir -p ~/ros2_overlay_ws/src
 cd ~/ros2_overlay_ws/src
 git clone https://github.com/intel/ros2_openvino_toolkit
 git clone https://github.com/intel/ros2_object_msgs
 git clone https://github.com/ros-perception/vision_opencv -b ros2
 git clone https://github.com/ros2/message_filters.git
 git clone https://github.com/ros-perception/image_common.git -b ros2
 git clone https://github.com/intel/ros2_intel_realsense.git

Build package

 source ~/ros2_ws/install/local_setup.bash
 source /opt/intel/computer_vision_sdk/bin/setupvars.sh
 export OpenCV_DIR=$HOME/code/opencv/build
 cd ~/ros2_overlay_ws
 colcon build --symlink-install
 source ./install/local_setup.bash
 sudo mkdir -p /opt/openvino_toolkit
 sudo ln -sf ~/ros2_overlay_ws/src/ros2_openvino_toolkit /opt/openvino_toolkit/ros2_openvino_toolkit

（10）Running the Demo

这里虽然是Running the Demo，但是不Running一下，OA就跑不起来，原因是这里会需要下一个model，而这些model也是OA需要的
download and convert a trained model to produce an optimized Intermediate Representation (IR) of the model

#object segmentation model
cd /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/install_prerequisites
sudo ./install_prerequisites.sh
mkdir -p ~/Downloads/models
cd ~/Downloads/models
wget http://download.tensorflow.org/models/object_detection/mask_rcnn_inception_v2_coco_2018_01_28.tar.gz
tar -zxvf mask_rcnn_inception_v2_coco_2018_01_28.tar.gz
cd mask_rcnn_inception_v2_coco_2018_01_28
python3 /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo_tf.py --input_model frozen_inference_graph.pb --tensorflow_use_custom_operations_config /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/extensions/front/tf/mask_rcnn_support.json --tensorflow_object_detection_api_pipeline_config pipeline.config --reverse_input_channels --output_dir ./output/
sudo mkdir -p /opt/models
sudo ln -sf ~/Downloads/models/mask_rcnn_inception_v2_coco_2018_01_28 /opt/models/
#object detection model
cd /opt/intel/computer_vision_sdk/deployment_tools/model_downloader
sudo python3 ./downloader.py --name mobilenet-ssd
#FP32 precision model
sudo python3 /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo.py --input_model /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --output_dir /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP32 --mean_values [127.5,127.5,127.5] --scale_values [127.5]
#FP16 precision model
sudo python3 /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo.py --input_model /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --output_dir /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP16 --data_type=FP16 --mean_values [127.5,127.5,127.5] --scale_values [127.5]

copy label files (excute once)

 sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/emotions-recognition/FP32/emotions-recognition-retail-0003.labels /opt/intel/computer_vision_sdk/deployment_tools/intel_models/emotions-recognition-retail-0003/FP32
 sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/face_detection/face-detection-adas-0001.labels /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-adas-0001/FP32
 sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/face_detection/face-detection-adas-0001.labels /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-adas-0001/FP16
 sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/object_segmentation/frozen_inference_graph.labels ~/Downloads/models/mask_rcnn_inception_v2_coco_2018_01_28/output
 sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/object_detection/mobilenet-ssd.labels /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP32
 sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/object_detection/mobilenet-ssd.labels /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP16

set ENV LD_LIBRARY_PATH and environment

 source /opt/intel/computer_vision_sdk/bin/setupvars.sh
 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib

（11）Build ros2_object_analytics

# get code
mkdir ~/ros2_ws/src -p
cd ~/ros2_ws/src
git clone https://github.com/intel/ros2_object_analytics.git -b devel (devel branch is the latest code with 2D tracking features, while master branch is stable for ros2 bloom release)

# Build
cd ~/ros2_ws
source /opt/ros/crystal/setup.bash
colcon build --symlink-install

（12）Object Analytics with OpenVINO

# Start OA demo with OpenVINO
source /opt/ros/crystal/setup.bash
source ~/ros2_ws/install/local_setup.bash
source /opt/intel/computer_vision_sdk/bin/setupvars.sh
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib



Option1: if ros2_openvino_toolkit built from source code
ros2 launch object_analytics_node object_analytics_with_openvino_oss.launch.py

Option2: if ros2_openvino_toolkit got from Robotics_SDK
ros2 launch object_analytics_node object_analytics_with_openvino_sdk.launch.py

注意lanch之前前面四个必不可少的source和export，后两个是和OpenVINO有关的。

然后
…
就成功啦！！！哈哈哈…
整个效果看上去还是非常牛逼的，到这一步算是装好了，主要还是自己太菜，接下来终于可以开心地看源码了～