基于docker搭建tx2的ROS2交叉编译环境

基于docker搭建TX2的ROS2交叉编译环境

概述

ROS2官方文档有交叉编译相关说明。本文使用TX2最新官方镜像JetPack4.4版本，自带ubuntu18.04。有现成的ubuntu18.04就可以使用apt-get install安装预编译的ROS2，从而避免从ROS2源代码安装可能引入的折腾。
ROS2在ubuntu18.04支持的版本有两个：dashing和eloquent。这里选择安装dashing,具体参照官方安装指引Installing ROS 2 via Debian Packages¶。构建工具使用cmake，在交叉编译过程中，围绕着 --sysroot, Wl,-rpath,-rpath-link等参数进行配置，解决链接过程中各种找不到Warning libm.so.6 needed by .... not found.等基本库找不到的问题。

名词解释

关键词	说明
host	开发和编译应用程序的PC（ubuntu）
target	TX2的ubuntu环境
container	运行于host上的docker容器，这里包含用于交叉编译TX2的所有工具链和sysroot

操作步骤

• TX2刷好机，第一次启动ubuntu18.04的时候创建主用户名为nvidia。
• TX2安装openssh-serversudo apt-get install openssh-server,方便host 通过ssh访问TX2。
• 在TX2中安装ros-dashing-ros-base sudo apt install ros-dashing-ros-base
• 从TX2 target导出sysroot用于构建docker image
• 构建一个docker镜像，准备好全部工具链。
• 基于镜像创建一个container,host上运行IDE(VSCode/CLion)通过ssh访问到container中的ROS2 package，编辑和构建ROS2 package。
• 拷贝构建好的ROS2 package 可执行文件到 TX2 target中运行。

获取ubuntu18.04docker镜像

host 端执行的操作

• 如果你还没有ubuntu:18.04镜像，拉取一个docker pull ubuntu:18.04
• 运行一个ubuntu:18.04镜像的容器docker run -tid ubuntu:18.04
• 以root用户通过bash登录到新创建的容器中docker exec -ti ba8e bash

$ docker pull ubuntu:18.04
$ docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
ubuntu                               16.04               4b22027ede29        2 months ago        127MB
ubuntu                               18.04               6526a1858e5d        2 months ago        642MB           <?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
  <name>service_demo</name>
  <version>0.3.4</version>
  <description>TODO</description>
  <maintainer email="demo@mail.com">Demo</maintainer>

  <license>Apache-2.0</license><?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
  <name>service_demo</name>
  <version>0.3.4</version>
  <description>TODO</description>
  <maintainer email="demo@mail.com">Demo</maintainer>

  <license>Apache-2.0</license>
  <build_depend>rclcpp</build_depend><?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
  <name>service_demo</name>
  <version>0.3.4</version>
  <description>TODO</description>
  <maintainer email="demo@mail.com">Demo</maintainer>

  <license>Apache-2.0</license>
  <build_depend>rclcpp</build_depend>
  <build_depend>builtin_interfaces</build_depend>
  <buildtool_depend>rosidl_default_generators</buildtool_de<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
  <name>service_demo</name>
  <version>0.3.4</version>
  <description>TODO</description>
  <maintainer email="demo@mail.com">Demo</maintainer>

  <license>Apache-2.0</license>
  <build_depend>rclcpp</build_depend>
  <build_depend>builtin_interfaces</build_depend>
  <buildtool_depend>rosidl_default_generators</buildtool_depend>

  <exec_depend>rclcpp</exec_depend>
  <exec_depend>std_msgs</exec_depend>
  <exec_depend>rosidl_default_runtime</exec_depend>

  <member_of_group>rosidl_interface_packages</member_of_group>

</package>
pend>

  <exec_depend>rclcpp</exec_depend>
  <exec_depend>std_msgs</exec_depend>
  <exec_depend>rosidl_default_runtime</exec_depend>

  <member_of_group>rosidl_interface_packages</member_of_group>

</package>

  <build_depend>builtin_interfaces</build_depend>
  <buildtool_depend>rosidl_default_generators</buildtool_depend>

  <exec_depend>rclcpp</exec_depend>
  <exec_depend>std_msgs</exec_depend>
  <exec_depend>rosidl_default_runtime</exec_depend>

  <member_of_group>rosidl_interface_packages</member_of_group>

</package>

  <build_depend>rclcpp</build_depend>
  <build_depend>builtin_interfaces</build_depend>
  <buildtool_depend>rosidl_default_generators</buildtool_depend>

  <exec_depend>rclcpp</exec_depend>
  <exec_depend>std_msgs</exec_depend>
  <exec_depend>rosidl_default_runtime</exec_depend>

  <member_of_group>rosidl_interface_packages</member_of_group>

</package>


$ docker run -tid ubuntu:18.04
ba8e88b1d2e673d61be2716d2e2911d4e441f7bd62117cda1201904389e6f126

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ba8e88b1d2e6        ubuntu:18.04        "/bin/bash"         3 seconds ago       Up 3 seconds                            awesome_diffie

$ docker exec -ti ba8e bash

容器中执行的操作：

• 在容器中创建一个用户"admin"，adduser admin
• 在容器中安装openssh-server以便VSCode/CLion远程访问，
• 测试openssh-server运行

root@ba8e88b1d2e6:/# adduser admin
Adding user `admin' ...
Adding new group `admin' (1000) ...
Adding new user `admin' (1000) with group `admin' ...
Creating home directory `/home/admin' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for admin
Enter the new value, or press ENTER for the default
        Full Name []: administrator
        Room Number []: 1
        Work Phone []: 1
        Home Phone []: 1
        Other []: 1
Is the information correct? [Y/n] Y

root@ba8e88b1d2e6:/# apt update && apt install openssh-server vim rsync build其中其中其中其中-essential -y
root@ba8e88b1d2e6:/# vim /etc/ssh/sshd_config 
root@ba8e88b1d2e6:/# mkdir /run/sshd
root@ba8e88b1d2e6:/# /usr/sbin/sshd
root@ba8e88b1d2e6:/# ssh admin@localhost
admin@ba8e88b1d2e6:~$

获取TX2根文件系统

由于之前给TX2刷机使用到Nvidia SDK manager, TX2的根文件系统会被缓存起来，缓存好的TX2根文件系统压缩文档是
~/Downloads/nvidia/sdkm_downloads/Tegra_Linux_Sample-Root-Filesystem_R32.1.0_aarch64.tbz2
host端执行如下操作

• 将根文件系统压缩文档拷贝到容器中
• 以admin用户登录到容器docker exec -ti ba8e login

$ docker cp Tegra_Linux_Sample-Root-Filesystem_R32.1.0_aarch64.tbz2 ba8e:/home/admin
$ winpty docker exec -ti ba login
ba8e88b1d2e6 login: admin
Password:
Last login: Mon Nov  9 02:45:16 UTC 2020 on pts/1
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.19.76-linuxkit x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

This system has been minimized by removing packages and content that are
not required on a system that users do not log into.

To restore this content, you can run the 'unminimize' command.
admin@ba8e88b1d2e6:~$

容器中运行如下指令：

admin@ba8e88b1d2e6:~$ mkdir tx2-rootfs
admin@ba8e88b1d2e6:~$ tar xvf Tegra_Linux_Sample-Root-Filesystem_R32.1.0_aarch64.tbz2 -C tx2-rootfs
admin@ba8e88b1d2e6:~$ ls tx2-rootfs/
bin  boot  dev  etc  home  lib  media  mnt  opt  proc  root  run  sbin  snap  srv  sys  tmp  usr
admin@ba8e88b1d2e6:~$ rm Tegra_Linux_Sample-Root-Filesystem_R32.1.0_aarch64.tbz2

获取TX2的ROS及相关依赖库

容器中执行下面操作

• 从TX2中拷贝ros2到tx2-rootfs根文件系统对应位置

admin@ba8e88b1d2e6:~$ rsync -av --progress nvidia@192.168.55.1:/opt/ros tx2-rootfs/opt
nvidia@192.168.55.1's password:
...........

admin@ba8e88b1d2e6:~$ ls tx2-rootfs/opt/ros/dashing/
_order_packages.py  bin  cmake  include  lib  local_setup.bash  local_setup.sh  local_setup.zsh  setup.bash  setup.sh  setup.zsh  share  src
admin@ba8e88b1d2e6:~$

构建依赖

下载新版本CMake

由于ubuntu18.04 apt install cmake所安装的版本是3.10.2，不支持add_link_options(),所以安装最新发布的cmake3.18.4。
从cmake官网下载最新的稳定发布版本3.18.4
如果官网下载速度过慢，可以从这里分流。

下载linaro交叉编译工具链

linaro gcc 最新官方下载渠道

拷贝构建工具到容器

host 执行下面指令：

• 拷贝cmake linux安装包到容器中docker cp cmake-3.18.4-Linux-x86_64.sh ba:/usr/local/lib

$ docker cp cmake-3.18.4-Linux-x86_64.sh ba:/usr/local/lib
$ docker cp lia

容器中执行下面指令：

• 安装交叉编译工具链gcc,g++
• 安装cmake3.18.4(root用户)
• 配置环境变量
• 创建用于cmake交叉编译的通用文件cross.cmake
  host 通过root登录到容器中

$ docker exec -ti ba 

root@ba8e88b1d2e6:/usr/local/lib# ./cmake-3.18.4-Linux-x86_64.sh
root@ba8e88b1d2e6:/usr/local/lib# rm cmake-3.18.4-Linux-x86_64.sh
root@ba8e88b1d2e6:/usr/local/lib# tar xvf gcc-linaro-7.5.0-2019.12-x86_64_aarch64-lin
ux-gnu.tar
root@ba8e88b1d2e6:/usr/local/lib# rm gcc-linaro-7.5.0-2019.12-x86_64_aarch64-lin
ux-gnu.tar
root@ba8e88b1d2e6:/usr/local/lib# mv gcc-linaro-7.5.0-2019.12-x86_64_aarch64-lin
ux-gnu/ gcc-linaro-7.5.0
root@ba8e88b1d2e6:/usr/local/lib# ls
cmake-3.18.4-Linux-x86_64  gcc-linaro-7.5.0  python3.6
root@ba8e88b1d2e6:/usr/local/lib#

配置用户环境变量

以普通用户登录容器后，修改用户环境变量文件vim ~/.bashrc
在~/.bashrc头部加上如下内容

CMAKE_HOME=/usr/local/lib/cmake-3.18.4-Linux-x86_64
export TX2_SYSROOT=/home/admin/tx2-rootfs
export ROS2_HOME=${TX2_SYSROOT}/opt/ros/dashing
export PYTHONPATH=${ROS2_HOME}/lib/python3.6/site-packages
export PATH=${PATH}:${CMAKE_HOME}/bin:${ROS2_HOME}/bin

创建cmake交叉编译文件

admin@ba8e88b1d2e6:~$ mkdir workspace && cd workspace && vim cross.cmake

cross.cmake内容如下

set(CMAKE_SYSTEM_VERSION 1)
set(CMAKE_SYSTEM_PROCESSOR aarch64)

set(SYSROOT_PATH  /home/admin/tx2-rootfs)
set(CMAKE_SYSROOT "${SYSROOT_PATH}")
message(STATUS  "Using sysroot path as ${SYSROOT_PATH}")

set(TOOLCHAIN_PATH /usr/local/lib/linaro-7.3.1)
set(TOOLCHAIN_HOST ${TOOLCHAIN_PATH}/bin/aarch64-linux-gnu)

set(TOOLCHAIN_CC "${TOOLCHAIN_HOST}-gcc")
set(TOOLCHAIN_CXX "${TOOLCHAIN_HOST}-g++")
set(TOOLCHAIN_LD "${TOOLCHAIN_HOST}-ld")

set(CMAKE_CROSSCOMPILING TRUE)
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_C_COMPILER ${TOOLCHAIN_CC})
set(CMAKE_CXX_COMPILER ${TOOLCHAIN_CXX})
set(CMAKE_LINKER ${TOOLCHAIN_LD})

add_link_options("LINKER:-rpath-link,/home/admin/tx2-rootfs/lib/aarch64-linux-gnu:/home/admin/tx2-rootfs/usr/lib/aarch64-linux-gnu")

message(STATUS "cmake prefix: ${CMAKE_PREFIX_PATH}")

set(CMAKE_FIND_ROOT_PATH "${SYSROOT_PATH}" "${CMAKE_PREFIX_PATH}" "${TOOLCHAIN_PATH}/aarch64-linux-gnu/libc")
message(STATUS "cmake find root path is ${CMAKE_FIND_ROOT_PATH}")

# search for programs in the build host directories
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# for libraries and headers in the target directories
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

测试ROS2交叉编译

新创建一个包，命名为service_demo

• package.xml的内容如下

<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
  <name>service_demo</name>
  <version>0.3.4</version>
  <description>TODO</description>
  <maintainer email="demo@mail.com">Demo</maintainer>
  <license>Apache-2.0</license>
  <build_depend>rclcpp</build_depend>
  <build_depend>builtin_interfaces</build_depend>
  <buildtool_depend>rosidl_default_generators</buildtool_depend>
  <exec_depend>rclcpp</exec_depend>
  <exec_depend>std_msgs</exec_depend>
  <exec_depend>rosidl_default_runtime</exec_depend>
  <member_of_group>rosidl_interface_packages</member_of_group>
</package>

• CMakeLists.txt内容如下：

cmake_minimum_required(VERSION 3.5)
project(service_demo)

set(CMAKE_CXX_STANDARD 14) 
find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(rosidl_default_generators REQUIRED)

rosidl_generate_interfaces(${PROJECT_NAME} "srv/ServiceDemo.srv")
ament_export_dependencies(rosidl_default_runtime)

add_executable(service_demo_server src/service_demo_server.cpp)
rosidl_target_interfaces(service_demo_server ${PROJECT_NAME} "rosidl_typesupport_cpp")
ament_target_dependencies(service_demo_server rclcpp) 

ament_package()

• 自定义service文件srv/ServiceDemo.srv内容如下:

int64 a
int64 b
---
int64 sum

• server源代码src/service_demo_server.cpp内容如下

#include "rclcpp/rclcpp.hpp"
#include "service_demo/srv/service_demo.hpp"

#include <memory>

void add(const std::shared_ptr<service_demo::srv::ServiceDemo::Request> request,
          std::shared_ptr<service_demo::srv::ServiceDemo::Response>      response)
{
  response->sum = request->a + request->b;
  RCLCPP_INFO(rclcpp::get_logger("rclcpp"), "Incoming request\na: %ld" " b: %ld",
                request->a, request->b);
  RCLCPP_INFO(rclcpp::get_logger("rclcpp"), "sending back response: [%ld]", (long int)response->sum);
}

int main(int argc, char **argv)
{
  rclcpp::init(argc, argv);

  std::shared_ptr<rclcpp::Node> node = rclcpp::Node::make_shared("add_two_ints_server");

  rclcpp::Service<service_demo::srv::ServiceDemo>::SharedPtr service =
    node->create_service<service_demo::srv::ServiceDemo>("add_two_ints", &add);

  RCLCPP_INFO(rclcpp::get_logger("rclcpp"), "Ready to add two ints.");

  rclcpp::spin(node);
  rclcpp::shutdown();
}

• client源代码src/service_demo_client.cpp内容如下

#include "rclcpp/rclcpp.hpp"
#include "service_demo/srv/service_demo.hpp"

#include <chrono>
#include <cstdlib>
#include <memory>

using namespace std::chrono_literals;

int main(int argc, char **argv)#include "rclcpp/rclcpp.hpp"
#include "service_demo/srv/service_demo.hpp"

#include <chrono>
#include <cstdlib>
#include <memory>

{
  rclcpp::init(argc, argv);

  if (argc != 3) {
      RCLCPP_INFO(rclcpp::get_logger("rclcpp"), "usage: add_two_ints_client X Y");
      return 1;
  }

  std::shared_ptr<rclcpp::Node> node = rclcpp::Node::make_shared("add_two_ints_client");
  rclcpp::Client<service_demo::srv::ServiceDemo>::SharedPtr client =
    node->create_client<service_demo::srv::ServiceDemo>("add_two_ints");

  auto request = std::make_shared<service_demo::srv::ServiceDemo::Request>();
  request->a = atoll(argv[1]);
  request->b = atoll(argv[2]);

  while (!client->wait_for_service(1s)) {
    if (!rclcpp::ok()) {
      RCLCPP_ERROR(rclcpp::get_logger("rclcpp"), "Interrupted while waiting for the service. Exiting.");
      return 0;
    }   
    RCLCPP_INFO(rclcpp::get_logger("rclcpp"), "service not available, waiting again...");
  }
  
  auto result = client->async_send_request(request);
  // Wait for the result.
  if (rclcpp::spin_until_future_complete(node, result) ==
    rclcpp::executor::FutureReturnCode::SUCCESS)
  {
    RCLCPP_INFO(rclcpp::get_logger("rclcpp"), "Sum: %ld", result.get()->sum);
  } else {
    RCLCPP_ERROR(rclcpp::get_logger("rclcpp"), "Failed to call service add_two_ints");  
  }

  rclcpp::shutdown();
  return 0;
}

编译

$ mkdir build
$ cd build
$ cmake -DCMAKE_TOOLCHAIN_FILE=~/workspace/cross.cmake ..
$ make -j11

注意：自定义的msg、srv或action文件命名都必须是首字母大写，具体可以参考这里

谢谢关注，欢迎留言