目标
在Ubuntu20.04LTS系统下安装D435i的驱动SDK2和ROS包，实现在ROS环境中使用D435i。
一、D435i简介
D435i是Inter公司RealSense系列摄像头的一种，采用英特尔最新的深度感应硬件和软件，封装成易于集成的产品。相比D435，D435i添加了IMU（惯性测量单元），不仅为 SLAM 和跟踪应用打开了大门，还使手持扫描系统的注册和校准更容易，在虚拟/增强现实和无人机等领域很重要。
D435i是一个RGB-D相机，也就是我们常说的深度相机，目前深度相机的实现原理主要分为三种：分别是结构光，Tof，双目成像。REALSENSE使用的是结构光的方案。正面的四个摄像头，从左向右以次是左红外相机，红外点阵投射仪，右红外相机，和RGB相机。
尽管具有红外投射器，但并不是采用红外反射测距。它的作用仅仅是投射不可见的固定的红外纹理样式，提高在纹理不明显的环境中（例如白墙）的深度计算精度，辅助双目视觉测距。左右两个相机将图像数据送入内置的深度处理器，在其中基于双目测距的原理计算每个像素的深度值。
最高 1280×720 双目深度分辨率;
最高 1920×1080 RGB 分辨率;
最高 90 FPS 深度视频流。
深度流与普通 RGB 视频流类似，只不过每个像素点的值不再是 RGB 或灰度值，而是物体相对于相机的距离。
只能对相同帧率的 RGB 与深度视频流做同步设置。
双目 baseline 为 50 mm;
深度探测范围 0.2 m ~ 10 m;
深度坐标系以左侧相机为中心（下图中 centerline of 1/4-20 是指三脚架螺丝空的中心）
二、环境配置
系统：Ubuntu20.04LTS
ROS：1.15.14
摄像机：D435i
三、RealSense的SDK2.0安装
方式一：(优先)
1.注册服务器的公钥
sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE
2.将服务器添加到存储库列表中
sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u
3.安装SDK2
方式一（apt）：
基础安装
sudo apt-get install librealsense2-dkms
sudo apt-get install librealsense2-utils
选装
sudo apt-get install librealsense2-dev
sudo apt-get install librealsense2-dbg
Tip:安装dev包后，可使用g++ -std=c++11 filename.cpp -lrealsense2 或 别的IDE来编译具有librealsense的应用。
测试安装结果
realsense-viewer
方式二（源代码）：
下载librealsense，然后进入该目录，运行下列指令安装和编译依赖项：
sudo apt-get install libudev-dev pkg-config libgtk-3-dev
sudo apt-get install libusb-1.0-0-dev pkg-config
sudo apt-get install libglfw3-dev
sudo apt-get install libssl-dev
sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && udevadm trigger
mkdir build
cd build
cmake ../ -DBUILD_EXAMPLES=true
make
sudo make install
测试安装结果
realsense-viewer
四、ROS包安装
方式一（apt）：
安装ROS版本的realsense2_camera
sudo apt-get install ros-$ROS_DISTRO-realsense2-camera
sudo apt-get install ros-$ROS_DISTRO-realsense2-description
方式二（源代码）：
进入catkin_ws工作空间的src目录，下载realsense和ddynamic_reconfigure，然后用ROS的catkin_make指令编译。
cd ~/catkin_ws/src
git clone https://github.com/IntelRealSense/realsense-ros.git
git clone https://github.com/pal-robotics/ddynamic_reconfigure.git
cd ~/catkin_ws && catkin_make
安装rgbd-launch
rgbd_launch是一组打开RGBD设备，并load 所有nodelets转化 raw depth/RGB/IR 流到深度图(depth image), 视差图(disparity image)和点云(point clouds)的launch文件集。
sudo apt-get install ros-noetic-rgbd-launch
测试编译结果：
roslaunch realsense2_camera demo_pointcloud.launch
五、摄像机CV的ROS包节点
在catkin_ws工作空间中创建一个usb_camera包，含有一个图像消息发布节点img_publisher与图像消息接收显示节点img_viewer。
package.xml：
<buildtool_depend>catkin</buildtool_depend>
<build_depend>cv_bridge</build_depend>
<build_depend>image_transport</build_depend>
<build_depend>roscpp</build_depend>
<build_depend>sensor_msgs</build_depend>
<build_depend>std_msgs</build_depend>
<build_depend>opencv2</build_depend>
<build_export_depend>cv_bridge</build_export_depend>
<build_export_depend>image_transport</build_export_depend>
<build_export_depend>roscpp</build_export_depend>
<build_export_depend>sensor_msgs</build_export_depend>
<build_export_depend>std_msgs</build_export_depend>
<build_export_depend>opencv2</build_export_depend>
<exec_depend>cv_bridge</exec_depend>
<exec_depend>image_transport</exec_depend>
<exec_depend>roscpp</exec_depend>
<exec_depend>sensor_msgs</exec_depend>
<exec_depend>std_msgs</exec_depend>
<exec_depend>opencv2</exec_depend>
<!-- The export tag contains other, unspecified, tags -->
<export>
<!-- Other tools can request additional information be placed here -->
img_publisher.cpp节点：
#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <opencv2/highgui.hpp>
#include <opencv2/calib3d.hpp>
#include <cv_bridge/cv_bridge.h>
#include <iostream>
int main(int argc, char** argv)
{
ros::init(argc, argv, "img_publisher");
ros::NodeHandle nh;
image_transport::ImageTransport it(nh);
image_transport::Publisher pub = it.advertise("camera/image", 1);
cv::VideoCapture cap;
cv::Mat frame;
int deviceID=0;
if(argc>1)
deviceID=argv[1][0]-'0';
int apiID=cv::CAP_ANY;
cap.open(deviceID+apiID);
if(!cap.isOpened()){
std::cerr<<"ERROR! Unable to open camera"<<std::endl;
return -1;
}
ros::Rate loop_rate(30);
while (nh.ok()) {
cap.read(frame);
if(!frame.empty()){
sensor_msgs::ImagePtr msg = cv_bridge::CvImage(std_msgs::Header(), "bgr8", frame).toImageMsg();
pub.publish(msg);
}
ros::spinOnce();
loop_rate.sleep();
}
return 0;
}
img_viewer：
//接收并查看图像消息的节点img_viewer.cpp
#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <opencv2/highgui/highgui.hpp>
#include <cv_bridge/cv_bridge.h>
//依赖包有std_msgs（消息传递），roscpp（c++），cv_bridge（ros和opencv图像消息转换），sensor_msgs（传感器消息），image_transport（图像编码传输）
void imageCallback(const sensor_msgs::ImageConstPtr& msg)
{
try
{
cv::imshow("view", cv_bridge::toCvShare(msg, "bgr8")->image);
}
catch (cv_bridge::Exception& e)
{
ROS_ERROR("Could not convert from '%s' to 'bgr8'.", msg->encoding.c_str());
}
}
int main(int argc, char **argv)
{
ros::init(argc, argv, "img_viewer");
ros::NodeHandle nh;
cv::namedWindow("view");
cv::startWindowThread();
image_transport::ImageTransport it(nh);
image_transport::Subscriber sub = it.subscribe("camera/image", 1, imageCallback);
ros::spin();
cv::destroyWindow("view");
return 0;
}
CMakeLists.txt：
cmake_minimum_required(VERSION 3.0.2)
project(usb_camera)
## Compile as C++11, supported in ROS Kinetic and newer
# add_compile_options(-std=c++11)
## Find catkin macros and libraries
## if COMPONENTS list like find_package(catkin REQUIRED COMPONENTS xyz)
## is used, also find other catkin packages
find_package(catkin REQUIRED COMPONENTS
cv_bridge
image_transport
roscpp
sensor_msgs
std_msgs
)
## System dependencies are found with CMake's conventions
# find_package(Boost REQUIRED COMPONENTS system)
find_package(OpenCV REQUIRED)
## Uncomment this if the package has a setup.py. This macro ensures
## modules and global scripts declared therein get installed
## See http://ros.org/doc/api/catkin/html/user_guide/setup_dot_py.html
# catkin_python_setup()
################################################
## Declare ROS messages, services and actions ##
################################################
## To declare and build messages, services or actions from within this
## package, follow these steps:
## * Let MSG_DEP_SET be the set of packages whose message types you use in
## your messages/services/actions (e.g. std_msgs, actionlib_msgs, ...).
## * In the file package.xml:
## * add a build_depend tag for "message_generation"
## * add a build_depend and a exec_depend tag for each package in MSG_DEP_SET
## * If MSG_DEP_SET isn't empty the following dependency has been pulled in
## but can be declared for certainty nonetheless:
## * add a exec_depend tag for "message_runtime"
## * In this file (CMakeLists.txt):
## * add "message_generation" and every package in MSG_DEP_SET to
## find_package(catkin REQUIRED COMPONENTS ...)
## * add "message_runtime" and every package in MSG_DEP_SET to
## catkin_package(CATKIN_DEPENDS ...)
## * uncomment the add_*_files sections below as needed
## and list every .msg/.srv/.action file to be processed
## * uncomment the generate_messages entry below
## * add every package in MSG_DEP_SET to generate_messages(DEPENDENCIES ...)
## Generate messages in the 'msg' folder
# add_message_files(
# FILES
# Message1.msg
# Message2.msg
# )
## Generate services in the 'srv' folder
# add_service_files(
# FILES
# Service1.srv
# Service2.srv
# )
## Generate actions in the 'action' folder
# add_action_files(
# FILES
# Action1.action
# Action2.action
# )
## Generate added messages and services with any dependencies listed here
# generate_messages(
# DEPENDENCIES
# sensor_msgs# std_msgs
# )
################################################
## Declare ROS dynamic reconfigure parameters ##
################################################
## To declare and build dynamic reconfigure parameters within this
## package, follow these steps:
## * In the file package.xml:
## * add a build_depend and a exec_depend tag for "dynamic_reconfigure"
## * In this file (CMakeLists.txt):
## * add "dynamic_reconfigure" to
## find_package(catkin REQUIRED COMPONENTS ...)
## * uncomment the "generate_dynamic_reconfigure_options" section below
## and list every .cfg file to be processed
## Generate dynamic reconfigure parameters in the 'cfg' folder
# generate_dynamic_reconfigure_options(
# cfg/DynReconf1.cfg
# cfg/DynReconf2.cfg
# )
###################################
## catkin specific configuration ##
###################################
## The catkin_package macro generates cmake config files for your package
## Declare things to be passed to dependent projects
## INCLUDE_DIRS: uncomment this if your package contains header files
## LIBRARIES: libraries you create in this project that dependent projects also need
## CATKIN_DEPENDS: catkin_packages dependent projects also need
## DEPENDS: system dependencies of this project that dependent projects also need
catkin_package(
# INCLUDE_DIRS include
# LIBRARIES usb_camera
# CATKIN_DEPENDS cv_bridge image_transport roscpp sensor_msgs std_msgs
# DEPENDS system_lib
)
###########
## Build ##
###########
## Specify additional locations of header files
## Your package locations should be listed before other locations
include_directories(
# include
${catkin_INCLUDE_DIRS}
${OpenCV_INCLUDE_DIRS}
)
## Declare a C++ library
# add_library(${PROJECT_NAME}
# src/${PROJECT_NAME}/usb_camera.cpp
# )
## Add cmake target dependencies of the library
## as an example, code may need to be generated before libraries
## either from message generation or dynamic reconfigure
# add_dependencies(${PROJECT_NAME} ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})
## Declare a C++ executable
## With catkin_make all packages are built within a single CMake context
## The recommended prefix ensures that target names across packages don't collide
# add_executable(${PROJECT_NAME}_node src/usb_camera_node.cpp)
## Rename C++ executable without prefix
## The above recommended prefix causes long target names, the following renames the
## target back to the shorter version for ease of user use
## e.g. "rosrun someones_pkg node" instead of "rosrun someones_pkg someones_pkg_node"
# set_target_properties(${PROJECT_NAME}_node PROPERTIES OUTPUT_NAME node PREFIX "")
## Add cmake target dependencies of the executable
## same as for the library above
# add_dependencies(${PROJECT_NAME}_node ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})
## Specify libraries to link a library or executable target against
# target_link_libraries(${PROJECT_NAME}_node
# ${catkin_LIBRARIES}
# )
add_executable(img_publisher src/img_publisher.cpp)
target_link_libraries(img_publisher ${catkin_LIBRARIES} ${OpenCV_LIBS})
add_executable(img_viewer src/img_viewer.cpp)
target_link_libraries(img_viewer ${catkin_LIBRARIES} ${OpenCV_LIBS})
#############
## Install ##
#############
# all install targets should use catkin DESTINATION variables
# See http://ros.org/doc/api/catkin/html/adv_user_guide/variables.html
## Mark executable scripts (Python etc.) for installation
## in contrast to setup.py, you can choose the destination
# catkin_install_python(PROGRAMS
# scripts/my_python_script
# DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )
## Mark executables for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_executables.html
# install(TARGETS ${PROJECT_NAME}_node
# RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )
## Mark libraries for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_libraries.html
# install(TARGETS ${PROJECT_NAME}
# ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
# LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
# RUNTIME DESTINATION ${CATKIN_GLOBAL_BIN_DESTINATION}
# )
## Mark cpp header files for installation
# install(DIRECTORY include/${PROJECT_NAME}/
# DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
# FILES_MATCHING PATTERN "*.h"
# PATTERN ".svn" EXCLUDE
# )
## Mark other files for installation (e.g. launch and bag files, etc.)
# install(FILES
# # myfile1
# # myfile2
# DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
# )
#############
## Testing ##
#############
## Add gtest based cpp test target and link libraries
# catkin_add_gtest(${PROJECT_NAME}-test test/test_usb_camera.cpp)
# if(TARGET ${PROJECT_NAME}-test)
# target_link_libraries(${PROJECT_NAME}-test ${PROJECT_NAME})
# endif()
## Add folders to be run by python nosetests
# catkin_add_nosetests(test)
六、问题排查
1. 查找usb摄像头
在最初的节点运行时，提示找不到usb摄像头，改变摄像机的索引号也不行。查摄像头占用情况：
ls /dev/video*
/dev/video0 /dev/video1 /dev/video2 /dev/video3 /dev/video4 /dev/video5
摄像机索引没问题。
根据输出信息[ WARN:0] OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Cannot identify device '/dev/video0'. [ WARN:0] OpenCV | GStreamer warning: unable to start pipeline [ WARN:0] OpenCV | GStreamer warning: GStreamer: pipeline have not been created
查找相关文献，试探性的调试如下：
find /lib/modules/ -name "*v4l2*.ko"
显示如下信息：
/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-flash-led-class.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-dv-timings.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-fwnode.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-mem2mem.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/i2c/adv7511-v4l2.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/common/videobuf2/videobuf2-v4l2.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/common/v4l2-tpg/v4l2-tpg.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-flash-led-class.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-dv-timings.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-fwnode.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-mem2mem.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/i2c/adv7511-v4l2.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/common/videobuf2/videobuf2-v4l2.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/common/v4l2-tpg/v4l2-tpg.ko
发现相关的ko，其中，adv7511-v4l2.ko这个是与usb摄像机相关的文件。运行：
sudo modprobe adv7511-v4l2
然后就可以打开USB摄像机了。modprobe命令用于自动处理可载入模块，设置摄像头设备后相当于默认使用该设备。
另外，用v4l2-ctl --list-devices 命令可以查看摄像头对应的设备号，如果没有v4l2-ctl，可先安装：
sudo apt install v4l-utils
v4l2-ctl --list-devices
Video Capture 9 (usb-0000:00:14.0-3):
/dev/video0
/dev/video1
/dev/video2
/dev/video3
/dev/video4
/dev/video5
/dev/media0
/dev/media1
经过上述操作后并不一定完全解决问题。
2. 了解Gstreamer框架
gstreamer是一个开发流式多媒体应用的开源框架，是为更好的处理音视频而设计开发的，但又不限于处理音视频，只要是有明显的数据流特征的应用。
gstreamer的功能核心是pipeline框架以及用于扩展功能的插件。通过gstreamer多媒体数据协商机制和同步机制，pipeline能够很好的地处理流式数据。开发者可以灵活利用已有插件，而且还可以自定义特定功能插件。
[1] Ubantu20.04安装libRealsense SDK 2.0 和ROS Wrapper for Intel RealSense Devices
[2] Ubuntu20.04+RealSense D435i
[3] ubuntu opencv 打开摄像头失败
[4] Gstreamer基础知识介绍
3219
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
