乐视体感astra pro深度摄像头在ros系统获取 深度图像 彩色图像 无色彩点云数据 彩色点云数据

 

 

• 1.astra pro深度摄像头介绍

• 2.astra pro驱动安装

• 3.astra pro获取深度图像   无色彩pointCloud2

• 4.astra pro获取彩色图像  带彩色的pointCloud2

 

1.astra pro深度摄像头介绍

  

摄像头有四个镜头,从左到右的镜头分别是1 ir发射源  2 最小距离传感器  3 彩色摄像头 4 ir接收器

 

2.astra pro驱动安装

2.1下载驱动及依赖

ubuntu16 kinetic及ubuntu18 melodic下载以下驱

OpenNI-Linux-x64-2.3.tar.bz2

$sudo apt-get install build-essential freeglut3 freeglut3-dev
$ldconfig -p | grep libudev.so.1
$cd /lib/x86_64-linux-gnu
$sudo ln -s libudev.so.x.x.x libudev.so.1

 

2.2编译驱动

 解压下载的OpenNI-Linux-x64-2.3驱动

$ cd OpenNI-Linux-x64-2.3
$ sudo chmod a+x install.sh

• 安装

$ sudo chmod a+x install.sh
$ sudo ./install.sh

插拔摄像头,source环境

$ source OpenNIDevEnvironment

• 编译

$ cd Samples/SimpleViewer
$ make

• 连接设备，执行示例

$ cd Bin/x64-Release
$ ./SimpleViewer 

 

编译安装libuvc

$ cd ~
$ git clone https://github.com/ktossell/libuvc
$ cd libuvc
$ mkdir build
$ cd build
$ cmake ..
$ make && sudo make install
$ sudo cp /usr/include/libusb-1.0/libusb.h  /usr/local/include/libuvc/

 

2.3 安装ros的官方astra_launch  astra_camera libuvc rgbd包， ubuntu melodic的需要下载源码安装这些包，因为apt-get不到

如果提示源码安装找不到包，到ros wiki上更新一下本地的ros软件源

$ sudo apt-get install ros-kinetic-astra-camera 
$ sudo apt-get install ros-kinetic-astra-launch  
$ sudo apt-get install ros-kinetic-libuvc-*
$ sudo apt-get install ros-kinetic-rgbd-launch 

3.astra pro获取深度图像   无色彩pointCloud2

$ roslaunch astra_launch astra.launch  

打开rqt_image_view查看图像话题

$ rqt_image_view

 

打开rviz查看无彩色点云数据

 

$ rviz

 

4.astra pro获取彩色图像  带彩色的pointCloud2

因为astra的驱动无发布彩色图像，导致点云数据也无彩色数据.

有三种思路取解决：

  1.修改官方驱动

  2.修改ros的底层astra_camera 和 rgbd_launch 的适配

  3.另外新开一个节点监听无彩色pointCloud2数据和彩色数据，用彩色数据的rgb通道数值填充pointCloud2的rgbd的通道，在重新发布出来就是彩色的点云数据

 

这里采用第二种做法，第一做法难度大，第三做法耗电脑资源

4.1获取彩色图像

    这里采用ros的uvc_camera包来获取彩色图像

$ sudo apt-get install uvc_camera

新建一个launch文件来启动uvc_camera

<launch>
  <node pkg="uvc_camera" type="uvc_camera_node" name="uvc_camera" output="screen">
        <param name="device" type="string" value="/dev/video1" /> 
        <param name="width" type="int" value="640" />
        <param name="height" type="int" value="480" />
        <param name="fps" type="int" value="30" />
        <param name="frame_id" type="string" value="camera_rgb_optical_frame" />
    </node>
</launch>

/dev/video1时摄像头的设备号,查看一下摄像头的设备号

$ ls /dev

启动上面的launch文件

$ roslauch uvc.launch

使用rqt_image_view查看

当然也可以采用usb_cam来获取彩色图像，参数ros wiki上有

 

4.2 校准彩色摄像头和深度摄像头

获取彩色点云数据需要对相机进行校准(个人感觉比kinect校准难，校准了好多遍)

校准深度摄像头

$ roslaunch astra_launch astrapro.launch
$ rosrun camera_calibration cameracalibrator.py image:=/camera/ir/image camera:=/camera/ir --size 8x6 --square 0.024

把校准得到的yaml文件的camera_name值设为 camera_name: depth_Astra_Orbbec

校准彩色摄像头

$ rosrun uvc_camera uvc_camera_node device:=/dev/video1 
$ rosrun camera_calibration cameracalibrator.py image:=/image_raw camera:=/ --size 8x6 --square 0.024

把校准得到的yaml文件的camera_name值设为 camera_name: rgb_Astra_Orbbec

 

创建launch文件

<launch>

  <!-- "camera" should uniquely identify the device. All topics are pushed down
       into the "camera" namespace, and it is prepended to tf frame ids. -->
  <arg name="camera" default="camera" />
  <arg name="rgb_frame_id"   default="$(arg camera)_rgb_optical_frame" />
  <arg name="depth_frame_id" default="$(arg camera)_depth_optical_frame" />

  <!-- device_id can have the following formats:
         "#n"            : the nth device found, starts from 1
         "2@n"           : the nth device on USB bus 2, n starts from 1
         "2bc5/0401@1/6" : uri in format <vendor ID>/<product ID>@<bus number>/<device number>
         "15120410023"   : serial number -->
  <arg name="device_id" default="#1" />
  <arg name="bootorder" default="0" />
  <arg name="devnums" default="1" />

  <!-- By default, calibrations are stored to file://${ROS_HOME}/camera_info/${NAME}.yaml,
       where ${NAME} is of the form "[rgb|depth]_[serial#]", e.g. "depth_B00367707227042B".
       See camera_info_manager docs for calibration URL details. -->
  <!--<arg name="rgb_camera_info_url"   default="" />
  <arg name="depth_camera_info_url" default="" />-->

  <arg name="rgb_camera_info_url"   default="file://$(find drive)/rgb.yaml" />
  <arg name="depth_camera_info_url" default="file://$(find drive)/ir.yaml" />

  <!-- Hardware depth registration -->
  <arg name="depth_registration" default="false" />

  <!-- Driver parameters -->
  <arg name="color_depth_synchronization"     default="false" />
  <arg name="auto_exposure"                   default="true" />
  <arg name="auto_white_balance"              default="true" />

  <!-- Arguments for remapping all device namespaces -->
  <arg name="rgb"              default="rgb" />
  <arg name="ir"               default="ir" />
  <arg name="depth"            default="depth" />

  <!-- Optionally suppress loading the driver nodelet and/or publishing the default tf
       tree. Useful if you are playing back recorded raw data from a bag, or are
       supplying a more accurate tf tree from calibration. -->
  <arg name="load_driver" default="true" />
  <arg name="publish_tf" default="true" />
  <!-- Processing Modules -->
  <arg name="rgb_processing"                  default="true"  />
  <arg name="debayer_processing"              default="false" />
  <arg name="ir_processing"                   default="false" />
  <arg name="depth_processing"                default="true" />
  <arg name="depth_registered_processing"     default="true" />
  <arg name="disparity_processing"            default="false" />
  <arg name="disparity_registered_processing" default="false" />
  <arg name="hw_registered_processing"        default="true" if="$(arg depth_registration)" />
  <arg name="sw_registered_processing"        default="false" if="$(arg depth_registration)" />
  <arg name="hw_registered_processing"        default="false" unless="$(arg depth_registration)" />
  <arg name="sw_registered_processing"        default="true" unless="$(arg depth_registration)" />

  <!-- Disable bond topics by default -->
  <arg name="respawn" default="false" />

  <!-- Worker threads for the nodelet manager -->
  <arg name="num_worker_threads" default="4" />

  <!-- Push down all topics/nodelets into "camera" namespace -->
  <group ns="$(arg camera)">

        <!-- Start nodelet manager -->
        <arg name="manager" value="$(arg camera)_nodelet_manager" />
        <arg name="debug" default="false" /> <!-- Run manager in GDB? -->
        <include file="$(find rgbd_launch)/launch/includes/manager.launch.xml">
          <arg name="name" value="$(arg manager)" />
          <arg name="debug" value="$(arg debug)" />
          <arg name="num_worker_threads"  value="$(arg num_worker_threads)" />
        </include>

    <!-- Load driver -->
    <include file="$(find astra_launch)/launch/includes/device.launch.xml" if="$(arg load_driver)">
      <arg name="manager"                         value="$(arg manager)" />
      <arg name="device_id"                       value="$(arg device_id)" />
      <arg name="bootorder"                       value="$(arg bootorder)" />
      <arg name="devnums"                         value="$(arg devnums)" />
      <arg name="rgb_frame_id"                    value="$(arg rgb_frame_id)" />
      <arg name="depth_frame_id"                  value="$(arg depth_frame_id)" />
      <arg name="rgb_camera_info_url"             value="$(arg rgb_camera_info_url)" />
      <arg name="depth_camera_info_url"           value="$(arg depth_camera_info_url)" />
      <arg name="rgb"                             value="$(arg rgb)" />
      <arg name="ir"                              value="$(arg ir)" />
      <arg name="depth"                           value="$(arg depth)" />
      <arg name="respawn"                         value="$(arg respawn)" />
      <arg name="depth_registration"              value="$(arg depth_registration)" />
      <arg name="color_depth_synchronization"     value="$(arg color_depth_synchronization)" />
      <arg name="auto_exposure"                   value="$(arg auto_exposure)" />
      <arg name="auto_white_balance"              value="$(arg auto_white_balance)" />
    </include>

    <!-- Load standard constellation of processing nodelets -->
    <include file="$(find rgbd_launch)/launch/includes/processing.launch.xml">
      <arg name="manager"                         value="$(arg manager)" />
      <arg name="rgb"                             value="$(arg rgb)" />
      <arg name="ir"                              value="$(arg ir)" />
      <arg name="depth"                           value="$(arg depth)" />
      <arg name="respawn"                         value="$(arg respawn)" />
      <arg name="rgb_processing"                  value="$(arg rgb_processing)" />
      <arg name="debayer_processing"              value="$(arg debayer_processing)" />
      <arg name="ir_processing"                   value="$(arg ir_processing)" />
      <arg name="depth_processing"                value="$(arg depth_processing)" />
      <arg name="depth_registered_processing"     value="$(arg depth_registered_processing)" />
      <arg name="disparity_processing"            value="$(arg disparity_processing)" />
      <arg name="disparity_registered_processing" value="$(arg disparity_registered_processing)" />
      <arg name="hw_registered_processing"        value="$(arg hw_registered_processing)" />
      <arg name="sw_registered_processing"        value="$(arg sw_registered_processing)" />
    </include>


  <node pkg="uvc_camera" type="uvc_camera_node" name="uvc_camera" output="screen">
        <param name="device"  value="/dev/video1" /> 
        <param name="camera_info_url"  value="$(arg rgb_camera_info_url)" />
        <param name="camera_name" value="rgb_Astra_Orbbec"/>
        <param name="width"  value="640" />
        <param name="height"  value="480" />
        <param name="fps" value="30" />
        <param name="frame_id"  value="camera_rgb_optical_frame" />
        
        <remap from="/camera/camera_info" to="/camera/rgb/camera_info"/>
        <remap from="/camera/image_raw" to="/camera/rgb/image_raw"/>
    </node>
  </group> <!-- camera -->

  <!-- Load reasonable defaults for the relative pose between cameras -->
  <include file="$(find astra_launch)/launch/includes/astra_frames.launch" if="$(arg publish_tf)">
    <arg name="camera" value="$(arg camera)" />
  </include>


</launch>

 

自行修改ir 和 rgb两个文件的路径，启动这个launch文件打开rviz订阅/camera/depth_registered/points就可以看到彩色点云了

 

总结：后期可以利用这些数据进行opencv tensorFlow 开发，及move_base的导航