ROS二维码识别以及OKR使用

二维码识别(ar_track_alvar)
安装
sudo apt-get install ros-kinetic-ar-track-alvar
生成二维码
rosrun ar_track_alvar createMarker AR_ID
eg:
rosrun ar_track_alvar createMarker 0
查看帮助
rosrun ar_track_alvar createMarker
创建一系列二维码标签
roscd robot_vision/config
rosrun ar_track_alvar createMarker -s 5 0
rosrun ar_track_alvar createMarker -s 5 1
rosrun ar_track_alvar createMarker -s 5 2

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

摄像头识别二维码
ar_track_alvar功能包支持USB和RGB-D摄像头作为识别二维码的视觉传感器,分别对应于individualMarkersNoKinect和invidualMarkers两个不同的识别节点。

asber@asber-X550VX:~$ rosrun ar_track_alvar 
createMarker               individualMarkers
findMarkerBundles          individualMarkersNoKinect
findMarkerBundlesNoKinect  trainMarkerBundle

因为不论是二维码还是物品都要坐标变换到全局
我们按照这篇文章教我们的来实践一下。

    <launch>
        <node pkg="tf" type="static_transform_publisher" name="world_to_cam" args="0 0 0 0 0 0 map usb_cam 10" />
        //map usb_cam 都是 frame_id. 这里切记 args 里边的结果 x,y,z 以及rpy会累加计算到

        <arg name="marker_size" default="12.0" />
        <arg name="max_new_marker_error" default="0.05" />
        <arg name="max_track_error" default="0.05" />
        <arg name="cam_image_topic" default="/usb_cam/image_raw" />
        <arg name="cam_info_topic" default="/usb_cam/camera_info" />
        <arg name="output_frame" default="/map" />
        <node name="ar_track_alvar" pkg="ar_track_alvar" type="individualMarkersNoKinect" respawn="false" output="screen">
            <param name="marker_size" type="double" value="$(arg marker_size)" />
            <param name="max_new_marker_error" type="double" value="$(arg max_new_marker_error)" />
            <param name="max_track_error" type="double" value="$(arg max_track_error)" />
            <param name="output_frame" type="string" value="$(arg output_frame)" />
            <remap from="camera_image" to="$(arg cam_image_topic)" />
            <remap from="camera_info" to="$(arg cam_info_topic)" />
        </node>
    </launch>

修改为

<launch>
	<arg name="marker_size" default="50" />  <!--定义marker最外框的尺寸,注意单位是厘米-->
	<arg name="max_new_marker_error" default="0.09" /> 
	<arg name="max_track_error" default="0.2" />
	<arg name="cam_image_topic" default="/camera/rgb/image_raw" /> <!--修改为自己发布的图像话题-->
	<arg name="cam_info_topic" default="/camera/rgb/camera_info" /> <!--修改为自己发布的标定参数话题-->
	<arg name="output_frame" default="/camera_link" /> <!--修改为图片所在的坐标系,关系到后续的坐标系自动转换-->

	<node name="ar_track_alvar" pkg="ar_track_alvar" type="individualMarkersNoKinect" respawn="false" output="screen"
	 args="$(arg marker_size) $(arg max_new_marker_error) $(arg max_track_error) $(arg cam_image_topic) $(arg cam_info_topic) $(arg output_frame)" />
</launch>

rosrun ar_track_alvar createMarker -1 toilet
rosrun ar_track_alvar createMarker -1 office_of_ye
rosrun ar_track_alvar createMarker -1 my_workspace

在这里插入图片描述
启动你的摄像头,启动标签检测节点,就可以看到有话题发布出来了,观察标签话题
在gazebo里面我们弄出一个带标签的box
https://zhuanlan.zhihu.com/p/92453712

<static>true</static>

把这句话插入link节点的上方,就可以使模型不能移动——https://blog.csdn.net/xk_t9_98568/article/details/21116575
下面是最后的sdf文件 可以看到我把牌子设定为0.496m的宽度

<?xml version='1.0'?>
<sdf version='1.4'>
  <model name="mark_label">
    <static>true</static>
    <link name='link'>
      <pose>0 0 0.115 0 0 0</pose>
      <inertial>
        <mass>0.390</mass>
        <inertia>
          <ixx>0.00058</ixx>
          <ixy>0</ixy>
          <ixz>0</ixz>
          <iyy>0.00058</iyy>
          <iyz>0</iyz>
          <izz>0.00019</izz>
        </inertia>
      </inertial>
      <collision name='collision'>
        <geometry>
                <box>
                  <size>.496 .496 .01</size>
                </box>
        </geometry>
      </collision>

      <visual name='visual'>
        <geometry>
                <box>
                  <size>.496 .496 .01</size>
                </box>
        </geometry>

        <material>
          <script>
            <uri>model://mark_label/materials/scripts</uri>
            <uri>model://mark_label/materials/textures</uri>
            <name>Mark/Diffuse</name>
          </script>
        </material>
      </visual>
    </link>     
  </model>
</sdf>

在这里插入图片描述虽然在gazebo中发现并不是很好的识别,但是在真实环境下使用电脑摄像头进行标定之后,识别效果很不错
/ar_pose_marker 这个topic列出了

---
header: 
  seq: 1046
  stamp: 
    secs: 0
    nsecs:         0
  frame_id: ''
markers: 
  - 
    header: 
      seq: 0
      stamp: 
        secs: 1583043220
        nsecs: 292708382
      frame_id: "/usb_cam"
    id: 0
    confidence: 0
    pose: 
      header: 
        seq: 0
        stamp: 
          secs: 0
          nsecs:         0
        frame_id: ''
      pose: 
        position: 
          x: 0.0454303788343
          y: 0.0435731577591
          z: 0.713923954759
        orientation: 
          x: 0.0652061198575
          y: 0.984560100669
          z: -0.137542302708
          w: 0.0864389094638
---

如果检测到多个会出现

header: 
  seq: 175
  stamp: 
    secs: 0
    nsecs:         0
  frame_id: ''
markers: 
  - 
    header: 
      seq: 0
      stamp: 
        secs: 1583043843
        nsecs: 832021582
      frame_id: "/usb_cam"
    id: 13
    confidence: 0
    pose: 
      header: 
        seq: 0
        stamp: 
          secs: 0
          nsecs:         0
        frame_id: ''
      pose: 
        position: 
          x: -0.225492470716
          y: -0.0751703811534
          z: 2.33414874325
        orientation: 
          x: 0.968313888783
          y: 0.185389896448
          z: -0.0393726493893
          w: 0.162630235703
  - 
    header: 
      seq: 0
      stamp: 
        secs: 1583043843
        nsecs: 832021582
      frame_id: "/usb_cam"
    id: 16
    confidence: 0
    pose: 
      header: 
        seq: 0
        stamp: 
          secs: 0
          nsecs:         0
        frame_id: ''
      pose: 
        position: 
          x: -0.326423448551
          y: 0.167249999227
          z: 2.2873627328
        orientation: 
          x: 0.96040035328
          y: 0.190934984989
          z: 0.168130902833
          w: -0.113608945244

这样的情况(2个为例)
这里的frame_id都是/usb_cam
id: 16代表检测到16这个数字的arcode
之后测试如果是toilet这样的arcode id是什么
总之我们可以检测到了。下面我们

<launch>

    <node pkg="tf" type="static_transform_publisher" name="world_to_cam" 
          args="0 0 0.5 0 1.57 0 world usb_cam 10" />
        
    <arg name="marker_size" default="20" />
    <arg name="max_new_marker_error" default="0.08" />
    <arg name="max_track_error" default="0.2" />
    <arg name="cam_image_topic" default="/usb_cam/image_raw" />
    <arg name="cam_info_topic" default="/usb_cam/camera_info" />
    <arg name="output_frame" default="/usb_cam" />
        
    <node name="ar_track_alvar" pkg="ar_track_alvar" type="individualMarkersNoKinect" respawn="false" output="screen">
        <param name="marker_size"           type="double" value="$(arg marker_size)" />
        <param name="max_new_marker_error"  type="double" value="$(arg max_new_marker_error)" />
        <param name="max_track_error"       type="double" value="$(arg max_track_error)" />
        <param name="output_frame"          type="string" value="$(arg output_frame)" />

        <remap from="camera_image"  to="$(arg cam_image_topic)" />
        <remap from="camera_info"   to="$(arg cam_info_topic)" />
    </node>

    <!-- rviz view /-->
    <node pkg="rviz" type="rviz" name="rviz" args="-d $(find robot_vision)/config/ar_track_camera.rviz"/>

</launch>

接下去花另一csdn的章节来记录tf变换获取二维码的world pose。
https://zhuanlan.zhihu.com/p/59291168

参考:
https://blog.csdn.net/qq_23670601/article/details/91612289
http://chinaunix.net/uid-27875-id-5823988.html
http://wiki.ros.org/ar_track_alvar/
https://blog.csdn.net/zzu_seu/article/details/89289556


物体识别框架(OKR, Object Recognition Kitchen)
包含了多种三维物体识别的方法。
sudo apt-get install ros-indigo-object-recognition-kitchen-* 备注:没有安装成功(ubuntu16.04)
kinetic版本下的安装:
1)安装依赖库
sudo apt-get install meshlab
sudo apt-get install libosmesa6-dev
sudo apt-get install python-pyside.qtcore
sudo apt-get install python-pyside.qtgui
2)下载源码安装
mkdir ork_ws && cd ork_ws (我是在另一个文件夹 不是catkin_ws)
wstool init src https://raw.github.com/wg-perception/object_recognition_core/master/doc/source/ork.rosinstall.kinetic.plus
cd src && wstool update -j8
cd … && rosdep install --from-paths src -i -y
cd src && git clone https://github.com/jbohren/xdot.git
cd … && catkin_make
echo “source ~/ork_ws/devel/setup.bash” >> ~/.bashrc
source ~/.bashrc

该命令会安装OKR中以下功能包:
1)object-recognition-core: 核心功能包,提供多种物体识别的算法,以及模型训练和模型数据库配置的工具
2)object-recognition-linemod:基于OpenCV中linemod方法的物体识别,擅长刚性物体的识别
3)object-recognition-tabletop:用于同一平面上pick-and-place操作中的物体识别方法
4)object-recognition-tod: Textured Object Recognition,基于物体外部文理的识别方法
5)object-recognition-reconstruction: 使用RGB-D摄像头构建物体3D模型
6)object-recognition-renderer: 渲染物体模型的可视化显示
7)object-recognition-msgs:定义object-recognition-core功能包中所需要的message和action
8)object-recognition-capture: 从3D视图中获取物体信息
9)object-recognition-transparent-objects:识别和估计物体的位姿
10)object-recognition-ros-visualization: 物体识别可视化显示的rviz插件

建立物体模型库
安装
sudo apt-get install couchdb
测试是否安装完成
curl -X GET http://localhost:5984

{“couchdb”:“Welcome”,“uuid”:“81d25f44640bec653fe2487894749418”,“version”:“1.6.0”,“vendor”:{“name”:“Ubuntu”,“version”:“15.10”}}

创建一条可乐罐模型
rosrun object_recognition_core object_add.py -n "coke " -d “A universal can of coke” --commit
浏览器中打开网址http://localhost:5984/_utils/database.html?object_recognition/_design/objects/_view/by_object_name可以看到
在这里插入图片描述 复制ID“be4814630fa4e7a722eecd7f1c000ae3”
下载可乐罐模型(注意下载的路径,把该文件包下载到ork_ws/src/下)
git clone https://github.com/wg-perception/ork_tutorials
rosrun object_recognition_core mesh_add.py be4814630fa4e7a722eecd7f1c000ae3 ~/ork_ws/src/ork_tutorials/data/coke.stl --commit
安装couchapp工具,在浏览器中查看具体模型
sudo pip install git+https://github.com/couchapp/couchapp.git
若执行上个命令,提示如下信息,则需要安装pip

 sudo apt-get update
 sudo apt-get upgrade
sudo apt-get install python-pip

rosrun object_recognition_core push.sh
在浏览器中查看: http://localhost:5984/or_web_ui/_design/viewer/index.html
在这里插入图片描述
点击object_listing,查看数据列表
点击mesh,查看模型,点击后,如下图:
(本应该如下图 但是我没看到)
在这里插入图片描述

rosrun object_recognition_core training -c `rospack find object_recognition_linemod`/conf/training.ork
Training 1 objects.
computing object_id: be4814630fa4e7a722eecd7f1c000ae3
Skipping object id "be4814630fa4e7a722eecd7f1c000ae3" : no mesh in the DB
Deleting the previous model be4814630fa4e7a722eecd7f1c001c0e of object be4814630fa4e7a722eecd7f1c000ae3
这样提示的话就是不对 没有载入,检查上面的步骤再实行一遍
正确提示:
Training 1 objects.
computing object_id: be4814630fa4e7a722eecd7f1c000ae3
Info,  T0: Load /tmp/fileCZwTVI.stl
Info,  T0: Found a matching importer for this file format
Info,  T0: Import root directory is '/tmp/'
Info,  T0: Entering post processing pipeline
Info,  T0: Points: 0, Lines: 0, Triangles: 1, Polygons: 0 (Meshes, X = removed)
Error, T0: FindInvalidDataProcess fails on mesh normals: Found zero-length vector
Info,  T0: FindInvalidDataProcess finished. Found issues ...
Info,  T0: GenVertexNormalsProcess finished. Vertex normals have been calculated
Error, T0: Failed to compute tangents; need UV data in channel0
Info,  T0: JoinVerticesProcess finished | Verts in: 1536 out: 258 | ~83.2%
Info,  T0: Cache relevant are 1 meshes (512 faces). Average output ACMR is 0.669922
Info,  T0: Leaving post processing pipeline
Deleting the previous model be4814630fa4e7a722eecd7f1c002927 of object be4814630fa4e7a722eecd7f1c000ae3

开启仿真或者Kinect
这里以我仿真为例,roslaunch turtlebot_gazebo turtlebot_world.launch发出的topic有

/camera/depth/camera_info
/camera/depth/image_raw
/camera/rgb/camera_info
/camera/rgb/image_raw

修改一下 detection.ros.ork 订阅的话题名

gedit `rospack find object_recognition_linemod`/conf/detection.ros.ork
原来是
source1:
  type: RosKinect
  module: 'object_recognition_ros.io'
  parameters:
    rgb_frame_id: 'camera_rgb_optical_frame'
    rgb_image_topic: '/camera/rgb/image_rect_color'
    rgb_camera_info: '/camera/rgb/camera_info'
    depth_frame_id: 'camera_depth_optical_frame'
    depth_image_topic: '/camera/depth_registered/image_raw'
    depth_camera_info: '/camera/depth_registered/camera_info'
改为
source1:
  type: RosKinect
  module: 'object_recognition_ros.io'
  parameters:
    rgb_frame_id: 'camera_rgb_optical_frame'
    rgb_image_topic: '/camera/rgb/image_raw'
    rgb_camera_info: '/camera/rgb/camera_info'
    depth_frame_id: 'camera_depth_optical_frame'
    depth_image_topic: '/camera/depth/image_raw'
    depth_camera_info: '/camera/depth/camera_info'

然后运行

rosrun object_recognition_core detection -c  `rospack find object_recognition_linemod`/conf/detection.ros.ork

rviz
添加 Pointcloud2,选择话题 /camera/depth/color/points
添加 OrkObject,选择话题 /recognized_object_array (若能够成功识别,则会发布此话题)
查看 /recognized_object_array 话题的实时内容
rostopic echo /recognized_object_array

在这里插入图片描述
就会得到pose 那么经过tf变换就可以将可乐的位置提出来。

---
header: 
  seq: 1809
  stamp: 
    secs: 2491
    nsecs: 330000000
  frame_id: "camera_depth_optical_frame"
objects: 
  - 
    header: 
      seq: 0
      stamp: 
        secs: 2491
        nsecs: 330000000
      frame_id: "camera_depth_optical_frame"
    type: 
      key: "be4814630fa4e7a722eecd7f1c000ae3"
      db: "{\"collection\":\"object_recognition\",\"root\":\"http://localhost:5984\",\"type\"\
  :\"CouchDB\"}"
    confidence: 93.6507949829
    point_clouds: []
    bounding_mesh: 
      triangles: []
      vertices: []
    bounding_contours: []
    pose: 
      header: 
        seq: 0
        stamp: 
          secs: 2491
          nsecs: 330000000
        frame_id: "camera_depth_optical_frame"
      pose: 
        pose: 
          position: 
            x: 0.180100440979
            y: 0.13609431684
            z: 1.55801010132
          orientation: 
            x: 0.556165874004
            y: -0.402824014425
            z: 0.429271042347
            w: 0.586633265018
        covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
cooccurrence: []
---

在这里插入图片描述

此处有使用教程:ROS kinetic + Realsens D435i + ORK + LINEMOD 物体识别
官方http://wg-perception.github.io/object_recognition_core/

感谢:
Ubuntu16.04+ ROS kinetic 使用kinect2 ORK功能包 linemod算法实现可乐罐识别
ROS IntelRealSenseZR300 PCL+ORK+Linemod 柱状物体检测 机械臂抓取 系列第二篇
KinectV2测试物体识别linemod算法 Ubuntu16 ROS-kinetic
通过ROS控制真实机械臂(17) — 视觉抓取之ORK实现三维物体识别
ORK包的安装与linemod算法识别测试 (使用kinect v2 出现很奇怪的问题和解决)
D435、linemod实现三维物体检测与识别

那么哪里可以下到更多的STL库 来做点云识别呢?

评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值