二维码识别(ar_track_alvar)
安装
sudo apt-get install ros-kinetic-ar-track-alvar
生成二维码
rosrun ar_track_alvar createMarker AR_ID
eg:
rosrun ar_track_alvar createMarker 0
查看帮助
rosrun ar_track_alvar createMarker
创建一系列二维码标签
roscd robot_vision/config
rosrun ar_track_alvar createMarker -s 5 0
rosrun ar_track_alvar createMarker -s 5 1
rosrun ar_track_alvar createMarker -s 5 2
摄像头识别二维码
ar_track_alvar功能包支持USB和RGB-D摄像头作为识别二维码的视觉传感器,分别对应于individualMarkersNoKinect和invidualMarkers两个不同的识别节点。
asber@asber-X550VX:~$ rosrun ar_track_alvar
createMarker individualMarkers
findMarkerBundles individualMarkersNoKinect
findMarkerBundlesNoKinect trainMarkerBundle
因为不论是二维码还是物品都要坐标变换到全局
我们按照这篇文章教我们的来实践一下。
<launch>
<node pkg="tf" type="static_transform_publisher" name="world_to_cam" args="0 0 0 0 0 0 map usb_cam 10" />
//map usb_cam 都是 frame_id. 这里切记 args 里边的结果 x,y,z 以及rpy会累加计算到
<arg name="marker_size" default="12.0" />
<arg name="max_new_marker_error" default="0.05" />
<arg name="max_track_error" default="0.05" />
<arg name="cam_image_topic" default="/usb_cam/image_raw" />
<arg name="cam_info_topic" default="/usb_cam/camera_info" />
<arg name="output_frame" default="/map" />
<node name="ar_track_alvar" pkg="ar_track_alvar" type="individualMarkersNoKinect" respawn="false" output="screen">
<param name="marker_size" type="double" value="$(arg marker_size)" />
<param name="max_new_marker_error" type="double" value="$(arg max_new_marker_error)" />
<param name="max_track_error" type="double" value="$(arg max_track_error)" />
<param name="output_frame" type="string" value="$(arg output_frame)" />
<remap from="camera_image" to="$(arg cam_image_topic)" />
<remap from="camera_info" to="$(arg cam_info_topic)" />
</node>
</launch>
修改为
<launch>
<arg name="marker_size" default="50" /> <!--定义marker最外框的尺寸,注意单位是厘米-->
<arg name="max_new_marker_error" default="0.09" />
<arg name="max_track_error" default="0.2" />
<arg name="cam_image_topic" default="/camera/rgb/image_raw" /> <!--修改为自己发布的图像话题-->
<arg name="cam_info_topic" default="/camera/rgb/camera_info" /> <!--修改为自己发布的标定参数话题-->
<arg name="output_frame" default="/camera_link" /> <!--修改为图片所在的坐标系,关系到后续的坐标系自动转换-->
<node name="ar_track_alvar" pkg="ar_track_alvar" type="individualMarkersNoKinect" respawn="false" output="screen"
args="$(arg marker_size) $(arg max_new_marker_error) $(arg max_track_error) $(arg cam_image_topic) $(arg cam_info_topic) $(arg output_frame)" />
</launch>
rosrun ar_track_alvar createMarker -1 toilet
rosrun ar_track_alvar createMarker -1 office_of_ye
rosrun ar_track_alvar createMarker -1 my_workspace
启动你的摄像头,启动标签检测节点,就可以看到有话题发布出来了,观察标签话题
在gazebo里面我们弄出一个带标签的box
https://zhuanlan.zhihu.com/p/92453712
<static>true</static>
把这句话插入link节点的上方,就可以使模型不能移动——https://blog.csdn.net/xk_t9_98568/article/details/21116575
下面是最后的sdf文件 可以看到我把牌子设定为0.496m的宽度
<?xml version='1.0'?>
<sdf version='1.4'>
<model name="mark_label">
<static>true</static>
<link name='link'>
<pose>0 0 0.115 0 0 0</pose>
<inertial>
<mass>0.390</mass>
<inertia>
<ixx>0.00058</ixx>
<ixy>0</ixy>
<ixz>0</ixz>
<iyy>0.00058</iyy>
<iyz>0</iyz>
<izz>0.00019</izz>
</inertia>
</inertial>
<collision name='collision'>
<geometry>
<box>
<size>.496 .496 .01</size>
</box>
</geometry>
</collision>
<visual name='visual'>
<geometry>
<box>
<size>.496 .496 .01</size>
</box>
</geometry>
<material>
<script>
<uri>model://mark_label/materials/scripts</uri>
<uri>model://mark_label/materials/textures</uri>
<name>Mark/Diffuse</name>
</script>
</material>
</visual>
</link>
</model>
</sdf>
虽然在gazebo中发现并不是很好的识别,但是在真实环境下使用电脑摄像头进行标定之后,识别效果很不错
/ar_pose_marker 这个topic列出了
---
header:
seq: 1046
stamp:
secs: 0
nsecs: 0
frame_id: ''
markers:
-
header:
seq: 0
stamp:
secs: 1583043220
nsecs: 292708382
frame_id: "/usb_cam"
id: 0
confidence: 0
pose:
header:
seq: 0
stamp:
secs: 0
nsecs: 0
frame_id: ''
pose:
position:
x: 0.0454303788343
y: 0.0435731577591
z: 0.713923954759
orientation:
x: 0.0652061198575
y: 0.984560100669
z: -0.137542302708
w: 0.0864389094638
---
如果检测到多个会出现
header:
seq: 175
stamp:
secs: 0
nsecs: 0
frame_id: ''
markers:
-
header:
seq: 0
stamp:
secs: 1583043843
nsecs: 832021582
frame_id: "/usb_cam"
id: 13
confidence: 0
pose:
header:
seq: 0
stamp:
secs: 0
nsecs: 0
frame_id: ''
pose:
position:
x: -0.225492470716
y: -0.0751703811534
z: 2.33414874325
orientation:
x: 0.968313888783
y: 0.185389896448
z: -0.0393726493893
w: 0.162630235703
-
header:
seq: 0
stamp:
secs: 1583043843
nsecs: 832021582
frame_id: "/usb_cam"
id: 16
confidence: 0
pose:
header:
seq: 0
stamp:
secs: 0
nsecs: 0
frame_id: ''
pose:
position:
x: -0.326423448551
y: 0.167249999227
z: 2.2873627328
orientation:
x: 0.96040035328
y: 0.190934984989
z: 0.168130902833
w: -0.113608945244
这样的情况(2个为例)
这里的frame_id都是/usb_cam
id: 16代表检测到16这个数字的arcode
之后测试如果是toilet这样的arcode id是什么
总之我们可以检测到了。下面我们
<launch>
<node pkg="tf" type="static_transform_publisher" name="world_to_cam"
args="0 0 0.5 0 1.57 0 world usb_cam 10" />
<arg name="marker_size" default="20" />
<arg name="max_new_marker_error" default="0.08" />
<arg name="max_track_error" default="0.2" />
<arg name="cam_image_topic" default="/usb_cam/image_raw" />
<arg name="cam_info_topic" default="/usb_cam/camera_info" />
<arg name="output_frame" default="/usb_cam" />
<node name="ar_track_alvar" pkg="ar_track_alvar" type="individualMarkersNoKinect" respawn="false" output="screen">
<param name="marker_size" type="double" value="$(arg marker_size)" />
<param name="max_new_marker_error" type="double" value="$(arg max_new_marker_error)" />
<param name="max_track_error" type="double" value="$(arg max_track_error)" />
<param name="output_frame" type="string" value="$(arg output_frame)" />
<remap from="camera_image" to="$(arg cam_image_topic)" />
<remap from="camera_info" to="$(arg cam_info_topic)" />
</node>
<!-- rviz view /-->
<node pkg="rviz" type="rviz" name="rviz" args="-d $(find robot_vision)/config/ar_track_camera.rviz"/>
</launch>
接下去花另一csdn的章节来记录tf变换获取二维码的world pose。
https://zhuanlan.zhihu.com/p/59291168
参考:
https://blog.csdn.net/qq_23670601/article/details/91612289
http://chinaunix.net/uid-27875-id-5823988.html
http://wiki.ros.org/ar_track_alvar/
https://blog.csdn.net/zzu_seu/article/details/89289556
物体识别框架(OKR, Object Recognition Kitchen)
包含了多种三维物体识别的方法。
sudo apt-get install ros-indigo-object-recognition-kitchen-* 备注:没有安装成功(ubuntu16.04)
kinetic版本下的安装:
1)安装依赖库
sudo apt-get install meshlab
sudo apt-get install libosmesa6-dev
sudo apt-get install python-pyside.qtcore
sudo apt-get install python-pyside.qtgui
2)下载源码安装
mkdir ork_ws && cd ork_ws (我是在另一个文件夹 不是catkin_ws)
wstool init src https://raw.github.com/wg-perception/object_recognition_core/master/doc/source/ork.rosinstall.kinetic.plus
cd src && wstool update -j8
cd … && rosdep install --from-paths src -i -y
cd src && git clone https://github.com/jbohren/xdot.git
cd … && catkin_make
echo “source ~/ork_ws/devel/setup.bash” >> ~/.bashrc
source ~/.bashrc
该命令会安装OKR中以下功能包:
1)object-recognition-core: 核心功能包,提供多种物体识别的算法,以及模型训练和模型数据库配置的工具
2)object-recognition-linemod:基于OpenCV中linemod方法的物体识别,擅长刚性物体的识别
3)object-recognition-tabletop:用于同一平面上pick-and-place操作中的物体识别方法
4)object-recognition-tod: Textured Object Recognition,基于物体外部文理的识别方法
5)object-recognition-reconstruction: 使用RGB-D摄像头构建物体3D模型
6)object-recognition-renderer: 渲染物体模型的可视化显示
7)object-recognition-msgs:定义object-recognition-core功能包中所需要的message和action
8)object-recognition-capture: 从3D视图中获取物体信息
9)object-recognition-transparent-objects:识别和估计物体的位姿
10)object-recognition-ros-visualization: 物体识别可视化显示的rviz插件
建立物体模型库
安装
sudo apt-get install couchdb
测试是否安装完成
curl -X GET http://localhost:5984
{“couchdb”:“Welcome”,“uuid”:“81d25f44640bec653fe2487894749418”,“version”:“1.6.0”,“vendor”:{“name”:“Ubuntu”,“version”:“15.10”}}
创建一条可乐罐模型
rosrun object_recognition_core object_add.py -n "coke " -d “A universal can of coke” --commit
浏览器中打开网址http://localhost:5984/_utils/database.html?object_recognition/_design/objects/_view/by_object_name可以看到
复制ID“be4814630fa4e7a722eecd7f1c000ae3”
下载可乐罐模型(注意下载的路径,把该文件包下载到ork_ws/src/下)
git clone https://github.com/wg-perception/ork_tutorials
rosrun object_recognition_core mesh_add.py be4814630fa4e7a722eecd7f1c000ae3 ~/ork_ws/src/ork_tutorials/data/coke.stl --commit
安装couchapp工具,在浏览器中查看具体模型
sudo pip install git+https://github.com/couchapp/couchapp.git
若执行上个命令,提示如下信息,则需要安装pip
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python-pip
rosrun object_recognition_core push.sh
在浏览器中查看: http://localhost:5984/or_web_ui/_design/viewer/index.html
点击object_listing,查看数据列表
点击mesh,查看模型,点击后,如下图:
(本应该如下图 但是我没看到)
rosrun object_recognition_core training -c `rospack find object_recognition_linemod`/conf/training.ork
Training 1 objects.
computing object_id: be4814630fa4e7a722eecd7f1c000ae3
Skipping object id "be4814630fa4e7a722eecd7f1c000ae3" : no mesh in the DB
Deleting the previous model be4814630fa4e7a722eecd7f1c001c0e of object be4814630fa4e7a722eecd7f1c000ae3
这样提示的话就是不对 没有载入,检查上面的步骤再实行一遍
正确提示:
Training 1 objects.
computing object_id: be4814630fa4e7a722eecd7f1c000ae3
Info, T0: Load /tmp/fileCZwTVI.stl
Info, T0: Found a matching importer for this file format
Info, T0: Import root directory is '/tmp/'
Info, T0: Entering post processing pipeline
Info, T0: Points: 0, Lines: 0, Triangles: 1, Polygons: 0 (Meshes, X = removed)
Error, T0: FindInvalidDataProcess fails on mesh normals: Found zero-length vector
Info, T0: FindInvalidDataProcess finished. Found issues ...
Info, T0: GenVertexNormalsProcess finished. Vertex normals have been calculated
Error, T0: Failed to compute tangents; need UV data in channel0
Info, T0: JoinVerticesProcess finished | Verts in: 1536 out: 258 | ~83.2%
Info, T0: Cache relevant are 1 meshes (512 faces). Average output ACMR is 0.669922
Info, T0: Leaving post processing pipeline
Deleting the previous model be4814630fa4e7a722eecd7f1c002927 of object be4814630fa4e7a722eecd7f1c000ae3
开启仿真或者Kinect
这里以我仿真为例,roslaunch turtlebot_gazebo turtlebot_world.launch发出的topic有
/camera/depth/camera_info
/camera/depth/image_raw
/camera/rgb/camera_info
/camera/rgb/image_raw
修改一下 detection.ros.ork 订阅的话题名
gedit `rospack find object_recognition_linemod`/conf/detection.ros.ork
原来是
source1:
type: RosKinect
module: 'object_recognition_ros.io'
parameters:
rgb_frame_id: 'camera_rgb_optical_frame'
rgb_image_topic: '/camera/rgb/image_rect_color'
rgb_camera_info: '/camera/rgb/camera_info'
depth_frame_id: 'camera_depth_optical_frame'
depth_image_topic: '/camera/depth_registered/image_raw'
depth_camera_info: '/camera/depth_registered/camera_info'
改为
source1:
type: RosKinect
module: 'object_recognition_ros.io'
parameters:
rgb_frame_id: 'camera_rgb_optical_frame'
rgb_image_topic: '/camera/rgb/image_raw'
rgb_camera_info: '/camera/rgb/camera_info'
depth_frame_id: 'camera_depth_optical_frame'
depth_image_topic: '/camera/depth/image_raw'
depth_camera_info: '/camera/depth/camera_info'
然后运行
rosrun object_recognition_core detection -c `rospack find object_recognition_linemod`/conf/detection.ros.ork
rviz
添加 Pointcloud2,选择话题 /camera/depth/color/points
添加 OrkObject,选择话题 /recognized_object_array (若能够成功识别,则会发布此话题)
查看 /recognized_object_array 话题的实时内容
rostopic echo /recognized_object_array
就会得到pose 那么经过tf变换就可以将可乐的位置提出来。
---
header:
seq: 1809
stamp:
secs: 2491
nsecs: 330000000
frame_id: "camera_depth_optical_frame"
objects:
-
header:
seq: 0
stamp:
secs: 2491
nsecs: 330000000
frame_id: "camera_depth_optical_frame"
type:
key: "be4814630fa4e7a722eecd7f1c000ae3"
db: "{\"collection\":\"object_recognition\",\"root\":\"http://localhost:5984\",\"type\"\
:\"CouchDB\"}"
confidence: 93.6507949829
point_clouds: []
bounding_mesh:
triangles: []
vertices: []
bounding_contours: []
pose:
header:
seq: 0
stamp:
secs: 2491
nsecs: 330000000
frame_id: "camera_depth_optical_frame"
pose:
pose:
position:
x: 0.180100440979
y: 0.13609431684
z: 1.55801010132
orientation:
x: 0.556165874004
y: -0.402824014425
z: 0.429271042347
w: 0.586633265018
covariance: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
cooccurrence: []
---
此处有使用教程:ROS kinetic + Realsens D435i + ORK + LINEMOD 物体识别
官方http://wg-perception.github.io/object_recognition_core/
感谢:
Ubuntu16.04+ ROS kinetic 使用kinect2 ORK功能包 linemod算法实现可乐罐识别
ROS IntelRealSenseZR300 PCL+ORK+Linemod 柱状物体检测 机械臂抓取 系列第二篇
KinectV2测试物体识别linemod算法 Ubuntu16 ROS-kinetic
通过ROS控制真实机械臂(17) — 视觉抓取之ORK实现三维物体识别
ORK包的安装与linemod算法识别测试 (使用kinect v2 出现很奇怪的问题和解决)
D435、linemod实现三维物体检测与识别
那么哪里可以下到更多的STL库 来做点云识别呢?