ZED相机使用
zed ros wrapper分为三部分:
- zed-ros-wrapper : 提供 ZED ROS Wrapper 节点的主包
- zed-ros-interfaces:声明自定义主题、服务和操作的包
- zed-ros-examples:一个支持包,包含有关如何使用 ZED ROS Wrapper 的示例和教程
启动ZED节点:
- ZED相机:
roslaunch zed_wrapper zed.launch
- ZED 迷你相机:
roslaunch zed_wrapper zedm.launch
- ZED 2相机:
roslaunch zed_wrapper zed2.launch
- ZED 2i相机:
roslaunch zed_wrapper zed2i.launch
设置相机参数:修改文件param/common.yaml(所有相机通用参数)和param/zed2.yaml
参数调整:https://www.stereolabs.com/docs/ros/zed-node/
RVIZ可视化ZED2:roslaunch zed_display_rviz display_zed2.launch
配置RVIZ:https://www.stereolabs.com/docs/ros/rviz/
显示其余ZED数据:https://www.stereolabs.com/docs/ros/
注:
1. rgb/image_rect_color : 颜色校正图像(默认为左传感器)
2. zed/zed_node/point_cloud/cloud_registered:3D彩色点云
3. mapping/fused_cloud:融合颜色点云,仅在启用映时发布
检测目标功能
1.对象检测参数:https://www.stereolabs.com/docs/ros/zed-node/ 对象检测类别及其他API参考:https://www.stereolabs.com/docs/api/python/classpyzed_1_1sl_1_1OBJECT__SUBCLASS.html
(人、车辆、包、动物、电子设备、水果和蔬菜)
model: 要使用的对象检测模块‘0’: MULTI_CLASS_BOX - ‘1’: MULTI_CLASS_BOX_ACCURATE - ‘2’: HUMAN_BODY_FAST - ‘3’: HUMAN_BODY_ACCURATE - ‘4’: MULTI_CLASS_BOX_MEDIUM - ‘5’: HUMAN_BODY_MEDIUM
object_tracking_enabled:启用/禁用对检测到的对象的跟踪 TRUE/FALSE
2.检测到的对象msg:https://www.stereolabs.com/docs/ros/object-detection/
zed_interfaces/ObjectsStamped消息定义为:
#Standard Header
std_msgs/Header header
#Array of `object_stamped` topics
zed_interfaces/Object[] objects
zed_interfaces/Object定义为:
# Object label
string label
# Object label ID
int16 label_id
# Object sub
string sublabel
# Object confidence level (1-99)
float32 confidence
# Object centroid position
float32[3] position
# Position covariance
float32[6] position_covariance
# Object velocity
float32[3] velocity
# Tracking available
bool tracking_available
# Tracking state
# 0 -> OFF (object not valid)
# 1 -> OK
# 2 -> SEARCHING (occlusion occurred, trajectory is estimated)
int8 tracking_state
# Action state
# 0 -> IDLE
# 2 -> MOVING
int8 action_state
# 2D Bounding box projected to Camera image
zed_interfaces/BoundingBox2Di bounding_box_2d
# 3D Bounding box in world frame
zed_interfaces/BoundingBox3D bounding_box_3d
# 3D dimensions (width, height, lenght)
float32[3] dimensions_3d
# Is skeleton available?
bool skeleton_available
# 2D Bounding box projected to Camera image of the person head
zed_interfaces/BoundingBox2Df head_bounding_box_2d
# 3D Bounding box in world frame of the person head
zed_interfaces/BoundingBox3D head_bounding_box_3d
# 3D position of the centroid of the person head
float32[3] head_position
# 2D Person skeleton projected to Camera image
zed_interfaces/Skeleton2D skeleton_2d
# 3D Person skeleton in world frame
zed_interfaces/Skeleton3D skeleton_3d
子消息定义:
zed_interfaces/BoundingBox2Df:
# 0 ------- 1
# | |
# | |
# | |
# 3 ------- 2
zed_interfaces/Keypoint2Df[4] corners
zed_interfaces/BoundingBox2Df:
# 0 ------- 1
# | |
# | |
# | |
# 3 ------- 2
zed_interfaces/Keypoint2Di[4] corners
zed_interfaces/BoundingBox3D:
# 1 ------- 2
# /. /|
# 0 ------- 3 |
# | . | |
# | 5.......| 6
# |. |/
# 4 ------- 7
zed_interfaces/Keypoint3D[8] corners
zed_interfaces/Keypoint2Df:
float32[2] kp
zed_interfaces/Keypoint2Di:
uint32[2] kp
zed_interfaces/Keypoint3D:
float32[3] kp
zed_interfaces/Skeleton2D:
# Skeleton joints indices
# 16-14 15-17
# \ /
# 0
# |
# 2------1------5
# | | | |
# | | | |
# 3 | | 6
# | | | |
# | | | |
# 4 8 11 7
# | |
# | |
# | |
# 9 12
# | |
# | |
# | |
# 10 13
zed_interfaces/Keypoint2Df[18] keypoints
zed_interfaces/Skeleton3D:
# Skeleton joints indices
# 16-14 15-17
# \ /
# 0
# |
# 2------1------5
# | | | |
# | | | |
# 3 | | 6
# | | | |
# | | | |
# 4 8 11 7
# | |
# | |
# | |
# 9 12
# | |
# | |
# | |
# 10 13
zed_interfaces/Keypoint3D[18] keypoints
3.nodelets的使用:https://www.stereolabs.com/docs/ros/zed-nodelets/
4.读取IMU、设备位置路径等信息:https://www.stereolabs.com/docs/ros/sensor-data/以及https://www.stereolabs.com/docs/ros/zed-node/
5.检测人、其他物品
- 参数修改:common.yaml,zed2i.yaml,在zed2i.yaml中model内修改检测目标等
- ZED SDK 检测图像中存在的所有对象并计算它们的 3D 位置和速度。 对象与相机的距离以公制单位(例如米)表示,并从相机左眼后部到场景对象计算。(https://www.stereolabs.com/docs/object-detection/)
6.rosbag存信息及plotjuggler展示
plotjuggler下载时使用:sudo apt install ros-melodic-plotjuggler-ros
而不是sudo apt install ros-melodic-plotjuggler
,才能实时订阅信息显示,否则只能读取bag。
补充信息
https://facontidavide.github.io/PlotJuggler/visualization_howto/index.html
rosbag:
http://wiki.ros.org/rosbag/Commandline
https://guyuehome.com/35777
https://guyuehome.com/34588