ros 学习 (4)

 1. rosbag

 参考：

rosbag - ROS Wiki

rosbag/Commandline - ROS Wiki

rosbag/Code API - ROS Wiki

机器人操作系统ROS从入门到放弃(七):使用rosbag - 简书

ROS使用介绍——rosbag_a850565178的博客-CSDN博客_rosbag

rosbag是用来储存ros消息的东西。
在我们平时处理一些robotics问题的时候，我们得到一些数据，有些时候我们需要实时的处理，即获得数据，写好的程序分析数据然后得出结果，但另一些时候我们只是想采集数据，事后分析，这些数据存储在哪儿呢？就存储在rosbag里了。
1）使用rosbag采集数据
2）重新使用储存在rosbag里的数据

rosbag 既可以指命令行中数据包相关命令，也可以指 c++/python 的 rosbag (即c++/python  API)库。
 

bag文件rviz中可视化：

Rosbag使用：bag文件可视化实现_Walliam_Wu的博客-CSDN博客

先rostopic list
rostopic echo xxx

之后rosrun rviz rviz，->add选项之后，在相应选项下的topic进行挑选即可或手动输入（不推荐手动）

2. ros 与vision

2.1 ROS cv_bridge package (Python)

cv_bridge/Tutorials/ConvertingBetweenROSImagesAndOpenCVImagesPython - ROS Wiki

OpenCV Image Processing (Python) — Industrial Training documentation

 

Converting ROS image messages to OpenCV images

To convert a ROS image message into an cv::Mat, module cv_bridge.CvBridge provides the following function:

  1 from cv_bridge import CvBridge
  2 bridge = CvBridge()
  3 cv_image = bridge.imgmsg_to_cv2(image_message, desired_encoding='passthrough')

Converting OpenCV images to ROS image messages

To convert an cv::Mat into a ROS image message, CvBridge provides the following function:

  1 from cv_bridge import CvBridge
  2 bridge = CvBridge()
  3 image_message = bridge.cv2_to_imgmsg(cv_image, encoding="passthrough")

sensor_msgs/Image :

sensor_msgs/Image Documentation

sensor_msgs/Image Message

File: sensor_msgs/Image.msg

Raw Message Definition

# This message contains an uncompressed image
# (0, 0) is at top-left corner of image
#

Header header        # Header timestamp should be acquisition time of image
                     # Header frame_id should be optical frame of camera
                     # origin of frame should be optical center of camera
                     # +x should point to the right in the image
                     # +y should point down in the image
                     # +z should point into to plane of the image
                     # If the frame_id here and the frame_id of the CameraInfo
                     # message associated with the image conflict
                     # the behavior is undefined

uint32 height         # image height, that is, number of rows
uint32 width          # image width, that is, number of columns

# The legal values for encoding are in file src/image_encodings.cpp
# If you want to standardize a new string format, join
# ros-users@lists.sourceforge.net and send an email proposing a new encoding.

string encoding       # Encoding of pixels -- channel meaning, ordering, size
                      # taken from the list of strings in include/sensor_msgs/image_encodings.h

uint8 is_bigendian    # is this data bigendian?
uint32 step           # Full row length in bytes
uint8[] data          # actual matrix data, size is (step * rows)
 

Compact Message Definition

std_msgs/Header header
uint32 height
uint32 width
string encoding
uint8 is_bigendian
uint32 step
uint8[] data

Here is a node that listens to a ROS image message topic, converts the images into an cv::Mat, draws a circle on it and displays the image using OpenCV. The image is then republished over ROS.

In your manifest (alternatively when using roscreate-pkg or catkin_create_pkg), add the following dependencies:

sensor_msgs
opencv2
cv_bridge
rospy
std_msgs

  1 #!/usr/bin/env python
  2 from __future__ import print_function
  3 
  4 import roslib
  5 roslib.load_manifest('my_package')
  6 import sys
  7 import rospy
  8 import cv2
  9 from std_msgs.msg import String
  10 from sensor_msgs.msg import Image
  11 from cv_bridge import CvBridge, CvBridgeError
  12 
  13 class image_converter:
  14 
  15   def __init__(self):
  16     self.image_pub = rospy.Publisher("image_topic_2",Image)
  17 
  18     self.bridge = CvBridge()
  19     self.image_sub = rospy.Subscriber("image_topic",Image,self.callback)
  20 
  21   def callback(self,data):
  22     try:
  23       cv_image = self.bridge.imgmsg_to_cv2(data, "bgr8")
  24     except CvBridgeError as e:
  25       print(e)
  26 
  27     (rows,cols,channels) = cv_image.shape
  28     if cols > 60 and rows > 60 :
  29       cv2.circle(cv_image, (50,50), 10, 255)
  30 
  31     cv2.imshow("Image window", cv_image)
  32     cv2.waitKey(3)
  33 
  34     try:
  35       self.image_pub.publish(self.bridge.cv2_to_imgmsg(cv_image, "bgr8"))
  36     except CvBridgeError as e:
  37       print(e)
  38 
  39 def main(args):
  40   ic = image_converter()
  41   rospy.init_node('image_converter', anonymous=True)
  42   try:
  43     rospy.spin()
  44   except KeyboardInterrupt:
  45     print("Shutting down")
  46   cv2.destroyAllWindows()
  47 
  48 if __name__ == '__main__':
  49     main(sys.argv)

Don't forget to chmod +x your file.

***注意这一步将发布的topic可视化：self.pose_pub.publish(pose) #可视化功能，在rviz中界面操作Add -> By topic ->image_topic_2

2.2 基本的providing an image stream and view it：

• provide an image stream from a live camera on topic /cam/image_raw: roslaunch video_stream_opencv camera.launch video_stream_provider:=/dev/video0 camera_name:=cam visualize:=true
• provide an image stream from a video file on topic /cam/image_raw: roslaunch video_stream_opencv camera.launch video_stream_provider:=video.mp4 camera_name:=cam visualize:=true
• relay a topic (useful to feed link up two pipelines), e.g. rosrun topic_tools relay /thorvald_001/kinect2_left_camera/hd/image_color_rect /cam/image_raw
• view image streams: rqt_image_view #（窗口显示之后，选择对应需要可视化的传感器即对应摄像头）（rviz中add-> camera进行图像可视化：ROS视觉和图像-Kinect V1使用_倔强不倒翁的博客-CSDN博客）
• measure the frequency of an image topic: rostopic hz <topic_name>

#更复杂功能可参考教程：​​​​​​https://blog.csdn.net/weixin_39059031/article/details/84100948

2.3 view image

Viewing a single image topic

to view raw images on the topic /camera/image, use:

rosrun topic_tools relay /thorvald_001/kinect2_left_camera/hd/image_color_rect /cam/image_raw #设定话题或者使用rqt_image_view查看所有话题并指定需要的话题

rosrun image_view image_view image:=/cam/image_raw #对应话题（使用rqt_image_view查看）

You may save the current image by right-clicking on the display window. By default, images will be saved as frame0000.jpg, frame0001.jpg, ....（ctrl+s保存图像）

If you want to view a compressed image stream (usually a good idea over wireless!) using the capabilities of image_transport：

rosrun image_view image_view image:=/cam/image theora

Viewing stereo images

to view stereo image pairs on topics /my_stereo_cam/left/image_rect_color and /my_stereo_cam/right/image_rect_color, use:

rosrun image_view stereo_view stereo:=/my_stereo_cam image:=image_rect_color

image_saver

This tool allows you to save images as jpg/png file from streaming (ROS sensor_msgs/Image topic) to a file. From command line, you can run by rosrun image_view image_saver image:=[your topic]

extract_images

This tool also allows you to save images as jpg/png file from streaming (ROS sensor_msgs/Image topic) to a file. image_saver node provide very similar functionalities, such as providing service call to trigger the node to save images, save images other than Jpeg format, etc.

video_recorder

This tool allows you to record a video stream (ROS sensor_msgs/Image topic) to a file. It relies on OpenCV's VideoWriter class. With the default options, it encodes the video as MPG, encapsulated in a AVI container at 15 fps, and produces a file called output.avi in the current directory.

Some OpenCV nodes readily available

rosrun topic_tools relay /thorvald_001/kinect2_left_camera/hd/image_color_rect /cam/image_raw #设定话题或者使用rqt_image_view查看所有话题并指定需要的话题

• rosrun opencv_apps simple_flow image:=/cam/image_raw
• rosrun opencv_apps find_contours image:=/cam/image_raw
• always view output with rqt_image_view

2D Object Detection

• sudo apt-get install ros-melodic-find-object-2d

rosrun topic_tools relay /thorvald_001/kinect2_left_camera/hd/image_color_rect /cam/image_raw #设定话题或者使用rqt_image_view查看所有话题并指定需要的话题

• run as rosrun find_object_2d find_object_2d image:=/cam/image_raw

YOLO

First, read about YOLO, and even the original paper.

• clone into your workspace： git clone --recursive https://github.com/leggedrobotics/darknet_ros.git
• install its depepndencies: rosdep update; rosdep install --from-paths . -i -y
• #######################################################
• #（注意安装rosdep不要使用sudo apt install python-rosdep2，

• 对于不同版本的ros，使用apt安装方法如下：

  # ROS Noetic
  sudo apt-get install python3-rosdep
  # ROS Melodic and earlier
  sudo apt-get install python-rosdep

  ）参考：Rosdep的安装与使用_Gallant King的博客-CSDN博客_rosdep

###########################################################

• build it:
• cd ~/catkin_ws 
• catkin_make

编译报错“"make -j12 -l12" in "/home/memory/catkin_ws/build"”，需要将darknet_ros/CMakeLists.txt中的

-O3
    -gencode arch=compute_30,code=sm_30
    -gencode arch=compute_35,code=sm_35
    -gencode arch=compute_50,code=[sm_50,compute_50]
    -gencode arch=compute_52,code=[sm_52,compute_52]
    -gencode arch=compute_61,code=sm_61
    -gencode arch=compute_62,code=sm_62

修改为：

-O3
    -gencode arch=compute_61,code=sm_61 #（通过这里结合自己的显卡修改为对应的参数https://en.wikipedia.org/wiki/CUDA#Supported_GPUs）

###rosrun topic_tools relay /thorvald_001/kinect2_left_camera/hd/image_color_rect /cam/image_raw #设定话题或者使用rqt_image_view查看所有话题并指定需要的话题

###或启动摄像头话题roslaunch video_stream_opencv camera.launch video_stream_provider:=/dev/video0 camera_name:=cam visualize:=true

#通过rqt_image_view查看对应想进行检测的图像或摄像头的topic name,然后修改下面的文件

• edit ./darknet_ros/darknet_ros/config/ros.yaml to use the correct image topic, e.g.:

  camera_reading:
    topic: /cam/image_raw
    queue_size: 1

• source your workspace（$ cd ~/catkin_ws/；$ source devel/setup.bash） and run as roslaunch darknet_ros darknet_ros.launch

***Yolo v3移植到ROS后检测结果/darknet_ros/detection_image在rviz中显示乱码或视频中的检测图片不动：

ROS问题：Yolo v4移植到ROS后检测结果/darknet_ros/detection_image在rviz中显示乱码_竭尽全力的专栏-CSDN博客

注意修改cpp文件后需要重新编译yolo.

2.4 Geometry and 3D

图像坐标系，相机坐标系和世界坐标系

OpenCV学习——图像坐标系，相机坐标系和世界坐标系_坚果壳的学习之旅-CSDN博客

相机模型中四个坐标系的关系 - 知乎

​​​​​OpenCV学习之世界坐标系、相机坐标系、图像坐标系和像素坐标系之间的转换关系_xueluowutong的博客-CSDN博客_图像坐标系和像素坐标系

仿射变换与透视变换

（【学习opencv】透视变换 Perspective Transformation/仿射变换_开源节流-CSDN博客）

透视变换(Perspective Transformation)是将图片投影到一个新的视平面(Viewing Plane)，也称作投影映射(Projective Mapping)。

仿射变换就是图像的线性变换加上平移。

通常，在2D平面中，仿射变换的应用较多，而在3D平面中，透视变换又有了自己的一席之地。