ROS学习笔记52（新的图像传输）

1 概述

image_transport真正威力在于其扩展性。

节点使用image_transport进行新的传输，也就是当发布和接受图像时，并不用重新编译。

在这个教程中，我们将会看到：如何写一个传输，关于“compression”。发布时进行下采样，订阅时进行上采样，这样就重新回到原始的尺寸 。

传输分四步：

1. 定义传输的消息类型。For example, a streaming video transport would use some Packet type for internal communication.

2. 写一个发布的插件。This encodes a sensor_msgs/Image into one or more instances of the transport-specific message type and publishes them on the dedicated transport topic.

3. 写一个订阅的插件。This listens on the transport topic and decodes the transport-specific messages back into sensor_msgs/Image messages passed to the user callback.

4. 注册你的传输插件以便image_transport能够使用它。

The above outline assumes that your transport uses a single ROS topic for all publisher-subscriber communication, which should normally be the case. This assumption allows us to use the C++ base classes SimplePublisherPlugin and SimpleSubscriberPlugin. These classes take care of all the scaffolding and bookkeeping so that we can simply get on with writing our encoding and decoding functions. If for some reason your transport requires a more exotic communication structure, it is possible (but much less pleasant!) to subclass the lower-level base classes PublisherPlugin and SubscriberPlugin instead.

The code for that tutorial is in the tutorial folder of image_transport at: https://github.com/ros-perception/image_common.

2 定义消息的类型

定义了一个msg/ResizedImage.msg消息在learning_image_transport包中，如下：

uint32 original_height
uint32 original_width
sensor_msgs/Image image

图像是下采样的图像，而高度和宽度则是原始图像的尺寸，以便于后期图像恢复。

3 写一个简单的发布插件

3.1 程序

创建一个头文件include/image_transport_tutorial/resized_publisher.h，image_transport_tutorial/ResizedImage.h这个头文件是生成消息文件的时候产生的。如下

#include <image_transport/simple_publisher_plugin.h>
#include "learning_image_transport/ResizedImage.h"

class ResizedPublisher : public image_transport::SimplePublisherPlugin<learning_image_transport::ResizedImage>
{
public:
  virtual std::string getTransportName() const
  {
    return "resized";
  }

protected:
  virtual void publish(const sensor_msgs::Image& message,
                       const PublishFn& publish_fn) const;
};

然后另外一个文件src/resized_publisher.cpp

#include <image_transport_tutorial/resized_publisher.h>
#include <opencv2/imgproc/imgproc.hpp>
#include <cv_bridge/cv_bridge.h>

void ResizedPublisher::publish(const sensor_msgs::Image& message,
                               const PublishFn& publish_fn) const
{
  cv::Mat cv_image;
  boost::shared_ptr<void const> tracked_object;
  try
  {
    cv_image = cv_bridge::toCvShare(message, tracked_object, message.encoding)->image;
  }
  catch (cv::Exception &e)
  {
    ROS_ERROR("Could not convert from '%s' to '%s'.", message.encoding.c_str(), message.encoding.c_str());
    return;
  }

  // Retrieve subsampling factor from the parameter server
  double subsampling_factor;
  std::string param_name;
  nh().param<double>("resized_image_transport_subsampling_factor", subsampling_factor, 2.0);

  // Rescale image
  int new_width = cv_image.cols / subsampling_factor + 0.5;
  int new_height = cv_image.rows / subsampling_factor + 0.5;
  cv::Mat buffer;
  cv::resize(cv_image, buffer, cv::Size(new_width, new_height));

  // Set up ResizedImage and publish
  image_transport_tutorial::ResizedImage resized_image;
  resized_image.original_height = cv_image.rows;
  resized_image.original_width = cv_image.cols;
  resized_image.image = *(cv_bridge::CvImage(message.header, "bgr8", cv_image).toImageMsg());
  publish_fn(resized_image);
}

3.2 程序解释

#include <image_transport/simple_publisher_plugin.h>
#include "learning_image_transport/ResizedImage.h"

包含头文件。

class ResizedPublisher : public image_transport::SimplePublisherPlugin<learning_image_transport::ResizedImage>

声明发布者插件类，继承于SimplePublisherPlugin。SimplePublisherPlugin is templated on the transport-specific message type.

  virtual std::string getTransportName() const
  {
    return "resized";
  }

SimplePublisherPlugin类需要我们重新定义两个成员函数，其中一个特别简单。getTransportName()返回transport的名字，我们这里选择了？getTransportName“resized”。

virtual void publish(const sensor_msgs::Image& message,
                       const PublishFn& publish_fn) const;

另外一个成员函数是publish()。这个函数编码sensor_msgs/Image消息，并重新变换为指定的尺寸然后发布。但是这里找不到发布的参数，这是因为这些已经在SimplePublisherPlugin定义了。PublishFn is a function object that takes our transport-specific message type as an argument and publishes it using some ROS primitive.

为什么存在这种间接性。据证明，ROS存在两种方法发布消息给订阅者。正常的方法是发布消息给所有的订阅者，另外一种方法是，发布消息给单独的一个订阅者 。

SimplePublisherPlugin performs some tricky plumbing to use our implementation of publish() in both cases.

For the details of publish(), the code is self-explanatory.

4 写一个订阅者

4.1 程序

创建一个头文件include/learning_image_transport/resized_subscriber.h，如下：image_transport_tutorial/ResizedImage.h这个头文件是生成消息文件的时候产生的。

#include <image_transport/simple_subscriber_plugin.h>
#include <image_transport_tutorial/ResizedImage.h>

class ResizedSubscriber : public image_transport::SimpleSubscriberPlugin<image_transport_tutorial::ResizedImage>
{
public:
  virtual ~ResizedSubscriber() {}

  virtual std::string getTransportName() const
  {
    return "resized";
  }

protected:
  virtual void internalCallback(const typename image_transport_tutorial::ResizedImage::ConstPtr& message,
                                const Callback& user_cb);
};

然后是这个src/resized_subscriber.cpp

#include <image_transport_tutorial/resized_subscriber.h>
#include <cv_bridge/cv_bridge.h>
#include <opencv2/imgproc/imgproc.hpp>

void ResizedSubscriber::internalCallback(const image_transport_tutorial::ResizedImage::ConstPtr& msg,
                                         const Callback& user_cb)
{
  // This is only for optimization, not to copy the image
  boost::shared_ptr<void const> tracked_object_tmp;
  cv::Mat img_rsz = cv_bridge::toCvShare(msg->image, tracked_object_tmp)->image;
  // Resize the image to its original size
  cv::Mat img_restored;
  cv::resize(img_rsz, img_restored, cv::Size(msg->original_width, msg->original_height));

  // Call the user callback with the restored image
  cv_bridge::CvImage cv_img(msg->image.header, msg->image.encoding, img_restored);
  // 回掉函数
  user_cb(cv_img.toImageMsg());
};

5 注册插件

已经写好了传输插件，但是还需要进行一些步骤，以便ROS可以找到这些插件。至于更多的细节问题，可以查看pluginlib，below we will note only features of special relevance to image_transport.

5.1 添加插件列表

创建一个src/manifest.cpp 然后如下：

#include <pluginlib/class_list_macros.h>
#include <image_transport_tutorial/resized_publisher.h>
#include <image_transport_tutorial/resized_subscriber.h>

PLUGINLIB_REGISTER_CLASS(resized_pub, ResizedPublisher, image_transport::PublisherPlugin)

PLUGINLIB_REGISTER_CLASS(resized_sub, ResizedSubscriber, image_transport::SubscriberPlugin)

We register our plugin classes as resized_pub and resized_sub. These names follow an important convention; image_transport assumes that for a transport named <foo>, the publisher and subscriber plugins have respective lookup names <foo>_pub and <foo>_sub.

We register our plugins as instances of the low-level base classes PublisherPlugin and SubscriberPlugin. The "simple" base classes we used in our code subclass PublisherPlugin and SubscriberPlugin to provide a more convenient high-level interface.

5.2 为插件写描述文件

创建一个resized_plugins.xml 然后如下：

<library path="lib/libresized_image_transport">
  <class name="resized_pub" type="ResizedPublisher" base_class_type="image_transport::PublisherPlugin">
    <description>
      This plugin publishes a decimated version of the image.
    </description>
  </class>

  <class name="resized_sub" type="ResizedSubscriber" base_class_type="image_transport::SubscriberPlugin">
    <description>
      This plugin rescales a decimated image to its original size.
    </description>
  </class>
</library>

这个文件将有助于ROS找到这些插件。

5.3 导出插件

最后我们必须指出我们的插件的描述文件，使用Manifest，如下：

<export>
  <image_transport plugin="${prefix}/resized_plugins.xml"/>
</export>

6 编译

cmake_minimum_required(VERSION 2.8)
project(image_transport_tutorial)

find_package(catkin REQUIRED cv_bridge genmsg image_transport sensor_msgs)

# add the resized image message
add_message_files(DIRECTORY msg
   FILES ResizedImage.msg
)
generate_messages(DEPENDENCIES sensor_msgs)

catkin_package()

find_package(OpenCV)

include_directories(include ${OpenCV_INCLUDE_DIRS})

# add the plugin examples
add_library(resized_publisher src/manifest.cpp src/resized_publisher.cpp src/resized_subscriber.cpp)
target_link_libraries(resized_publisher ${catkin_LIBRARIES} ${OpenCV_LIBRARIES})