This article makes the case for using DDS as the middleware for ROS, outlining the pros and cons of this approach, as well as considering the impact to the user experience and code API that using DDS would have. The results of the “ros_dds” prototype are also summarized and used in the exploration of the issue.
本文介绍了使用DDS作为ROS中间件的情况,概述了这种方法的优缺点,并考虑了使用DDS对用户体验和代码API的影响。本文还对“ROS-DDS”原型的研究成果进行了总结,并将其应用于该问题的探索。
Why Consider DDS
When exploring options for the next generation communication system of ROS, the initial options were to either improve the ROS 1.x transport or build a new middleware using component libraries such as ZeroMQ, Protocol Buffers, and zeroconf (Bonjour/Avahi). However, in addition to those options, both of which involved us building a middleware from parts or scratch, other end-to-end middlewares were considered. During our research, one middleware that stood out was DDS.
在探索下一代ROS通信系统的选项时,最初的选项是要么改进ROS 1.x传输,要么使用组件库(如zeromq、协议缓冲区和zeroconf(bonjour/avahi))构建新的中间件。但是,除了这些选项外,这两个选项都涉及我们从零开始或从头开始构建中间件,还考虑了其他端到端的中间件。在我们的研究中,最突出的中间件是DDS。
An End-to-End Middleware
The benefit of using an end-to-end middleware, like DDS, is that there is much less code to maintain and the behavior and exact specifications of the middleware have already been distilled into documentation. In addition to system-level documentation, DDS also has recommended use cases and a software API. With this concrete specification, third parties can review, audit, and implement the middleware with varying degrees of interoperability. This is something that ROS has never had, besides a few basic descriptions in a wiki and a reference implementation. Additionally, this type of specification would need to be created anyway if a new middleware were to be built from existing libraries.
使用端到端中间件(如DDS)的好处在于,要维护的代码要少得多,而且中间件的行为和准确的规范已经被提炼成文档。除了系统级文档之外,DDS还提供了推荐的用例和软件API。有了这个具体的规范,第三方可以审查、审计和实现具有不同程度互操作性的中间件。这是ROS从未有过的东西,除了在wiki和参考实现中的一些基本描述。此外,如果要从现有库中构建新的中间件,则无论如何都需要创建这种类型的规范。
The drawback of using an end-to-end middleware is that ROS must work within that existing design. If the design did not target a relevant use case or is not flexible, it might be necessary to work around the design. On some level, adopting an end-to-end middleware includes adopting the philosophy and culture of that middleware, which should not be taken lightly.
使用端到端中间件的缺点是,ROS必须在现有设计中工作。如果设计没有针对相关的用例或者不灵活,那么可能需要围绕设计工作。在某种程度上,采用端到端中间件包括采用该中间件的理念和文化,这一点不应轻视。
What is DDS
DDS provides a publish-subscribe transport which is very similar to ROS’s publish-subscribe transport. DDS uses the “Interface Description Language (IDL)” as defined by the Object Management Group (OMG) for message definition and serialization. DDS has a request-response style transport, which would be like ROS’s service system, in beta 2 as of June 2016 (called DDS-RPC).
DDS提供了一个发布订阅传输,与ROS的发布订阅传输非常相似。DDS使用对象管理组(OMG)定义的“接口描述语言(IDL)”进行消息定义和序列化。截至2016年6月,DDS在beta 2中有一个请求-响应式传输,类似于ROS的服务系统(称为DDS-RPC)。
The default discovery system provided by DDS, which is required to use DDS’s publish-subscribe transport, is a distributed discovery system. This allows any two DDS programs to communicate without the need for a tool like the ROS master. This makes the system more fault tolerant and flexible. It is not required to use the dynamic discovery mechanism, however, as multiple DDS vendors provide
DDS提供的默认发现系统是分布式发现系统,使用DDS的发布-订阅传输是必需的。这允许任何两个DDS程序通信,而不需要像ROS主机这样的工具。这使得系统更具容错性和灵活性。但是,由于有多个DDS供应商提供,所以不需要使用动态发现机制
Where did DDS come from
DDS got its start as a group of companies which had similar middleware frameworks and became a standard when common customers wanted to get better interoperability between the vendors. The DDS standard was created by the Object Management Group, which are the same people that brought us UML, CORBA, SysML, and other generic software related standards. Now, depending on your perspective, this may be a positive endorsement or a negative endorsement. On the one hand you have a standards committee which is perennial and clearly has a huge influence on the software engineering community, but on the other hand you have a slow moving body which is slow to adapt to changes and therefore arguably doesn’t always keep up with the latest trends in software engineering.
DDS作为一个拥有类似中间件框架的公司集团开始起步,当普通客户希望在供应商之间获得更好的互操作性时,DDS成为了一个标准。DDS标准是由对象管理组创建的,对象管理组是为我们带来UML、CORBA、SysML和其他通用软件相关标准的同一个人。现在,根据你的观点,这可能是一个积极的认可或消极的认可。一方面,你有一个标准委员会,这个委员会是长期存在的,显然对软件工程界有着巨大的影响,另一方面,你有一个缓慢移动的机构,它缓慢地适应变化,因此可以说并不总是跟上软件工程的最新趋势。
DDS was originally several similar middlewares which eventually became so close to one another that writing a standard to unify them made sense. So in this way, even though the DDS specification has been written by a committee, bit has evolved to its current formy reacting to the needs of its users. This type of organic evolution of the specification before it was ratified helps to alleviate the concern that the system was designed in a vacuum and that it does not perform well in real environments. There are some examples of committees coming up with well intentioned and well described specifications that nobody wants to use or that don’t meet the needs of the community they serve, but this does not appear to be the case for DDS.
DDS最初是几个类似的中间商,它们最终变得如此接近,以至于编写一个统一它们的标准是有意义的。因此,通过这种方式,即使DDS规范是由一个委员会编写的,它也通过响应用户的需求而发展成了当前的形式。在规范被批准之前,这类规范的有机演变有助于减轻人们对系统是在真空中设计的以及在实际环境中性能不佳的关注。有一些委员会提出了意图明确、描述良好的规范,这些规范没有人愿意使用,或者不满足他们所服务社区的需要,但这似乎不是DDS的情况。
There is also a concern that DDS is a static specification which was defined and is used in “legacy” systems, but has not kept current. This kind of stereotype comes from horror stories about things like UML and CORBA, which are also products of OMG. On the contrary, DDS seems to have an active and organic specification, which in the recent past has added, or is adding, more specifications for things like websockets, security over SSL, extensible types, request and response transport, and a new, more modern C++11 style API specification for the core API to replace the existing C++ interface. This type of evolution in the standard body for DDS is an encouraging thing to observe, and even though the body is relatively slow, as compared to software engineering technology trends, it is evolving to meet demands of its users.
还有一个问题是,DDS是一个静态规范,它被定义并用于“遗留”系统,但没有保持最新。这种刻板印象来自于关于UML和CORBA之类的恐怖故事,它们也是OMG的产品。相反,DDS似乎有一个积极的和有机的规范,在最近的过去,它已经添加或添加了更多的规范,如WebSoCube、SSL安全性、可扩展类型、请求和响应传输,以及一种新的、更现代的C++ 11风格API规范,用于核心API以取代现有的C++接口。DDS标准体中的这类演变是值得观察的一件令人鼓舞的事情,尽管与软件工程技术趋势相比,DDS的标准体发展相对缓慢,但它正在发展以满足用户的需求。
Technical Credibility
DDS has an extensive list of varied installations which are typically mission critical. DDS has been used in:
- battleships
- large utility installations like dams
- financial systems
- space systems
- flight systems
- train switchboard systems
DDS有一个广泛的各种装置清单,这些装置通常是关键任务。DDS已用于:
- 战列舰
- 大坝等大型公用设施
- 金融系统
- 空间系统
- 飞行系统
- 列车总机系统
and many other equally important and varied scenarios. These successful use cases lend credibility to DDS’s design being both reliable and flexible.
以及许多其他同样重要和不同的场景。这些成功的用例使DDS的设计既可靠又灵活。
Not only has DDS met the needs of these use cases, but after talking with users of DDS (in this case government and NASA employees who are also users of ROS), they have all praised its reliability and flexibility. Those same users will note that the flexibility of DDS comes at the cost of complexity. The complexity of the API and configuration of DDS is something that ROS would need to address.
DDS不仅满足了这些用例的需求,而且在与DDS的用户(在这种情况下,政府和NASA的雇员也是ROS的用户)交谈之后,他们都称赞了其可靠性和灵活性。这些用户也会注意到,DDS的灵活性是以复杂性为代价的。API的复杂性和DDS的配置是ROS需要解决的问题。
The DDS wire specification (DDSI-RTPS) is extremely flexible, allowing it to be used for reliable, high level systems integration as well as real-time applications on embedded devices. Several of the DDS vendors have special implementations of DDS for embedded systems which boast specs related to library size and memory footprint on the scale of tens or hundreds of kilobytes. Since DDS is implemented, by default, on UDP, it does not depend on a reliable transport or hardware for communication. This means that DDS has to reinvent the reliability wheel (basically TCP plus or minus some features), but in exchange DDS gains portability and control over the behavior. Control over several parameters of reliability, what DDS calls Quality of Service (QoS), gives maximum flexibility in controlling the behavior of communication. For example, if you are concerned about latency, like for soft real-time, you can basically tune DDS to be just a UDP blaster. In another scenario you might need something that behaves like TCP, but needs to be more tolerant to long dropouts, and with DDS all of these things can be controlled by changing the QoS parameters.
DDS线规范(DDSI-RTPS)非常灵活,可用于可靠的高级别系统集成以及嵌入式设备上的实时应用。一些DDS供应商为嵌入式系统提供了DDS的特殊实现,这些嵌入式系统拥有与库大小和内存占用相关的规范,规模为数十或数百千字节。由于DDS默认在UDP上实现,因此它不依赖可靠的传输或硬件进行通信。这意味着DDS必须重新设计可靠性车轮(基本上是TCP加上或减去一些特性),但作为交换,DDS获得了可移植性和对行为的控制。对可靠性的几个参数进行控制,DDS称之为服务质量(QoS),在控制通信行为方面提供了最大的灵活性。例如,如果您关心延迟,比如软实时,那么您基本上可以将DDS调到仅仅是一个UDP爆炸机。在另一个场景中,您可能需要一些类似于TCP的东西,但是需要对长时间的退出更为宽容,并且使用DDS,可以通过更改QoS参数来控制所有这些东西。
Though the default implementation of DDS is over UDP, and only requires that level of functionality from the transport, OMG also added support for DDS over TCP in version 1.2 of their specification. Only looking briefly, two of the vendors (RTI and PrismTech) both support DDS over TCP.
虽然DDS的默认实现是通过UDP实现的,并且只需要传输的功能级别,但是OMG在其规范的1.2版本中还添加了对通过TCP的DDS的支持。简单地说,两个供应商(RTI和PrismTech)都支持TCP上的DDS。
From RTI’s website (http://community.rti.com/kb/xml-qos-example-using-rti-connext-dds-tcp-transport):
By default, RTI Connext DDS uses the UDPv4 and Shared Memory transport to communicate with other DDS applications. In some circumstances, the TCP protocol might be needed for discovery and data exchange. For more information on the RTI TCP Transport, please refer to the section in the RTI Core Libraries and Utilities User Manual titled “RTI TCP Transport”.
From PrismTech’s website, they support TCP as of OpenSplice v6.4:
http://www.prismtech.com/vortex/vortex-opensplice/communication/ddsi2
从RTI的网站(http://community.rti.com/kb/xml-qos示例,使用rti-connext-dds-tcp-transport):
默认情况下,rti connext dds使用udpv4和共享内存传输与其他dds应用程序通信。在某些情况下,发现和数据交换可能需要TCP协议。有关RTI TCP传输的更多信息,请参阅RTI核心库和实用程序用户手册中标题为“RTI TCP传输”的部分。
从Prismtech的网站上,他们支持从OpenSplice V6.4开始的TCP:
http://www.prismtech.com/vortex/vortex-opensplice/communication/ddsi2
Vendors and Licensing
The OMG defined the DDS specification with several companies which are now the main DDS vendors. Popular DDS vendors include:
- RTI
- PrismTech
- Twin Oaks Software
供应商和许可
OMG与现在主要DDS供应商的几家公司一起定义了DDS规范。最受欢迎的DDS供应商包括:
- RTI
- 普里斯米特
- Twin Oaks软件
这些供应商中有一系列具有不同策略和许可证的参考实现。OMG维护DDS供应商的活动列表。
Amongst these vendors is an array of reference implementations with different strategies and licenses. The OMG maintains an active list of DDS vendors.
In addition to vendors providing implementations of the DDS specification’s API, there are software vendors which provide an implementation with more direct access to the DDS wire protocol, RTPS. For example:
除了提供DDS规范的API实现的供应商之外,还有软件供应商提供的实现可以更直接地访问DDS有线协议RTPS。例如:
- eProsima
These RTPS-centric implementations are also of interest because they can be smaller in scope and still provide the needed functionality for implementing the necessary ROS capabilities on top.
这些以RTP为中心的实现也很有趣,因为它们的范围可以更小,并且仍然提供了在顶部实现必要的ROS功能所需的功能。
RTI’s Connext DDS is available under a custom “Community Infrastructure” License, which is compatible with the ROS community’s needs but requires further discussion with the community in order to determine its viability as the default DDS vendor for ROS. By “compatible with the ROS community’s needs,” we mean that, though it is not an OSI-approved license, research has shown it to be adequately permissive to allow ROS to keep a BSD style license and for anyone in the ROS community to redistribute it in source or binary form. RTI also appears to be willing to negotiate on the license to meet the ROS community’s needs, but it will take some iteration between the ROS community and RTI to make sure this would work. Like the other vendors this license is available for the core set of functionality, basically the basic DDS API, whereas other parts of their product like development and introspection tools are proprietary. RTI seems to have the largest on-line presence and installation base.
RTI的Connext DDS在定制的“社区基础设施”许可证下可用,该许可证符合ROS社区的需求,但需要与社区进一步讨论,以确定其作为ROS默认DDS供应商的可行性。通过“兼容ROS社区的需求”,我们的意思是,尽管它不是一个OSI批准的许可证,研究表明它是充分的许可证,允许ROS保留一个BSD风格的许可证,并且允许ROS社区中的任何人以源代码或二进制形式重新分配它。RTI似乎也愿意就许可证进行谈判,以满足ROS社区的需求,但需要在ROS社区和RTI之间进行一些迭代,以确保这一点可行。与其他供应商一样,此许可证可用于核心功能集,基本上是基本的DDS API,而其产品的其他部分(如开发和内省工具)是专有的。RTI似乎拥有最大的在线存在和安装基础。
PrismTech’s DDS implementation, OpenSplice, is licensed under the LGPL, which is the same license used by many popular open source libraries, like glibc, ZeroMQ, and Qt. It is available on Github:
prismtech的DDS实现opensplice是在lgpl下获得许可的,这与许多流行的开源库(如glibc、zeromq和qt)使用的许可相同。在Github上提供:
https://github.com/PrismTech/opensplice
PrismTech’s implementation comes with a basic, functioning build system and was fairly easy to package. OpenSplice appears to be the number two DDS implementation in use, but that is hard to tell for sure.
TwinOaks’s CoreDX DDS implementation is proprietary only, but apparently they specialize in minimal implementations which are able to run on embedded devices and even bare metal.
eProsima’s FastRTPS implementation is available on GitHub and is LGPL licensed:
Prismtech的实现带有一个基本的、运行正常的构建系统,并且非常容易打包。OpenSplice似乎是使用中的第二个DDS实现,但这很难确定。
TwinOaks的coredx-dds实现仅为专有,但显然它们专门用于能够在嵌入式设备甚至裸机上运行的最小实现。
eprosima的fastrtps实现可在GitHub上获得,并且是LGPL许可的:
https://github.com/eProsima/Fast-RTPS
eProsima Fast RTPS is a relatively new, lightweight, and open source implementation of RTPS. It allows direct access to the RTPS protocol settings and features, which is not always possible with other DDS implementations. eProsima’s implementation also includes a minimum DDS API, IDL support, and automatic code generation and they are open to working with the ROS community to meet their needs.
EPROSIMA Fast RTPS是一种相对较新的、轻量级的、开源的RTPS实现。它允许直接访问RTPS协议设置和功能,这在其他DDS实现中并不总是可能的。EPROSIMA的实现还包括最低限度的DDS API、IDL支持和自动代码生成,并且他们愿意与ROS社区合作以满足他们的需求。
Given the relatively strong LGPL option and the encouraging but custom license from RTI, it seems that depending on and even distributing DDS as a dependency should be straightforward. One of the goals of this proposal would be to make ROS 2.0 DDS vendor agnostic. So, just as an example, if the default implementation is Connext, but someone wants to use one of the LGPL options like OpenSplice or FastRTPS, they simply need to recompile the ROS source code with some options flipped and they can use the implementation of their choice.
考虑到相对强大的LGPL选项和来自RTI的令人鼓舞但定制的许可证,依赖甚至分发DDS作为依赖项应该是很简单的。该提案的目标之一是使ROS 2.0 DDS供应商不可知论。因此,就像一个例子一样,如果默认实现是connext,但是有人想要使用一个lgpl选项,比如opensplice或fastrtps,那么他们只需要使用翻转的一些选项重新编译ROS源代码,并且他们可以使用自己选择的实现。
This is made possible because of the fact that DDS defines an API in its specification. Research has shown that making code which is vendor agnostic is possible if not a little painful since the APIs of the different vendors is almost identical, but there are minor differences like return types (pointer versus shared_ptr like thing) and header file organization.
这是可能的,因为DDS在其规范中定义了一个API。研究表明,如果不是有点痛苦的话,将代码与供应商无关是可能的,因为不同供应商的API几乎是相同的,但是有一些细微的差异,如返回类型(指针与共享的指针类似的东西)和头文件组织。
Ethos and Community
DDS comes out of a set of companies which are decades old, was laid out by the OMG which is an old-school software engineering organization, and is used largely by government and military users. So it comes as no surprise that the community for DDS looks very different from the ROS community and that of similar modern software projects like ZeroMQ. Though RTI has a respectable on-line presence, the questions asked by community members are almost always answered by an employee of RTI and though technically open source, neither RTI nor OpenSplice has spent time to provide packages for Ubuntu or Homebrew or any other modern package manager. They do not have extensive user-contributed wikis or an active Github repository.
DDS源于一系列有几十年历史的公司,由OMG(一个老式的软件工程组织)开发,主要由政府和军事用户使用。因此,DDS社区与ROS社区以及类似的现代软件项目(如zeromq)看起来非常不同也就不足为奇了。尽管RTI在网上有着令人尊敬的地位,但社区成员提出的问题几乎总是由RTI的一名员工回答的,尽管技术上是开源的,但RTI和OpenSplice都没有花时间为Ubuntu、Homebrew或任何其他现代包管理器提供包。他们没有广泛的用户贡献的维基或活动的Github存储库。
This staunch difference in ethos between the communities is one of the most concerning issues with depending on DDS. Unlike options like keeping TCPROS or using ZeroMQ, there isn’t the feeling that there is a large community to fall back on with DDS. However, the DDS vendors have been very responsive to our inquiries during our research and it is hard to say if that will continue when it is the ROS community which brings the questions.
社区之间的这种坚定的精神差异是依赖DDS最令人关注的问题之一。与保留TCPros或使用ZeroMQ等选项不同,没有感觉到有一个大型社区可以依靠DDS。然而,DDS供应商在我们的研究过程中对我们的询问作出了非常积极的回应,很难说当ROS社区提出问题时,这是否会继续。
Even though this is something which should be taken into consideration when making a decision about using DDS, it should not disproportionately outweigh the technical pros and cons of the DDS proposal.
尽管这是在决定使用DDS时应考虑的事项,但它不应过分地超过DDS建议的技术利弊。
ROS Built on DDS
The goal is to make DDS an implementation detail of ROS 2.0. This means that all DDS specific APIs and message definitions would need to be hidden. DDS provides discovery, message definition, message serialization, and publish-subscribe transport. Therefore, DDS would provide discovery, publish-subscribe transport, and at least the underlying message serialization for ROS. ROS 2.0 would provide a ROS 1.x like interface on top of DDS which hides much of the complexity of DDS for the majority of ROS users, but then separately provides access to the underlying DDS implementation for users that have extreme use cases or need to integrate with other, existing DDS systems.
目标是使DDS成为ROS 2.0的实现细节。这意味着需要隐藏所有特定于DDS的API和消息定义。DDS提供发现、消息定义、消息序列化和发布订阅传输。因此,DDS将为ROS提供发现、发布订阅传输和至少底层消息序列化。ROS 2.0将在DDS之上提供一个类似于ROS 1.x的接口,它为大多数ROS用户隐藏了DDS的复杂性,但随后分别为具有极端用例或需要与其他现有DDS系统集成的用户提供对底层DDS实现的访问。
DDS and ROS API Layout
DDS和ROS API布局DDS和ROS API布局
Accessing the DDS implementation would require depending on an additional package which is not normally used. In this way you can tell if a package has tied itself to a particular DDS vendor by just looking at the package dependencies. The goal of the ROS API, which is on top of DDS, should be to meet all the common needs for the ROS community, because once a user taps into the underlying DDS system, they will lose portability between DDS vendors. Portability among DDS vendors is not intended to encourage people to frequently choose different vendors, but rather to enable power users to select the DDS implementation that meets their specific requirements, as well as to future-proof ROS against changes in the DDS vendor options. There will be one recommended and best-supported default DDS implementation for ROS.
访问DDS实现需要依赖于通常不使用的附加包。通过这种方式,您可以通过查看包的依赖关系来判断包是否与特定的DDS供应商相关联。位于DDS之上的ROS API的目标应该是满足ROS社区的所有共同需求,因为一旦用户进入底层DDS系统,他们将失去DDS供应商之间的可移植性。DDS供应商之间的可移植性并不是为了鼓励人们经常选择不同的供应商,而是为了使电力用户能够选择满足其特定要求的DDS实施,以及为将来证明ROS不受DDS供应商选项变化的影响。对于ROS,将有一个推荐的和受最佳支持的默认DDS实现。
Discovery
DDS would completely replace the ROS master based discovery system. ROS would need to tap into the DDS API to get information like a list of all nodes, a list of all topics, and how they are connected. Accessing this information would be hidden behind a ROS defined API, preventing the users from having to call into DDS directly.
DDS将完全取代基于ROS主发现系统。ROS需要利用DDS API获取信息,比如所有节点的列表、所有主题的列表以及它们的连接方式。访问这些信息将隐藏在ROS定义的API之后,从而防止用户直接调用DDS。
The advantage of the DDS discovery system is that, by default, it is completely distributed, so there is no central point of failure which is required for parts of the system to communicate with each other. DDS also allows for user defined meta data in their discovery system, which will enable ROS to piggyback higher level concepts onto publish-subscribe.
DDS发现系统的优点是,默认情况下,它是完全分布式的,因此系统的各个部分之间不需要任何中心故障点来进行通信。DDS还允许在他们的发现系统中使用用户定义的元数据,这将使ROS能够将更高级别的概念移植到发布订阅上。
Efficient Transport Alternatives
In ROS 1.x there was never a standard shared-memory transport because it is negligibly faster than localhost TCP loop-back connections. It is possible to get non-trivial performance improvements from carefully doing zero-copy style shared-memory between processes, but anytime a task required faster than localhost TCP in ROS 1.x, nodelets were used. Nodelets allow publishers and subscribers to share data by passing around boost::shared_ptr
s to messages. This intraprocess communication is almost certainly faster than any interprocess communication options and is orthogonal to the discussion of the network publish-subscribe implementation.
在ROS1.X中,从来没有标准的共享内存传输,因为它比本地主机TCP环回连接快得多。通过在进程之间小心地执行零拷贝式共享内存,可以获得非常重要的性能改进,但只要任务需要比ROS1.x中的localhost tcp更快的速度,就可以使用nodelets。nodelets允许发布者和订阅者通过向消息传递boost::shared_ptrs来共享数据。这种进程内通信几乎肯定比任何进程间通信选项都快,并且与网络发布订阅实现的讨论是正交的。
In the context of DDS, most vendors will optimize message traffic (even between processes) using shared-memory in a transparent way, only using the wire protocol and UDP sockets when leaving the localhost. This provides a considerable performance increase for DDS, whereas it did not for ROS 1.x, because the localhost networking optimization happens at the call to send
. For ROS 1.x the process was: serialize the message into one large buffer, call TCP’s send
on the buffer once. For DDS the process would be more like: serialize the message, break the message into potentially many UDP packets, call UDP’s send
many times. In this way sending many UDP datagrams does not benefit from the same speed up as one large TCP send
. Therefore, many DDS vendors will short circuit this process for localhost messages and use a blackboard style shared-memory mechanism to communicate efficiently between processes.
在DDS的上下文中,大多数供应商将以透明的方式使用共享内存优化消息流量(甚至在进程之间),只有在离开本地主机时才使用有线协议和UDP套接字。这为DDS提供了相当大的性能提升,而不是ROS1.x,因为本地主机网络优化发生在发送调用上。对于ROS1.x,过程是:将消息序列化到一个大缓冲区中,在缓冲区上调用tcp的send一次。对于DDS,过程更像是:序列化消息,将消息分解为可能多个UDP包,多次调用UDP的发送。通过这种方式,发送多个UDP数据报不会受益于与一个大型TCP发送相同的速度。因此,许多DDS供应商将对本地主机消息短路此进程,并使用黑板式共享内存机制在进程之间进行有效的通信。
However, not all DDS vendors are the same in this respect, so ROS would not rely on this “intelligent” behavior for efficient intraprocess communication. Additionally, if the ROS message format is kept, which is discussed in the next section, it would not be possible to prevent a conversion to the DDS message type for intraprocess topics. Therefore a custom intraprocess communication system would need to be developed for ROS which would never serialize nor convert messages, but instead would pass pointers (to shared in-process memory) between publishers and subscribers using DDS topics. This same intraprocess communication mechanism would be needed for a custom middleware built on ZeroMQ, for example.
然而,并非所有DDS供应商在这方面都是相同的,因此ROS不会依赖于这种“智能”行为来实现高效的进程内通信。此外,如果保留了ROS消息格式(将在下一节中讨论),则不可能阻止将进程内主题转换为DDS消息类型。因此,需要为从不序列化或转换消息的ROS开发定制进程内通信系统,而是使用DDS主题在发布服务器和订阅服务器之间传递指针(到共享进程内内存)。例如,对于构建在zeromq上的定制中间件,也需要相同的进程内通信机制。
The point to take away here is that efficient intraprocess communication will be addressed regardless of the network/interprocess implementation of the middleware.
这里要做的一点是,不管中间件的网络/进程间实现如何,都将处理高效的进程内通信。
Messages
There is a great deal of value in the current ROS message definitions. The format is simple, and the messages themselves have evolved over years of use by the robotics community. Much of the semantic contents of current ROS code is driven by the structure and contents of these messages, so preserving the format and in-memory representation of the messages has a great deal of value. In order to meet this goal, and in order to make DDS an implementation detail, ROS 2.0 should preserve the ROS 1.x like message definitions and in-memory representation.
当前的ROS消息定义有很大的价值。这种格式很简单,信息本身经过机器人界多年的使用而不断发展。当前ROS代码的语义内容大多是由这些消息的结构和内容驱动的,因此保留消息的格式和内存表示具有很大的价值。为了实现这一目标,并且为了使DDS成为一个实现细节,ROS2.0应该保留ROS1.x类消息定义和内存表示。
Therefore, the ROS 1.x .msg
files would continue to be used and the .msg
files would be converted into .idl
files so that they could be used with the DDS transport. Language specific files would be generated for both the .msg
files and the .idl
files as well as conversion functions for converting between ROS and DDS in-memory instances. The ROS 2.0 API would work exclusively with the .msg
style message objects in memory and would convert them to .idl
objects before publishing.
因此,ROS1.x.msg文件将继续使用,.msg文件将转换为.idl文件,以便与DDS传输一起使用。将为.msg文件和.idl文件生成特定于语言的文件,以及用于在内存实例中的ROS和DDS之间转换的转换函数。ROS2.0API将专门处理内存中的.msg样式的消息对象,并在发布之前将其转换为.idl对象。
At first, the idea of converting a message field-by-field into another object type for each call to publish seems like a huge performance problem, but experimentation has shown that the cost of this copy is insignificant when compared to the cost of serialization. This ratio between the cost of converting types and the cost of serialization, which was found to be at least one order of magnitude, holds true with every serialization library that we tried, except Cap’n Proto which doesn’t have a serialization step. Therefore, if a field-by-field copy will not work for your use case, neither will serializing and transporting over the network, at which point you will have to utilize an intraprocess or zero-copy interprocess communication. The intraprocess communication in ROS would not use the DDS in-memory representation so this field-by-field copy would not be used unless the data is going to the wire. Because this conversion is only invoked in conjunction with a more expensive serialization step, the field-by-field copy seems to be a reasonable trade-off for the portability and abstraction provided by preserving the ROS .msg
files and in-memory representation.
一开始,将消息字段逐字段转换为另一个对象类型以便每次调用发布的想法似乎是一个巨大的性能问题,但实验表明,与序列化成本相比,此副本的成本是微不足道的。除了没有序列化步骤的cap'n proto之外,在我们尝试的每个序列化库中,转换类型的成本与序列化成本之间的比率至少为一个数量级。因此,如果一个逐字段复制不适用于您的用例,也不会通过网络进行序列化和传输,此时您必须使用进程内或零拷贝进程间通信。ROS中的进程内通信不会使用内存中的DDS表示,因此除非数据要连接,否则不会使用逐字段复制。由于此转换仅与更昂贵的序列化步骤一起调用,因此逐字段复制对于通过保留ros.msg文件和内存表示提供的可移植性和抽象性似乎是一个合理的折衷。
This does not preclude the option to improve the .msg
file format with things like default values and optional fields. But this is a different trade-off which can be decided later.
这并不排除使用诸如默认值和可选字段之类的内容来改进.msg文件格式的选项。但这是一个不同的权衡,可以稍后决定。
Services and Actions
DDS currently does not have a ratified or implemented standard for request-response style RPC which could be used to implement the concept of services in ROS. There is currently an RPC specification being considered for ratification in the OMG DDS working group, and several of the DDS vendors have a draft implementation of the RPC API. It is not clear, however, whether this standard will work for actions, but it could at least support non-preemptable version of ROS services. ROS 2.0 could either implement services and actions on top of publish-subscribe (this is more feasible in DDS because of their reliable publish-subscribe QoS setting) or it could use the DDS RPC specification once it is finished for services and then build actions on top, again like it is in ROS 1.x. Either way actions will be a first class citizen in the ROS 2.0 API and it may be the case that services just become a degenerate case of actions.
DDS目前没有批准或实现的请求-响应式RPC标准,该标准可用于在ROS中实现服务概念。目前,OMG DDS工作组正在考虑批准一个RPC规范,并且一些DDS供应商有一个RPC API的草案实现。然而,尚不清楚该标准是否适用于行动,但它至少可以支持ROS服务的不可抢占版本。ROS 2.0可以在发布订阅之上实现服务和操作(这在DDS中更可行,因为其可靠的发布订阅QoS设置),也可以在完成服务之后使用DDS RPC规范,然后在上面构建操作,就像在ROS 1.x中一样。无论哪种方式,操作都将是一流的Citizen在ROS 2.0API中,服务可能只是一个退化的行为案例。
Language Support
DDS vendors typically provide at least C, C++, and Java implementations since APIs for those languages are explicitly defined by the DDS specification. There are not any well established versions of DDS for Python that research has uncovered. Therefore, one goal of the ROS 2.0 system will be to provide a first-class, feature complete C API. This will allow bindings for other languages to be made more easily and to enable more consistent behavior between client libraries, since they will use the same implementation. Languages like Python, Ruby, and Lisp can wrap the C API in a thin, language idiomatic implementation.
DDS供应商通常提供至少C、C++和Java实现,因为这些语言的API是由DDS规范明确定义的。研究人员还没有发现任何成熟的针对Python的DDS版本。因此,ROS 2.0系统的一个目标是提供一流的、功能完整的C API。这将使其他语言的绑定变得更加容易,并使客户端库之间的行为更加一致,因为它们将使用相同的实现。像Python、Ruby和Lisp这样的语言可以将C API包装在一个瘦的、语言惯用的实现中
The actual implementation of ROS can either be in C, using the C DDS API, or in C++ using the DDS C++ API and then wrapping the C++ implementation in a C API for other languages. Implementing in C++ and wrapping in C is a common pattern, for example ZeroMQ does exactly this. The author of ZeroMQ, however, did not do this in his new library, nanomsg, citing increased complexity and the bloat of the C++ stdlib as a dependency. Since the C implementation of DDS is typically pure C, it would be possible to have a pure C implementation for the ROS C API all the way down through the DDS implementation. However, writing the entire system in C might not be the first goal, and in the interest of getting a minimal viable product working, the implementation might be in C++ and wrapped in C to begin with and later the C++ can be replaced with C if it seems necessary.
ROS的实际实现既可以在C中,也可以使用C DDS API,也可以在C++中使用DDS C++ API,然后将C++实现在C语言API中封装成其他语言。在C++中实现和打包C是一种常见的模式,例如ZeNoMQ就是这样做的。然而,ZrOMQ的作者并没有在他的新库NANOMSG中这样做,他引用了增加的复杂性和C++STDLIB作为依赖的膨胀。由于DDS的C实现通常是纯C的,所以有可能通过DDS实现为ROS C API提供纯C实现。然而,在C中编写整个系统可能不是第一个目标,为了获得最小可行产品的工作,实现可能在C++中,并以C包开始,并且如果需要的话,C++可以用C替换。
DDS as a Dependency
One of the goals of ROS 2.0 is to reuse as much code as possible (“do not reinvent the wheel”) but also minimize the number of dependencies to improve portability and to keep the build dependency list lean. These two goals are sometimes at odds, since it is often the choice between implementing something internally or relying on an outside source (dependency) for the implementation.
ROS 2.0的目标之一是尽可能多地重用代码(“不要重新设计轮子”),同时最小化依赖项的数量,以提高可移植性并保持构建依赖项列表的精简。这两个目标有时是不一致的,因为这通常是在内部实现某个东西或依赖外部源(依赖)实现之间的选择。
This is a point where the DDS implementations shine, because two of the three DDS vendors under evaluation build on Linux, OS X, Windows, and other more exotic systems with no external dependencies. The C implementation relies only on the system libraries, the C++ implementations only rely on a C++03 compiler, and the Java implementation only needs a JVM and the Java standard library. Bundled as a binary (during prototyping) on both Ubuntu and OS X, the C, C++, Java, and C# implementations of OpenSplice (LGPL) is less than three megabytes in size and has no other dependencies. As far as dependencies go, this makes DDS very attractive because it significantly simplifies the build and run dependencies for ROS. Additionally, since the goal is to make DDS an implementation detail, it can probably be removed as a transitive run dependency, meaning that it will not even need to be installed on a deployed system.
这是DDS实现的亮点,因为正在评估中的三个DDS供应商中有两个是基于Linux、OS X、Windows和其他更具异国情调的无外部依赖性系统构建的。C实现仅依赖于系统库,C++实现仅依赖于C++ 03编译器,而Java实现只需要JVM和Java标准库。在Ubuntu和OS X上,作为二进制(在原型过程中)捆绑,OpenCPLICE(LGPL)的C、C++、Java和C实现都小于三兆字节,并且没有其他依赖关系。就依赖性而言,这使得DDS非常有吸引力,因为它显著简化了ROS的构建和运行依赖性。此外,由于目标是使DDS成为一个实现细节,因此它可能作为一个可传递的运行依赖项被删除,这意味着它甚至不需要安装在部署的系统上。
The ROS on DDS Prototype
Following the research into the feasibility of ROS on DDS, several questions were left, including but not limited to:
- Can the ROS 1.x API and behavior be implemented on top of DDS?
- Is it practical to generate IDL messages from ROS MSG messages and use them with DDS?
- How hard is it to package (as a dependency) DDS implementations?
- Does the DDS API specification actually make DDS vendor portability a reality?
- How difficult is it to configure DDS?
在对DDS上的ROS可行性进行研究之后,还剩下几个问题,包括但不限于:
- ROS 1.x API和行为是否可以在DDS之上实现?
- 从ROS消息生成IDL消息并将其与DDS一起使用是否可行?
- 打包(作为依赖项)DDS实现有多困难?
- DDS API规范是否使DDS供应商的可移植性成为现实?
- 配置DDS有多困难?
In order to answer some of these questions a prototype and several experiments were created in this repository:
为了回答其中的一些问题,在此存储库中创建了一个原型和几个实验:
https://github.com/osrf/ros_dds
More questions and some of the results were captured as issues:
https://github.com/osrf/ros_dds/issues?labels=task&page=1&state=closed
The major piece of work in this repository is in the prototype
folder and is a ROS 1.x like implementation of the Node, Publisher, and Subscriber API using DDS:
此存储库中的主要工作在Prototype文件夹中,是使用DDS实现的类似于ROS 1.x的节点、发布者和订阅者API:
https://github.com/osrf/ros_dds/tree/master/prototype
Specifically this prototype includes these packages:
-
Generation of DDS IDLs from
.msg
files: https://github.com/osrf/ros_dds/tree/master/prototype/src/genidl -
Generation of DDS specific C++ code for each generated IDL file: https://github.com/osrf/ros_dds/tree/master/prototype/src/genidlcpp
-
Minimal ROS Client Library for C++ (rclcpp): https://github.com/osrf/ros_dds/tree/master/prototype/src/rclcpp
-
Talker and listener for pub-sub and service calls: https://github.com/osrf/ros_dds/tree/master/prototype/src/rclcpp_examples
-
A branch of
ros_tutorials
in whichturtlesim
has been modified to build against therclcpp
library: https://github.com/ros/ros_tutorials/tree/ros_dds/turtlesim. This branch ofturtlesim
is not feature-complete (e.g., services and parameters are not supported), but the basics work, and it demonstrates that the changes required to transition from ROS 1.xroscpp
to the prototype of ROS 2.0rclcpp
are not dramatic.
具体来说,此原型包括以下包:
- 从.msg文件生成dds idls:https://github.com/osrf/ros_dds/tree/master/prototype/src/genidl
- 为每个生成的IDL文件生成DDS专用C++代码:HTTPS:/GIHUUBCOM/OSPR/ROSY-DDS/Tee/Mista/Tyrime/Src/GeNIDLCPP
- 用于C++(RCCPP)的最小ROS客户端库:HTTPSE//GITHUB/COM/OSPR/ROSYDDS/TRAE/MSTASE/PROSTYTY/SRC/RCCPP
This is a rapid prototype which was used to answer questions, so it is not representative of the final product or polished at all. Work on certain features was stopped cold once key questions had been answered.
这是一个用于回答问题的快速原型,因此它根本不能代表最终产品或抛光。一旦回答了关键问题,某些功能的工作就停止了。
The examples in the rclcpp_example
package showed that it was possible to implement the basic ROS like API on top of DDS and get familiar behavior. This is by no means a complete implementation and doesn’t cover all of the features, but instead it was for educational purposes and addressed most of the doubts which were held with respect to using DDS.
rclcp ou示例包中的示例表明,可以在DDS之上实现基本的ROS类API,并获得熟悉的行为。这绝不是一个完整的实现,也不包括所有的功能,而是为了教育目的,解决了关于使用DDS的大多数疑问。
事实证明,IDL文件的生成有一些症结,但最终可以得到解决,而且实现诸如服务之类的基本内容是可以解决的问题。
Generation of IDL files proved to have some sticking points, but could ultimately be addressed, and implementing basic things like services proved to be tractable problems.
事实证明,IDL文件的生成有一些症结,但最终可以得到解决,而且实现诸如服务之类的基本内容是可以解决的问题。
In addition to the above basic pieces, a pull request was drafted which managed to completely hide the DDS symbols from any publicly installed headers for rclcpp
and std_msgs
:
除了上述基本部分外,还起草了一个拉请求,该请求成功地将DDS符号完全隐藏在RCCPP和Std_-MSG的任何公开安装头中:
https://github.com/osrf/ros_dds/pull/17
This pull request was ultimately not merged because it was a major refactoring of the structure of the code and other progress had been made in the meantime. However, it served its purpose in that it showed that the DDS implementation could be hidden, though there is room for discussion on how to actually achieve that goal.
这个请求最终没有被合并,因为它是代码结构的一个主要重构,同时也取得了其他进展。然而,它的目的是表明DDS的实现是可以隐藏的,尽管对于如何实际实现这一目标还有讨论的余地。
Conclusion
After working with DDS and having a healthy amount of skepticism about the ethos, community, and licensing, it is hard to come up with any real technical criticisms. While it is true that the community surrounding DDS is very different from the ROS community or the ZeroMQ community, it appears that DDS is just solid technology on which ROS could safely depend. There are still many questions about exactly how ROS would utilize DDS, but they all seem like engineering exercises at this point and not potential deal breakers for ROS.
在与DDS合作并对企业精神、社区和许可证持相当数量的怀疑态度之后,很难提出任何真正的技术批评。虽然DDS周围的社区确实与ROS社区或ZeroMQ社区非常不同,但DDS似乎只是ROS可以安全依赖的可靠技术。关于ROS将如何使用DDS,仍有许多问题,但它们似乎都是目前的工程实践,而不是潜在的ROS交易破坏者。