深度学习绘图_自动驾驶制图中的深度学习

深度学习绘图

The applications of deep learning has been explored in various components throughout the autonomous driving stack, for example, in perception, prediction, and planning. Deep learning can also be used in mapping, a critical component for higher-level autonomous driving.

深度学习的应用已在整个自动驾驶堆栈的各个组件中进行了探索，例如在感知，预测和计划中。 深度学习也可以用于制图 ，这是高级自动驾驶的关键组成部分。

Having accurate maps is essential to the success of autonomous driving for routing, localization as well as to ease perception. Maps with varying degrees of information can be obtained through subscribing to the commercially available map service. However, in areas where maps are not available, self-driving vehicles need to rely on their own map building capability to ensure the functionality and safety of autonomous driving.

拥有精确的地图对于自动驾驶成功进行路由，本地化以及简化感知至关重要。 可以通过订阅市售地图服务来获取具有不同程度信息的地图。 但是，在没有地图的地区，自动驾驶汽车需要依靠自己的地图构建能力来确保自动驾驶的功能和安全性。

离线映射与在线映射 (Offline Mapping vs Online Mapping)

In an offline mapping scenario, sensor data are aggregated in a centralized location. The data could either be satellite imagery or data collected by onboard sensors such as cameras or lidars. It could be from multiple passes of the same vehicle through the same location or crowd-sourced to a fleet of cars. A rendering of the map is built offline, and human annotators are required to annotate semantic structures on the map and review the final results. Traditional mapping service works in this offline fashion, then the annotated and curated maps are then served to vehicles on the road.

在离线映射方案中，传感器数据被聚集在一个集中的位置。 数据可以是卫星图像，也可以是机载传感器(例如相机或激光雷达)收集的数据。 它可能来自同一辆车经过同一地点的多次通行，也可能来自众包车辆。 地图的呈现是离线构建的，并且需要人工注释者来注释地图上的语义结构并查看最终结果。 传统的地图服务以这种离线方式工作，然后将带注释和精选的地图提供给道路上的车辆。

Online mapping happens onboard the vehicle and are thus cannot afford to have the human in the loop. Typical examples are SLAM systems for simultaneous localization and mapping. Recently, semantic SLAM focusing on the geometry and semantic meanings of the surface markings on the road are explored as a lightweight solution to mapping. In addition, monocular semantic online mapping (monoSOM) is one trending topic where a neural network is used to fuse temporal sequences of monocular images from multiple camera into a semantic birds-eye-view map. These topics are beyond the scope of this post and will be reviewed elsewhere.

在线地图是在车辆上进行的，因此无法让人类陷入困境。 典型示例是用于同时定位和映射的SLAM系统 。 最近，重点研究道路上表面标记的几何和语义含义的语义SLAM作为一种轻量级的制图解决方案。 另外， 单眼语义在线映射(monoSOM)是一个趋势话题，其中使用神经网络将来自多个摄像机的单眼图像的时间序列融合到语义鸟瞰图中。 这些主题超出了本文的范围，将在其他地方进行讨论。

SD地图与HD地图 (SD Maps vs HD Maps)

Depending on the input resolution, there are roughly two types of applications of deep learning in mapping. The first line of work focuses on the discovery of the topology of maps, such as road network, and do not usually contain lane level information. They only require a relatively low-resolution image with roughly meter-level accuracy. The other type of application focuses on extracting lane level information such as lane lines, road surface arrows, and other semantic markings. This requires a higher resolution image with centimeter-level accuracy. Correspondingly, these two types of maps will be loosely referred to as SD maps and HD maps for the rest of this post.

根据输入分辨率，大约有两种类型的深度学习在地图中的应用。 第一行工作重点是地图拓扑的发现，例如道路网络，通常不包含车道级别信息。 他们只需要相对低分辨率的图像， 精度大约为米级 。 另一种类型的应用程序侧重于提取车道级别信息，例如车道线，路面箭头和其他语义标记。 这需要具有厘米级精度的高分辨率图像。 相应地，在本文的其余部分中，这两类地图将被简称为SD地图和HD地图。

DeepRoadMapper DeepRoadMapper DAGMapper DAGMapper

This post focuses on the offline generation of HD maps. Note that some of the methods can be applied to online mapping as well, and a short review session is dedicated to some related works of SD mapping.

这篇文章重点介绍了高清地图的离线生成。 注意，有些方法也可以应用于在线映射，简短的回顾会专门讨论SD映射的一些相关工作。

注释者友好的映射 (Annotator-friendly Mapping)

In offline mapping, it is affordable and essential to have a human reviewer to review the results of deep learning. Thus not only does the overall accuracy matter but also the presentation of the results in the sense that the results should be easy to be modified by a human annotator.

在离线地图中，让人工审阅者审阅深度学习的结果是负担得起且必不可少的。 因此，总的准确性不仅很重要，而且结果的表示也很重要，因为结果应该易于被人工注释者修改。

As most motion planners can only handle lane graphs that are structured and represent the right topology, the output of the mapping algorithm is usually a structured representation (polylines for lane lines, etc). This representation luckily lends itself to efficient modification of the human annotator.

由于大多数运动计划者只能处理结构化并代表正确拓扑的车道图，因此映射算法的输出通常是 结构化表示 (车道线的折线等)。 幸运的是，这种表示形式有助于对人类注释者进行有效的修改。

SD映射(道路拓扑发现) (SD Mapping (Road Topology Discovery))

Early application of deep learning on mapping focuses on extracting road level topology from relatively low-resolution aerial images. Deep learning creates an affordable solution with large coverage for SD mapping. The primary purpose of the generated road topology in SD maps is relatively limited in the context of autonomous driving— for routing and navigation. Yet the methodologies proposed in some of these studies are highly relevant to later works on HD maps and are thus reviewed here.

深度学习在地图上的早期应用着重于从相对低分辨率的航空图像中提取道路拓扑。 深度学习创建了一种负担得起的解决方案，涵盖了SD映射的大量内容。 SD地图中生成的道路拓扑的主要目的在自动驾驶(用于路线选择和导航)的环境中相对受限。 然而，其中一些研究提出的方法与高清地图的后续工作高度相关，因此在此进行了综述。

DeepRoadMapper