CARLA 笔记（08）— 传感器和数据（传感器 Setting、Spawning、Listening、Data 以及传感器类型 Cameras、Detectors）

传感器是用于从周围环境中采集数据。它们对于为驾驶员创造学习环境至关重要。每一个传感器具体细节可以在 https://carla.readthedocs.io/en/0.9.13/ref_sensors/ 中查到。

1. 传感器搭建步骤

carla.Sensor 定义了一些特殊的能够测量和分析数据的角色。

• 数据是什么？ 它取决于传感器的类型，所有类型的数据都继承自通用的 carla.SensorData
• 什么时候搜集数据？无论是在每个仿真步骤中，还是在某个事件触发时。这取决于传感器的类型。
• 怎样搜集数据？每个传感器都有一个 listen() 方法来接收和管理数据。

尽管不同，所有传感器的使用方式都是相似的。

1.1 Setting

与其他参与者一样，查找蓝图并设置特定属性。这在处理传感器时是必不可少的。它们的属性将决定所获得的结果。这些在 传感器参考中有详细说明。

设置仪表盘高清摄像头的示例如下。

# Find the blueprint of the sensor.
blueprint = world.get_blueprint_library().find('sensor.camera.rgb')
# Modify the attributes of the blueprint to set image resolution and field of view.
blueprint.set_attribute('image_size_x', '1920')
blueprint.set_attribute('image_size_y', '1080')
blueprint.set_attribute('fov', '110')
# Set the time in seconds between sensor captures
blueprint.set_attribute('sensor_tick', '1.0')

1.2 Spawning

attachment_to 和 attachment_type 是至关重要的。传感器应该连接到父参与者，通常是一辆车，跟踪它并收集信息。 attachment_type 将决定其位置如何更新有关车辆。

• Rigid attachment. 移动对于它的父位置是严格的。这是从模拟中检索数据的适当 attachment。
• SpringArm attachment. 运动过程中很少有加速和减速。此 attachment 仅建议用于录制模拟视频。运动是平稳的，“跳跃”是避免更新相机的位置。

transform = carla.Transform(carla.Location(x=0.8, z=1.7))
sensor = world.spawn_actor(blueprint, transform, attach_to=my_vehicle)

When spawning with attachment, location must be relative to the parent actor.

1.3 Listening

每个传感器都有一个 listen() 方法。每次传感器检索数据时都会调用这个函数。参数 callback 是一个 lambda 函数。它描述了检索数据时传感器应该做什么。它必须将检索的数据作为参数。

# do_something() will be called each time a new image is generated by the camera.
sensor.listen(lambda data: do_something(data))

...

# This collision sensor would print everytime a collision is detected. 
def callback(event):
    for actor_id in event:
        vehicle = world_ref().get_actor(actor_id)
        print('Vehicle too close: %s' % vehicle.type_id)

sensor02.listen(callback)

1.4 Data

大多数传感器数据对象都有将信息保存到磁盘的功能。这将允许它在其他环境中使用。

不同传感器类型的传感器数据差异很大。看一看 ref_sensors 得到详细的解释。然而，所有这些都被标记了一些基本信息。

Sensor data attribute	Type	Description
frame	int	Frame number when the measurement took place.
timestamp	double	Timestamp of the measurement in simulation seconds since the beginning of the episode.
transform	carla.Transform	World reference of the sensor at the time of the measurement.

• is_listening is a sensor attribute that enables/disables data listening at will.
• sensor_tick is a blueprint attribute that sets the simulation time between data received.

2. Types of sensors

2.1 Cameras

从摄像机的角度下观察这个世界，返回 carla.Image 对象，可以使用 carla.ColorConverter 类来修改图像用于表示不同的信息。

• Retrieve data every simulation step.

Sensor	Output	Overview
Depth	carla.Image	Renders the depth of the elements in the field of view in a gray-scale map.
RGB	carla.Image	Provides clear vision of the surroundings. Looks like a normal photo of the scene.
Optical Flow	carla.Image	Renders the motion of every pixel from the camera.
Semantic segmentation	carla.Image	Renders elements in the field of view with a specific color according to their tags.
DVS	carla.DVSEventArray	Measures changes of brightness intensity asynchronously as an event stream.

2.2 Detectors

Retrieve data when the object they are attached to registers a specific event.

• Retrieve data when triggered.

Sensor	Output	Overview
Collision	carla.CollisionEvent	Retrieves collisions between its parent and other actors.
Lane invasion	carla.LaneInvasionEvent	Registers when its parent crosses a lane marking.
Obstacle	carla.ObstacleDetectionEvent	Detects possible obstacles ahead of its parent.

2.3 Other

不同的功能，如导航、物理属性测量和场景的2D/3D点地图。

• Retrieve data every simulation step.

Sensor	Output	Overview
GNSS	carla.GNSSMeasurement	Retrieves the geolocation of the sensor.
IMU	carla.IMUMeasurement	Comprises an accelerometer, a gyroscope, and a compass.
LIDAR	carla.LidarMeasurement	A rotating LIDAR. Generates a 4D point cloud with coordinates and intensity per point to model the surroundings.
Radar	carla.RadarMeasurement	2D point map modelling elements in sight and their movement regarding the sensor.
RSS	carla.RssResponse	Modifies the controller applied to a vehicle according to safety checks. This sensor works in a different manner than the rest, and there is specific RSS documentation for it.
Semantic LIDAR	carla.SemanticLidarMeasurement	A rotating LIDAR. Generates a 3D point cloud with extra information regarding instance and semantic segmentation.