Carla系列——4.Cara模拟器添加语义分割相机（Semantic segmentation camera）

本节的内容是生成一个语义分割相机，主要分为：四部分。

一、初始化世界以及添加车辆

二、生成语义分割相机Semantic segmentation camera

三、监听语义分割相机，进行数据处理

四、呈现最终效果

 

一、初始化世界以及添加车辆

这一节承接之前生成车辆的内容（Carla生成车辆）.设置vehicle 为自动控制模式

1.添加车辆的完整代码为：

import glob
import os
import sys
import time
import random
import time
import numpy as np
import cv2	
try:
    sys.path.append(glob.glob('../carla/dist/carla-*%d.%d-%s.egg' % (
        sys.version_info.major,
        sys.version_info.minor,
        'win-amd64' if os.name == 'nt' else 'linux-x86_64'))[0])
except IndexError:
    pass
 
import carla
 
actor_list =  []
try:
	#create client
	client = carla.Client('localhost', 2000)
	client.set_timeout(4.0)
	world = client.get_world() 
	blueprint_library = world.get_blueprint_library()
	bp = blueprint_library.filter("model3")[0]
	spawn_point = random.choice(world.get_map().get_spawn_points())
	vehicle = world.spawn_actor(bp,spawn_point)
    #设置vehicle 为自动控制模式
	vehicle.set_autopilot(enabled=True)
	actor_list.append(vehicle)
	time.sleep(50)
finally:
	for actor in actor_list:
		actor.destroy()
	print("All cleaned up!")

二、添加语义分割相机（Semantic segmentation camera）

1.添加语义分割相机（Semantic segmentation camera）， 生成语义分割相机（Semantic segmentation camera）的方法类似于生成vehicle。

    sem_bp = world.get_blueprint_library().find('sensor.camera.semantic_segmentation')

2.设置Semantic segmentation camera的属性

    IM_WIDTH = 640
    IM_HEIIGHT = 480 
	sem_bp.set_attribute("image_size_x",f"{IM_WIDTH}")
	sem_bp.set_attribute("image_size_y",f"{IM_HEIIGHT}")
	sem_bp.set_attribute("fov",str(105))
	sem_location = carla.Location(0,0,2.5)
	sem_rotation = carla.Rotation(0,0,0)
	sem_transform = carla.Transform(sem_location,sem_rotation)

3.添加Semantic segmentation camera到world中，并在actor list 中添加Semantic segmentation camera。

	sem_cam = world.spawn_actor(sem_bp,sem_transform,attach_to=vehicle,         
                                 attachment_type=carla.AttachmentType.Rigid)
	actor_list.append(sem_cam)

三、监听Semantic segmentation camera数据，并处理Semantic segmentation camera的数据

    sem_cam.listen(lambda image: process_semantic(image))
    def process_semantic(image):
        #将原始图像（即image）转化成语义分割图像
	    image.convert(carla.ColorConverter.CityScapesPalette)
        #对转化后的image进行数据处理，image的初始shape是height*width*4，类型为 32-bit BGRA 
	    array = np.frombuffer(image.raw_data, dtype=np.dtype("uint8"))
	    array = np.reshape(array, (image.height, image.width, 4))
        # 转化后的数据，提取RGB 故只需要前3列数据
	    array = array[:, :, :3]
        #显示图像
	    cv2.imshow("",array)
	    cv2.waitKey(20)
        #return array/255.0 是归一化图像
	    return array/255.0

6.最终完整的代码为

import glob
import os
import sys
import time
import random
import time
import numpy as np
import cv2		
try:
    sys.path.append(glob.glob('../carla/dist/carla-*%d.%d-%s.egg' % (
        sys.version_info.major,
        sys.version_info.minor,
        'win-amd64' if os.name == 'nt' else 'linux-x86_64'))[0])
except IndexError:
    pass
 
import carla
from carla import Transform, Location, Rotation
IM_WIDTH = 640
IM_HEIIGHT = 480 
 
def process_semantic(image):
	image.convert(carla.ColorConverter.CityScapesPalette)
	array = np.frombuffer(image.raw_data, dtype=np.dtype("uint8"))
	array = np.reshape(array, (image.height, image.width, 4))
	array = array[:, :, :3]
	cv2.imshow("",array)
	cv2.waitKey(20)
	return array/255.0
actor_list =  []
try:
	# 连接master
	client = carla.Client('localhost', 2000)
	client.set_timeout(5.0)
	world = client.get_world() 
	blueprint_library = world.get_blueprint_library()
	bp = blueprint_library.filter("model3")[0]
	spawn_point = Transform(Location(x=5.4, y=30.5, z=1.5), Rotation(pitch=0, yaw=-75.8, roll=0.0))
	vehicle = world.spawn_actor(bp,spawn_point)

	vehicle.set_autopilot(enabled=True)
	actor_list.append(vehicle)
	# 添加一个语义分割相机
	sem_cam = None
	sem_bp = world.get_blueprint_library().find('sensor.camera.semantic_segmentation')
	sem_bp.set_attribute("image_size_x",f"{IM_WIDTH}")
	sem_bp.set_attribute("image_size_y",f"{IM_HEIIGHT}")
	sem_bp.set_attribute("fov",str(105))
	sem_location = carla.Location(0,0,2.5)
	sem_rotation = carla.Rotation(0,0,0)
	sem_transform = carla.Transform(sem_location,sem_rotation)
	sem_cam = world.spawn_actor(sem_bp,sem_transform,attach_to=vehicle, attachment_type=carla.AttachmentType.Rigid)
	actor_list.append(sem_cam)
    #监听相机，并显示图像
	sem_cam.listen(lambda image: process_semantic(image))
	time.sleep(50)

finally:
	for actor in actor_list:
		actor.destroy()
	print("All cleaned up!")

7.保存代码为spawn_Semantic_camera.py

8.运行CarlaUE4.exe（位于WindowsNoEditor\文件夹下）

9.打开终端运行spawn_Semantic_camera.py （注意terminal的路径应该是你存放py文件的位置） python spawn_Semantic_camera.py