0. 前言
- 全景分割调研系列文章分为4篇
- 问题引入与性能指标
- 常用数据集（本文）
- 当前研究现状（现有模型分类以及当前常见研究方向）
- 根据 paperswithcode 中描述，常用的全景分割数据集包括 COCO/Cityscapes/Mapillary/KITTI/IDD，下面分别介绍
1 COCO
- 官网，Github
- 官网直接下载（只不过如果没有梯子，随缘进官网）
- 数据：就是COCO2017的train/val/test数据集
- 类别：
- 前景（things）就是COCO检测的80类
- 背景（stuff）就是 COCO Stuff segmentation 任务中的子集，共36类，36类经过合并后共17类。
- 具体类别查看附录。
2 Cityscapes
- 官网，Github
- 分割领域最常用的数据集之一。
- 官网注册（可能要用学校邮箱，企业邮箱好像有点问题）并下载。
- 提供的标签是instance segmentation的标签，如果用于 panoptic segmentation，则需要用到
csCreatePanopticImgs工具。 - 数据量：共5000张图片，共30类物体，50个城市，不同季节，白天，较好的天气，手动筛选视频帧。
- 标签详解
- 代码参考这里
- 前景和背景猜测根据
hasInstances列划分。
- 类别查看附录
3 Mapillary
- 官网
- 下载：官网注册直接下载。
- 基本情况：
- 自动驾驶场景（街景）
- 25000张高清图片，横跨六大洲。
- 124个类别，77类things。
- 不同的天气、时间、摄像头、视角。
- licence： Creative Commons Attribution NonCommercial Share Alike (CC BY-NC-SA)
4 KITTI
- SemanticKITTI
- 该数据集是基于 Kitti odometry 的，换句话说，原始图像要下载 odometry 的 color数据。分割标签在前面的官网中下载。
- sequence 00-10 有详细标注结果，11-21 作为 test set，在 evalution server 上使用。
- 这里的全景分割数据是基于点云的，不是基于RGB图像的。
- 分类类别：28类
- EfficientPS
- EfficientPS 论文的作者选择了 Kitti 中的部分图片，标注对应的全景分割标签。
- 标签文件在上面的官网中可以直接下载得到，图片应该是从Kitti中选的吧，不知道情况如何。
- 共有1055张图片，855训练集，200验证集。
- 图片分辨率 1280*384
- 11类背景，8类前景，类别从 Cityscapes 中选择
5 Indian Driving Dataset
-
简称IDD，官网
-
基本情况
- 是一个在印度拍摄的自动驾驶场景数据集
- 免费注册，无需认证，免费下载数据集。
- 数据集有很多部分，包括了分割、检测等，目前只研究了分割，其他的没细看。
-
分割数据集基本情况
- 摄像头是装在车上，向前拍摄。
- 图像大部分是1080p，少部分是720p。
- 类别分为四个层级，具体信息参考下面配图。全景分割是，前景（things）类别就是所有 vehicles 和 living things 中的所有类别，其他的都是背景（stuff）
- 数据集图片、视频分布如下
附录
COCO 数据集类别
[
{
"supercategory":"person",
"color":[
220,
20,
60
],
"isthing":1,
"id":1,
"name":"person"
},
{
"supercategory":"vehicle",
"color":[
119,
11,
32
],
"isthing":1,
"id":2,
"name":"bicycle"
},
{
"supercategory":"vehicle",
"color":[
0,
0,
142
],
"isthing":1,
"id":3,
"name":"car"
},
{
"supercategory":"vehicle",
"color":[
0,
0,
230
],
"isthing":1,
"id":4,
"name":"motorcycle"
},
{
"supercategory":"vehicle",
"color":[
106,
0,
228
],
"isthing":1,
"id":5,
"name":"airplane"
},
{
"supercategory":"vehicle",
"color":[
0,
60,
100
],
"isthing":1,
"id":6,
"name":"bus"
},
{
"supercategory":"vehicle",
"color":[
0,
80,
100
],
"isthing":1,
"id":7,
"name":"train"
},
{
"supercategory":"vehicle",
"color":[
0,
0,
70
],
"isthing":1,
"id":8,
"name":"truck"
},
{
"supercategory":"vehicle",
"color":[
0,
0,
192
],
"isthing":1,
"id":9,
"name":"boat"
},
{
"supercategory":"outdoor",
"color":[
250,
170,
30
],
"isthing":1,
"id":10,
"name":"traffic light"
},
{
"supercategory":"outdoor",
"color":[
100,
170,
30
],
"isthing":1,
"id":11,
"name":"fire hydrant"
},
{
"supercategory":"outdoor",
"color":[
220,
220,
0
],
"isthing":1,
"id":13,
"name":"stop sign"
},
{
"supercategory":"outdoor",
"color":[
175,
116,
175
],
"isthing":1,
"id":14,
"name":"parking meter"
},
{
"supercategory":"outdoor",
"color":[
250,
0,
30
],
"isthing":1,
"id":15,
"name":"bench"
},
{
"supercategory":"animal",
"color":[
165,
42,
42
],
"isthing":1,
"id":16,
"name":"bird"
},
{
"supercategory":"animal",
"color":[
255,
77,
255
],
"isthing":1,
"id":17,
"name":"cat"
},
{
"supercategory":"animal",
"color":[
0,
226,
252
],
"isthing":1,
"id":18,
"name":"dog"
},
{
"supercategory":"animal",
"color":[
182,
182,
255
],
"isthing":1,
"id":19,
"name":"horse"
},
{
"supercategory":"animal",
"color":[
0,
82,
0
],
"isthing":1,
"id":20,
"name":"sheep"
},
{
"supercategory":"animal",
"color":[
120,
166,
157
],
"isthing":1,
"id":21,
"name":"cow"
},
{
"supercategory":"animal",
"color":[
110,
76,
0
],
"isthing":1,
"id":22,
"name":"elephant"
},
{
"supercategory":"animal",
"color":[
174,
57,
255
],
"isthing":1,
"id":23,
"name":"bear"
},
{
"supercategory":"animal",
"color":[
199,
100,
0
],
"isthing":1,
"id":24,
"name":"zebra"
},
{
"supercategory":"animal",
"color":[
72,
0,
118
],
"isthing":1,
"id":25,
"name":"giraffe"
},
{
"supercategory":"accessory",
"color":[
255,
179,
240
],
"isthing":1,
"id":27,
"name":"backpack"
},
{
"supercategory":"accessory",
"color":[
0,
125,
92
],
"isthing":1,
"id":28,
"name":"umbrella"
},
{
"supercategory":"accessory",
"color":[
209,
0,
151
],
"isthing":1,
"id":31,
"name":"handbag"
},
{
"supercategory":"accessory",
"color":[
188,
208,
182
],
"isthing":1,
"id":32,
"name":"tie"
},
{
"supercategory":"accessory",
"color":[
0,
220,
176
],
"isthing":1,
"id":33,
"name":"suitcase"
},
{
"supercategory":"sports",
"color":[
255,
99,
164
],
"isthing":1,
"id":34,
"name":"frisbee"
},
{
"supercategory":"sports",
"color":[
92,
0,
73
],
"isthing":1,
"id":35,
"name":"skis"
},
{
"supercategory":"sports",
"color":[
133,
129,
255
],
"isthing":1,
"id":36,
"name":"snowboard"
},
{
"supercategory":"sports",
"color":[
78,
180,
255
],
"isthing":1,
"id":37,
"name":"sports ball"
},
{
"supercategory":"sports",
"color":[
0,
228,
0
],
"isthing":1,
"id":38,
"name":"kite"
},
{
"supercategory":"sports",
"color":[
174,
255,
243
],
"isthing":1,
"id":39,
"name":"baseball bat"
},
{
"supercategory":"sports",
"color":[
45,
89,
255
],
"isthing":1,
"id":40,
"name":"baseball glove"
},
{
"supercategory":"sports",
"color":[
134,
134,
103
],
"isthing":1,
"id":41,
"name":"skateboard"
},
{
"supercategory":"sports",
"color":[
145,
148,
174
],
"isthing":1,
"id":42,
"name":"surfboard"
},
{
"supercategory":"sports",
"color":[
255,
208,
186
],
"isthing":1,
"id":43,
"name":"tennis racket"
},
{
"supercategory":"kitchen",
"color":[
197,
226,
255
],
"isthing":1,
"id":44,
"name":"bottle"
},
{
"supercategory":"kitchen",
"color":[
171,
134,
1
],
"isthing":1,
"id":46,
"name":"wine glass"
},
{
"supercategory":"kitchen",
"color":[
109,
63,
54
],
"isthing":1,
"id":47,
"name":"cup"
},
{
"supercategory":"kitchen",
"color":[
207,
138,
255
],
"isthing":1,
"id":48,
"name":"fork"
},
{
"supercategory":"kitchen",
"color":[
151,
0,
95
],
"isthing":1,
"id":49,
"name":"knife"
},
{
"supercategory":"kitchen",
"color":[
9,
80,
61
],
"isthing":1,
"id":50,
"name":"spoon"
},
{
"supercategory":"kitchen",
"color":[
84,
105,
51
],
"isthing":1,
"id":51,
"name":"bowl"
},
{
"supercategory":"food",
"color":[
74,
65,
105
],
"isthing":1,
"id":52,
"name":"banana"
},
{
"supercategory":"food",
"color":[
166,
196,
102
],
"isthing":1,
"id":53,
"name":"apple"
},
{
"supercategory":"food",
"color":[
208,
195,
210
],
"isthing":1,
"id":54,
"name":"sandwich"
},
{
"supercategory":"food",
"color":[
255,
109,
65
],
"isthing":1,
"id":55,
"name":"orange"
},
{
"supercategory":"food",
"color":[
0,
143,
149
],
"isthing":1,
"id":56,
"name":"broccoli"
},
{
"supercategory":"food",
"color":[
179,
0,
194
],
"isthing":1,
"id":57,
"name":"carrot"
},
{
"supercategory":"food",
"color":[
209,
99,
106
],
"isthing":1,
"id":58,
"name":"hot dog"
},
{
"supercategory":"food",
"color":[
5,
121,
0
],
"isthing":1,
"id":59,
"name":"pizza"
},
{
"supercategory":"food",
"color":[
227,
255,
205
],
"isthing":1,
"id":60,
"name":"donut"
},
{
"supercategory":"food",
"color":[
147,
186,
208
],
"isthing":1,
"id":61,
"name":"cake"
},
{
"supercategory":"furniture",
"color":[
153,
69,
1
],
"isthing":1,
"id":62,
"name":"chair"
},
{
"supercategory":"furniture",
"color":[
3,
95,
161
],
"isthing":1,
"id":63,
"name":"couch"
},
{
"supercategory":"furniture",
"color":[
163,
255,
0
],
"isthing":1,
"id":64,
"name":"potted plant"
},
{
"supercategory":"furniture",
"color":[
119,
0,
170
],
"isthing":1,
"id":65,
"name":"bed"
},
{
"supercategory":"furniture",
"color":[
0,
182,
199
],
"isthing":1,
"id":67,
"name":"dining table"
},
{
"supercategory":"furniture",
"color":[
0,
165,
120
],
"isthing":1,
"id":70,
"name":"toilet"
},
{
"supercategory":"electronic",
"color":[
183,
130,
88
],
"isthing":1,
"id":72,
"name":"tv"
},
{
"supercategory":"electronic",
"color":[
95,
32,
0
],
"isthing":1,
"id":73,
"name":"laptop"
},
{
"supercategory":"electronic",
"color":[
130,
114,
135
],
"isthing":1,
"id":74,
"name":"mouse"
},
{
"supercategory":"electronic",
"color":[
110,
129,
133
],
"isthing":1,
"id":75,
"name":"remote"
},
{
"supercategory":"electronic",
"color":[
166,
74,
118
],
"isthing":1,
"id":76,
"name":"keyboard"
},
{
"supercategory":"electronic",
"color":[
219,
142,
185
],
"isthing":1,
"id":77,
"name":"cell phone"
},
{
"supercategory":"appliance",
"color":[
79,
210,
114
],
"isthing":1,
"id":78,
"name":"microwave"
},
{
"supercategory":"appliance",
"color":[
178,
90,
62
],
"isthing":1,
"id":79,
"name":"oven"
},
{
"supercategory":"appliance",
"color":[
65,
70,
15
],
"isthing":1,
"id":80,
"name":"toaster"
},
{
"supercategory":"appliance",
"color":[
127,
167,
115
],
"isthing":1,
"id":81,
"name":"sink"
},
{
"supercategory":"appliance",
"color":[
59,
105,
106
],
"isthing":1,
"id":82,
"name":"refrigerator"
},
{
"supercategory":"indoor",
"color":[
142,
108,
45
],
"isthing":1,
"id":84,
"name":"book"
},
{
"supercategory":"indoor",
"color":[
196,
172,
0
],
"isthing":1,
"id":85,
"name":"clock"
},
{
"supercategory":"indoor",
"color":[
95,
54,
80
],
"isthing":1,
"id":86,
"name":"vase"
},
{
"supercategory":"indoor",
"color":[
128,
76,
255
],
"isthing":1,
"id":87,
"name":"scissors"
},
{
"supercategory":"indoor",
"color":[
201,
57,
1
],
"isthing":1,
"id":88,
"name":"teddy bear"
},
{
"supercategory":"indoor",
"color":[
246,
0,
122
],
"isthing":1,
"id":89,
"name":"hair drier"
},
{
"supercategory":"indoor",
"color":[
191,
162,
208
],
"isthing":1,
"id":90,
"name":"toothbrush"
},
{
"supercategory":"textile",
"color":[
255,
255,
128
],
"isthing":0,
"id":92,
"name":"banner"
},
{
"supercategory":"textile",
"color":[
147,
211,
203
],
"isthing":0,
"id":93,
"name":"blanket"
},
{
"supercategory":"building",
"color":[
150,
100,
100
],
"isthing":0,
"id":95,
"name":"bridge"
},
{
"supercategory":"raw-material",
"color":[
168,
171,
172
],
"isthing":0,
"id":100,
"name":"cardboard"
},
{
"supercategory":"furniture-stuff",
"color":[
146,
112,
198
],
"isthing":0,
"id":107,
"name":"counter"
},
{
"supercategory":"textile",
"color":[
210,
170,
100
],
"isthing":0,
"id":109,
"name":"curtain"
},
{
"supercategory":"furniture-stuff",
"color":[
92,
136,
89
],
"isthing":0,
"id":112,
"name":"door-stuff"
},
{
"supercategory":"floor",
"color":[
218,
88,
184
],
"isthing":0,
"id":118,
"name":"floor-wood"
},
{
"supercategory":"plant",
"color":[
241,
129,
0
],
"isthing":0,
"id":119,
"name":"flower"
},
{
"supercategory":"food-stuff",
"color":[
217,
17,
255
],
"isthing":0,
"id":122,
"name":"fruit"
},
{
"supercategory":"ground",
"color":[
124,
74,
181
],
"isthing":0,
"id":125,
"name":"gravel"
},
{
"supercategory":"building",
"color":[
70,
70,
70
],
"isthing":0,
"id":128,
"name":"house"
},
{
"supercategory":"furniture-stuff",
"color":[
255,
228,
255
],
"isthing":0,
"id":130,
"name":"light"
},
{
"supercategory":"furniture-stuff",
"color":[
154,
208,
0
],
"isthing":0,
"id":133,
"name":"mirror-stuff"
},
{
"supercategory":"structural",
"color":[
193,
0,
92
],
"isthing":0,
"id":138,
"name":"net"
},
{
"supercategory":"textile",
"color":[
76,
91,
113
],
"isthing":0,
"id":141,
"name":"pillow"
},
{
"supercategory":"ground",
"color":[
255,
180,
195
],
"isthing":0,
"id":144,
"name":"platform"
},
{
"supercategory":"ground",
"color":[
106,
154,
176
],
"isthing":0,
"id":145,
"name":"playingfield"
},
{
"supercategory":"ground",
"color":[
230,
150,
140
],
"isthing":0,
"id":147,
"name":"railroad"
},
{
"supercategory":"water",
"color":[
60,
143,
255
],
"isthing":0,
"id":148,
"name":"river"
},
{
"supercategory":"ground",
"color":[
128,
64,
128
],
"isthing":0,
"id":149,
"name":"road"
},
{
"supercategory":"building",
"color":[
92,
82,
55
],
"isthing":0,
"id":151,
"name":"roof"
},
{
"supercategory":"ground",
"color":[
254,
212,
124
],
"isthing":0,
"id":154,
"name":"sand"
},
{
"supercategory":"water",
"color":[
73,
77,
174
],
"isthing":0,
"id":155,
"name":"sea"
},
{
"supercategory":"furniture-stuff",
"color":[
255,
160,
98
],
"isthing":0,
"id":156,
"name":"shelf"
},
{
"supercategory":"ground",
"color":[
255,
255,
255
],
"isthing":0,
"id":159,
"name":"snow"
},
{
"supercategory":"furniture-stuff",
"color":[
104,
84,
109
],
"isthing":0,
"id":161,
"name":"stairs"
},
{
"supercategory":"building",
"color":[
169,
164,
131
],
"isthing":0,
"id":166,
"name":"tent"
},
{
"supercategory":"textile",
"color":[
225,
199,
255
],
"isthing":0,
"id":168,
"name":"towel"
},
{
"supercategory":"wall",
"color":[
137,
54,
74
],
"isthing":0,
"id":171,
"name":"wall-brick"
},
{
"supercategory":"wall",
"color":[
135,
158,
223
],
"isthing":0,
"id":175,
"name":"wall-stone"
},
{
"supercategory":"wall",
"color":[
7,
246,
231
],
"isthing":0,
"id":176,
"name":"wall-tile"
},
{
"supercategory":"wall",
"color":[
107,
255,
200
],
"isthing":0,
"id":177,
"name":"wall-wood"
},
{
"supercategory":"water",
"color":[
58,
41,
149
],
"isthing":0,
"id":178,
"name":"water-other"
},
{
"supercategory":"window",
"color":[
183,
121,
142
],
"isthing":0,
"id":180,
"name":"window-blind"
},
{
"supercategory":"window",
"color":[
255,
73,
97
],
"isthing":0,
"id":181,
"name":"window-other"
},
{
"supercategory":"plant",
"color":[
107,
142,
35
],
"isthing":0,
"id":184,
"name":"tree-merged"
},
{
"supercategory":"structural",
"color":[
190,
153,
153
],
"isthing":0,
"id":185,
"name":"fence-merged"
},
{
"supercategory":"ceiling",
"color":[
146,
139,
141
],
"isthing":0,
"id":186,
"name":"ceiling-merged"
},
{
"supercategory":"sky",
"color":[
70,
130,
180
],
"isthing":0,
"id":187,
"name":"sky-other-merged"
},
{
"supercategory":"furniture-stuff",
"color":[
134,
199,
156
],
"isthing":0,
"id":188,
"name":"cabinet-merged"
},
{
"supercategory":"furniture-stuff",
"color":[
209,
226,
140
],
"isthing":0,
"id":189,
"name":"table-merged"
},
{
"supercategory":"floor",
"color":[
96,
36,
108
],
"isthing":0,
"id":190,
"name":"floor-other-merged"
},
{
"supercategory":"ground",
"color":[
96,
96,
96
],
"isthing":0,
"id":191,
"name":"pavement-merged"
},
{
"supercategory":"solid",
"color":[
64,
170,
64
],
"isthing":0,
"id":192,
"name":"mountain-merged"
},
{
"supercategory":"plant",
"color":[
152,
251,
152
],
"isthing":0,
"id":193,
"name":"grass-merged"
},
{
"supercategory":"ground",
"color":[
208,
229,
228
],
"isthing":0,
"id":194,
"name":"dirt-merged"
},
{
"supercategory":"raw-material",
"color":[
206,
186,
171
],
"isthing":0,
"id":195,
"name":"paper-merged"
},
{
"supercategory":"food-stuff",
"color":[
152,
161,
64
],
"isthing":0,
"id":196,
"name":"food-other-merged"
},
{
"supercategory":"building",
"color":[
116,
112,
0
],
"isthing":0,
"id":197,
"name":"building-other-merged"
},
{
"supercategory":"solid",
"color":[
0,
114,
143
],
"isthing":0,
"id":198,
"name":"rock-merged"
},
{
"supercategory":"wall",
"color":[
102,
102,
156
],
"isthing":0,
"id":199,
"name":"wall-other-merged"
},
{
"supercategory":"textile",
"color":[
250,
141,
255
],
"isthing":0,
"id":200,
"name":"rug-merged"
}
]
Cityscapes 类别
labels = [
# name id trainId category catId hasInstances ignoreInEval color
Label( 'unlabeled' , 0 , 255 , 'void' , 0 , False , True , ( 0, 0, 0) ),
Label( 'ego vehicle' , 1 , 255 , 'void' , 0 , False , True , ( 0, 0, 0) ),
Label( 'rectification border' , 2 , 255 , 'void' , 0 , False , True , ( 0, 0, 0) ),
Label( 'out of roi' , 3 , 255 , 'void' , 0 , False , True , ( 0, 0, 0) ),
Label( 'static' , 4 , 255 , 'void' , 0 , False , True , ( 0, 0, 0) ),
Label( 'dynamic' , 5 , 255 , 'void' , 0 , False , True , (111, 74, 0) ),
Label( 'ground' , 6 , 255 , 'void' , 0 , False , True , ( 81, 0, 81) ),
Label( 'road' , 7 , 0 , 'flat' , 1 , False , False , (128, 64,128) ),
Label( 'sidewalk' , 8 , 1 , 'flat' , 1 , False , False , (244, 35,232) ),
Label( 'parking' , 9 , 255 , 'flat' , 1 , False , True , (250,170,160) ),
Label( 'rail track' , 10 , 255 , 'flat' , 1 , False , True , (230,150,140) ),
Label( 'building' , 11 , 2 , 'construction' , 2 , False , False , ( 70, 70, 70) ),
Label( 'wall' , 12 , 3 , 'construction' , 2 , False , False , (102,102,156) ),
Label( 'fence' , 13 , 4 , 'construction' , 2 , False , False , (190,153,153) ),
Label( 'guard rail' , 14 , 255 , 'construction' , 2 , False , True , (180,165,180) ),
Label( 'bridge' , 15 , 255 , 'construction' , 2 , False , True , (150,100,100) ),
Label( 'tunnel' , 16 , 255 , 'construction' , 2 , False , True , (150,120, 90) ),
Label( 'pole' , 17 , 5 , 'object' , 3 , False , False , (153,153,153) ),
Label( 'polegroup' , 18 , 255 , 'object' , 3 , False , True , (153,153,153) ),
Label( 'traffic light' , 19 , 6 , 'object' , 3 , False , False , (250,170, 30) ),
Label( 'traffic sign' , 20 , 7 , 'object' , 3 , False , False , (220,220, 0) ),
Label( 'vegetation' , 21 , 8 , 'nature' , 4 , False , False , (107,142, 35) ),
Label( 'terrain' , 22 , 9 , 'nature' , 4 , False , False , (152,251,152) ),
Label( 'sky' , 23 , 10 , 'sky' , 5 , False , False , ( 70,130,180) ),
Label( 'person' , 24 , 11 , 'human' , 6 , True , False , (220, 20, 60) ),
Label( 'rider' , 25 , 12 , 'human' , 6 , True , False , (255, 0, 0) ),
Label( 'car' , 26 , 13 , 'vehicle' , 7 , True , False , ( 0, 0,142) ),
Label( 'truck' , 27 , 14 , 'vehicle' , 7 , True , False , ( 0, 0, 70) ),
Label( 'bus' , 28 , 15 , 'vehicle' , 7 , True , False , ( 0, 60,100) ),
Label( 'caravan' , 29 , 255 , 'vehicle' , 7 , True , True , ( 0, 0, 90) ),
Label( 'trailer' , 30 , 255 , 'vehicle' , 7 , True , True , ( 0, 0,110) ),
Label( 'train' , 31 , 16 , 'vehicle' , 7 , True , False , ( 0, 80,100) ),
Label( 'motorcycle' , 32 , 17 , 'vehicle' , 7 , True , False , ( 0, 0,230) ),
Label( 'bicycle' , 33 , 18 , 'vehicle' , 7 , True , False , (119, 11, 32) ),
Label( 'license plate' , -1 , -1 , 'vehicle' , 7 , False , True , ( 0, 0,142) ),
]
扫一扫
1374
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
