Mask-rcnn做检测应用

这几天在接触mask-rcnn的原理和检测数据，包括数据集的制作和标注，在一块1080ti上训练模型，到目前位置还有问题没有解决，在matterport/mask-rcnn的github仓库的issues下找解决方案，没有找到能够解决我的问题的回答。

我的问题的在balloon案例上做修改，对标记多个class的数据进行识别和实例分割，balloon是对一个class做检测，我也尝试了只对我的一个类别进行训练，是能够训练，并得到检测结果的，但是一般的应用都是需要进行多个类别的检测和识别，街上的行人，汽车，障碍物，交通灯等等都是无人驾驶的检测对象，如果我们尝试的视觉解决方案采用mask-rcnn做实验，就必要要进行多个类别的检测，所以必须探究多个class的实例分割，下面将相关的代码放上来，作为参考使用。

训练代码

#Mask R-CNN


import os
import sys
import json
import datetime
import numpy as np
import skimage.draw
from skimage import *
# Root directory of the project
ROOT_DIR = os.path.abspath("../../")

# Import Mask RCNN
sys.path.append(ROOT_DIR)  # To find local version of the library
from mrcnn.config import Config
from mrcnn import model as modellib, utils

# Path to trained weights file
COCO_WEIGHTS_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")

# Directory to save logs and model checkpoints, if not provided
# through the command line argument --logs
DEFAULT_LOGS_DIR = os.path.join(ROOT_DIR, "logs")

############################################################
#  Configurations
############################################################


class DefectConfig(Config):
    """Configuration for training on the toy  dataset.
    Derives from the base Config class and overrides some values.
    """
    # Give the configuration a recognizable name
    NAME = "defect"

    # We use a GPU with 12GB memory, which can fit two images.
    # Adjust down if you use a smaller GPU.
    IMAGES_PER_GPU = 1

    # Number of classes (including background)
    NUM_CLASSES = 1 + 6  # Background + defect

    # Number of training steps per epoch
    STEPS_PER_EPOCH = 100

    # Skip detections with < 90% confidence
    DETECTION_MIN_CONFIDENCE = 0.9


############################################################
#  Dataset
############################################################

class DefectDataset(utils.Dataset):

    def load_defect(self, dataset_dir, subset):
        """Load a subset of the Balloon dataset.
        dataset_dir: Root directory of the dataset.
        subset: Subset to load: train or val
        """
        # Add classes. We have 6 classes to add.
        self.add_class("x", 1, "a")
        self.add_class("x", 2, "b")
        self.add_class("x", 3, "c")
        self.add_class("x", 4, "d")
        self.add_class("x", 5, "e")
        self.add_class("x", 6, "f")

        # Train or validation dataset
        assert subset in ["train", "val"]
        dataset_dir = os.path.join(dataset_dir, subset)

        # Load annotations
        # We mostly care about the x and y coordinates of each region
        # Note: In VIA 2.0, regions was changed from a dict to a list.
        annotations = json.load(open(os.path.join(dataset_dir, "via_region_data.json")))
        annotations = list(annotations.values())  # don't need the dict keys

        # The VIA tool saves images in the JSON even if they don't have any
        # annotations. Skip unannotated images.
        annotations = [a for a in annotations if a['regions']]
        # print("ANNOTATIONS INFO:", annotations)
        idlist = []
        # Add images
        for a in annotations:
            # Get the x, y coordinaets of points of the polygons that make up
            # the outline of each object instance. These are stores in the
            # shape_attributes (see json format above)
            # The if condition is needed to support VIA versions 1.x and 2.x.
            # print(type(a['regions'])) #list
            '''if type(a['regions']) is dict:
                polygons = [r['shape_attributes'] for r in a['regions'].values()]
            else:
                polygons = [r['shape_attributes'] for r in a['regions']]'''
            #print(a['regions'])
            polygons = [r['shape_attributes'] for r in a['regions']]
            #name = [r['region_attributes']['type'] for r in a['regions']]
            #print("[NAME INFO:]", name)
            #print(type(name)) #list
            '''
            [NAME INFO:] [{'porosity': True}, {'porosity': True}, {'surface hollow': True}, 
            {'porosity': True}, {'lacking of sintering': True}, {'porosity': True}, 
            {'porosity': True}, {'porosity'