Mask R-CNN:coco-inspect_model.ipynb解析和运行

最新推荐文章于 2021-07-24 18:19:36 发布
最新推荐文章于 2021-07-24 18:19:36 发布
阅读量705 收藏 3
点赞数 1
分类专栏： # 网络训练 文章标签： inspect_model
没有太晚的开始，不如就从今天行动。总有一天，那个一点一点可见的未来，会在你心里，也在你的脚下慢慢清透。生活，从不亏待每一个努力向上的人。
本文链接：https://blog.csdn.net/qq_41498261/article/details/109061417
版权

文章目录

前言

一、运行方式1——jupyter notebook

在coco文件夹下运行jupyter notebook：

从jupyter notebook中选中inspect_model.ipynb文件并点击：

1. Cell 1

1.1 代码

import os
import sys
import random
import math
import re
import time
import numpy as np
import tensorflow as tf
import matplotlib
import matplotlib.pyplot as plt
import matplotlib.patches as patches

# Root directory of the project
ROOT_DIR = os.path.abspath("../../")

# Import Mask RCNN
sys.path.append(ROOT_DIR)  # To find local version of the library
from mrcnn import utils
from mrcnn import visualize
from mrcnn.visualize import display_images
import mrcnn.model as modellib
from mrcnn.model import log

%matplotlib inline 

# Directory to save logs and trained model
MODEL_DIR = os.path.join(ROOT_DIR, "logs")

# Local path to trained weights file
COCO_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(COCO_MODEL_PATH):
    utils.download_trained_weights(COCO_MODEL_PATH)

# Path to Shapes trained weights
SHAPES_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_shapes.h5")

1.2 输出

1.3 解析

2. Cell 2

2.1 代码

# Run one of the code blocks

# Shapes toy dataset
# import shapes
# config = shapes.ShapesConfig()

# MS COCO Dataset
import coco
config = coco.CocoConfig()
COCO_DIR = "path to COCO dataset"  # TODO: enter value here

2.2 改动代码

改动代码1：COCO_DIR
COCO_DIR = "../../datasets/coco"

2.3 输出
无输出。
2.4 解析

3. Cell 3

3.1 代码

# Override the training configurations with a few
# changes for inferencing.
class InferenceConfig(config.__class__):
    # Run detection on one image at a time
    GPU_COUNT = 1
    IMAGES_PER_GPU = 1

config = InferenceConfig()
config.display()

3.2 输出

Configurations:
BACKBONE                       resnet101
BACKBONE_STRIDES               [4, 8, 16, 32, 64]
BATCH_SIZE                     1
BBOX_STD_DEV                   [0.1 0.1 0.2 0.2]
COMPUTE_BACKBONE_SHAPE         None
DETECTION_MAX_INSTANCES        100
DETECTION_MIN_CONFIDENCE       0.7
DETECTION_NMS_THRESHOLD        0.3
FPN_CLASSIF_FC_LAYERS_SIZE     1024
GPU_COUNT                      1
GRADIENT_CLIP_NORM             5.0
IMAGES_PER_GPU                 1
IMAGE_CHANNEL_COUNT            3
IMAGE_MAX_DIM                  1024
IMAGE_META_SIZE                93
IMAGE_MIN_DIM                  800
IMAGE_MIN_SCALE                0
IMAGE_RESIZE_MODE              square
IMAGE_SHAPE                    [1024 1024    3]
LEARNING_MOMENTUM              0.9
LEARNING_RATE                  0.01
LOSS_WEIGHTS                   {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0}
MASK_POOL_SIZE                 14
MASK_SHAPE                     [28, 28]
MAX_GT_INSTANCES               100
MEAN_PIXEL                     [123.7 116.8 103.9]
MINI_MASK_SHAPE                (56, 56)
NAME                           coco
NUM_CLASSES                    81
POOL_SIZE                      7
POST_NMS_ROIS_INFERENCE        1000
POST_NMS_ROIS_TRAINING         2000
PRE_NMS_LIMIT                  6000
ROI_POSITIVE_RATIO             0.33
RPN_ANCHOR_RATIOS              [0.5, 1, 2]
RPN_ANCHOR_SCALES              (32, 64, 128, 256, 512)
RPN_ANCHOR_STRIDE              1
RPN_BBOX_STD_DEV               [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD              0.7
RPN_TRAIN_ANCHORS_PER_IMAGE    256
STEPS_PER_EPOCH                1000
TOP_DOWN_PYRAMID_SIZE          256
TRAIN_BN                       False
TRAIN_ROIS_PER_IMAGE           200
USE_MINI_MASK                  True
USE_RPN_ROIS                   True
VALIDATION_STEPS               50
WEIGHT_DECAY                   0.0001

3.3 解析

4. Cell 4

4.1 代码

# Device to load the neural network on.
# Useful if you're training a model on the same 
# machine, in which case use CPU and leave the
# GPU for training.
DEVICE = "/cpu:0"  # /cpu:0 or /gpu:0

# Inspect the model in training or inference modes
# values: 'inference' or 'training'
# TODO: code for 'training' test mode not ready yet
TEST_MODE = "inference"

4.2 输出
无输出。
4.3 解析

5. Cell

5.1 代码

def get_ax(rows=1, cols=1, size=16):
    """Return a Matplotlib Axes array to be used in
    all visualizations in the notebook. Provide a
    central point to control graph sizes.
    
    Adjust the size attribute to control how big to render images
    """
    _, ax = plt.subplots(rows, cols, figsize=(size*cols, size*rows))
    return ax

5.2 输出
无输出。
5.3 解析

6. Cell

6.1 代码

# Build validation dataset
if config.NAME == 'shapes':
    dataset = shapes.ShapesDataset()
    dataset.load_shapes(500, config.IMAGE_SHAPE[0], config.IMAGE_SHAPE[1])
elif config.NAME == "coco":
    dataset = coco.CocoDataset()
    dataset.load_coco(COCO_DIR, "minival")

# Must call before using the dataset
dataset.prepare()

print("Images: {}\nClasses: {}".format(len(dataset.image_ids), dataset.class_names))

出错：

6.2 改动代码
运行cell6的时候报错，发现是找不到路径，仔细观察报错的信息，发现是因为数据集的问题。在测试inspect_data.ipynb的时候，我们将默认的2014改成了2017版本，而根据自动下载数据集的方法提示：
即 2017版本的数据集只包含"train" and “val”，不存在minival和valminusminival,所以代码中加载的dataset.load_coco(COCO_DIR, "minival")，就相当于加载不存在的minival数据集，自然会报错。

更改代码2：dataset.load_coco(COCO_DIR, “val”)

将加载的验证数据集更改成基于2017版主的val。

 dataset.load_coco(COCO_DIR, "val")

即：

# Build validation dataset
if config.NAME == 'shapes':
    dataset = shapes.ShapesDataset()
    dataset.load_shapes(500, config.IMAGE_SHAPE[0], config.IMAGE_SHAPE[1])
elif config.NAME == "coco":
    dataset = coco.CocoDataset()
    #dataset.load_coco(COCO_DIR, "minival")    #waitFoF
    dataset.load_coco(COCO_DIR, "val")     #waitFoF
# Must call before using the dataset
dataset.prepare()

print("Images: {}\nClasses: {}".format(len(dataset.image_ids), dataset.class_names))

6.3 输出
重新加载后，再次运行此cell：

6.3 解析

7. Cell 7

7.1 代码

# Create model in inference mode
with tf.device(DEVICE):
    model = modellib.MaskRCNN(mode="inference", model_dir=MODEL_DIR,
                              config=config)

# Set weights file path
if config.NAME == "shapes":
    weights_path = SHAPES_MODEL_PATH
elif config.NAME == "coco":
    weights_path = COCO_MODEL_PATH
# Or, uncomment to load the last model you trained
# weights_path = model.find_last()

# Load weights
print("Loading weights ", weights_path)
model.load_weights(weights_path, by_name=True)

7.2 输出

7.3 解析

8. Cell 8

8.1 代码

image_id = random.choice(dataset.image_ids)
image, image_meta, gt_class_id, gt_bbox, gt_mask =\
    modellib.load_image_gt(dataset, config, image_id, use_mini_mask=False)
info = dataset.image_info[image_id]
print("image ID: {}.{} ({}) {}".format(info["source"], info["id"], image_id, 
                                       dataset.image_reference(image_id)))
# Run object detection
results = model.detect([image], verbose=1)

# Display results
ax = get_ax(1)
r = results[0]
visualize.display_instances(image, r['rois'], r['masks'], r['class_ids'], 
                            dataset.class_names, r['scores'], ax=ax,
                            title="Predictions")
log("gt_class_id", gt_class_id)
log("gt_bbox", gt_bbox)
log("gt_mask", gt_mask)

8.2 更改代码
这个cell中明显有一个输出图像的语句visualize.display_instances，但是未出现图像，猜测又是matplotlib的终端是不显示图像界面的类型，即Agg。

更改代码3：matplotlib.use(‘TkAgg’)

只要代码中出现过import coco，就要更改终端界面，
即在import matplotlib.pyplot as plt前面加上一句matplotlib.use('TkAgg')。

若在jupyter notebook上面运行，还要把%matplotlib inline移动到import matplotlib.pyplot as plt后面。

cell1:

import os
import sys
import random
import math
import re
import time
import numpy as np
import tensorflow as tf
import matplotlib
import matplotlib.patches as patches

# Root directory of the project
ROOT_DIR = os.path.abspath("../../")

# Import Mask RCNN
sys.path.append(ROOT_DIR)  # To find local version of the library
from mrcnn import utils
from mrcnn import visualize
from mrcnn.visualize import display_images
import mrcnn.model as modellib
from mrcnn.model import log



# Directory to save logs and trained model
MODEL_DIR = os.path.join(ROOT_DIR, "logs")

# Local path to trained weights file
COCO_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(COCO_MODEL_PATH):
    utils.download_trained_weights(COCO_MODEL_PATH)

# Path to Shapes trained weights
SHAPES_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_shapes.h5")

cell2:

# Run one of the code blocks

# Shapes toy dataset
# import shapes
# config = shapes.ShapesConfig()

# MS COCO Dataset
import coco
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt
%matplotlib inline 


config = coco.CocoConfig()
COCO_DIR = "../../datasets/coco"  # TODO: enter value here  waitFoF

8.3 输出
刷新后再次运行后，图像出现。

image ID: coco.117197 (1557) http://cocodataset.org/#explore?id=117197
Processing 1 images
image                    shape: (1024, 1024, 3)       min:    0.00000  max:  255.00000  uint8
molded_images            shape: (1, 1024, 1024, 3)    min: -123.70000  max:  149.10000  float64
image_metas              shape: (1, 93)               min:    0.00000  max: 1024.00000  int32
anchors                  shape: (1, 261888, 4)        min:   -0.35390  max:    1.29134  float32
gt_class_id              shape: (2,)                  min:    1.00000  max:   68.00000  int32
gt_bbox                  shape: (2, 4)                min:  320.00000  max:  861.00000  int32
gt_mask                  shape: (1024, 1024, 2)       min:    0.00000  max:    1.00000  bool

8.4 解析

9. Cell 9

9.1. 代码

# Draw precision-recall curve
AP, precisions, recalls, overlaps = utils.compute_ap(gt_bbox, gt_class_id, gt_mask,
                                          r['rois'], r['class_ids'], r['scores'], r['masks'])
visualize.plot_precision_recall(AP, precisions, recalls)

9.2. 输出
在这里插入图片描述

9.3. 解析

10. Cell 10

10.1. 代码

# Grid of ground truth objects and their predictions
visualize.plot_overlaps(gt_class_id, r['class_ids'], r['scores'],
                        overlaps, dataset.class_names)

10.2. 输出

10.3. 解析

11. Cell 11

11.1. 代码

# Compute VOC-style Average Precision
def compute_batch_ap(image_ids):
    APs = []
    for image_id in image_ids:
        # Load image
        image, image_meta, gt_class_id, gt_bbox, gt_mask =\
            modellib.load_image_gt(dataset, config,
                                   image_id, use_mini_mask=False)
        # Run object detection
        results = model.detect([image], verbose=0)
        # Compute AP
        r = results[0]
        AP, precisions, recalls, overlaps =\
            utils.compute_ap(gt_bbox, gt_class_id, gt_mask,
                              r['rois'], r['class_ids'], r['scores'], r['masks'])
        APs.append(AP)
    return APs

# Pick a set of random images
image_ids = np.random.choice(dataset.image_ids, 10)
APs = compute_batch_ap(image_ids)
print("mAP @ IoU=50: ", np.mean(APs))

11.2. 输出

11.3. 解析

12. Cell 12

12.1. 代码

# Generate RPN trainig targets
# target_rpn_match is 1 for positive anchors, -1 for negative anchors
# and 0 for neutral anchors.
target_rpn_match, target_rpn_bbox = modellib.build_rpn_targets(
    image.shape, model.anchors, gt_class_id, gt_bbox, model.config)
log("target_rpn_match", target_rpn_match)
log("target_rpn_bbox", target_rpn_bbox)

positive_anchor_ix = np.where(target_rpn_match[:] == 1)[0]
negative_anchor_ix = np.where(target_rpn_match[:] == -1)[0]
neutral_anchor_ix = np.where(target_rpn_match[:] == 0)[0]
positive_anchors = model.anchors[positive_anchor_ix]
negative_anchors = model.anchors[negative_anchor_ix]
neutral_anchors = model.anchors[neutral_anchor_ix]
log("positive_anchors", positive_anchors)
log("negative_anchors", negative_anchors)
log("neutral anchors", neutral_anchors)

# Apply refinement deltas to positive anchors
refined_anchors = utils.apply_box_deltas(
    positive_anchors,
    target_rpn_bbox[:positive_anchors.shape[0]] * model.config.RPN_BBOX_STD_DEV)
log("refined_anchors", refined_anchors, )

12.2. 输出

target_rpn_match         shape: (261888,)             min:   -1.00000  max:    1.00000  int32
target_rpn_bbox          shape: (256, 4)              min:   -2.87262  max:    2.43068  float64
positive_anchors         shape: (20, 4)               min:   44.68629  max: 1088.00000  float64
negative_anchors         shape: (236, 4)              min:  -42.50967  max: 1048.00000  float64
neutral anchors          shape: (261632, 4)           min: -362.03867  max: 1322.03867  float64
refined_anchors          shape: (20, 4)               min:   43.00000  max: 1024.00000  float32

12.3. 解析

13. Cell 13

13.1. 代码

# Display positive anchors before refinement (dotted) and
# after refinement (solid).
visualize.draw_boxes(image, boxes=positive_anchors, refined_boxes=refined_anchors, ax=get_ax())

13.2. 输出
在这里插入图片描述

13.3. 解析

14. Cell 14

14.1. 代码

# Run RPN sub-graph
pillar = model.keras_model.get_layer("ROI").output  # node to start searching from

# TF 1.4 and 1.9 introduce new versions of NMS. Search for all names to support TF 1.3~1.10
nms_node = model.ancestor(pillar, "ROI/rpn_non_max_suppression:0")
if nms_node is None:
    nms_node = model.ancestor(pillar, "ROI/rpn_non_max_suppression/NonMaxSuppressionV2:0")
if nms_node is None: #TF 1.9-1.10
    nms_node = model.ancestor(pillar, "ROI/rpn_non_max_suppression/NonMaxSuppressionV3:0")

rpn = model.run_graph([image], [
    ("rpn_class", model.keras_model.get_layer("rpn_class").output),
    ("pre_nms_anchors", model.ancestor(pillar, "ROI/pre_nms_anchors:0")),
    ("refined_anchors", model.ancestor(pillar, "ROI/refined_anchors:0")),
    ("refined_anchors_clipped", model.ancestor(pillar, "ROI/refined_anchors_clipped:0")),
    ("post_nms_anchor_ix", nms_node),
    ("proposals", model.keras_model.get_layer("ROI").output),
])

14.2. 输出

rpn_class                shape: (1, 261888, 2)        min:    0.00000  max:    1.00000  float32
pre_nms_anchors          shape: (1, 6000, 4)          min:   -0.35390  max:    1.29134  float32
refined_anchors          shape: (1, 6000, 4)          min:   -0.51830  max:    1.51648  float32
refined_anchors_clipped  shape: (1, 6000, 4)          min:    0.00000  max:    1.00000  float32
post_nms_anchor_ix       shape: (1000,)               min:    0.00000  max: 3361.00000  int32
proposals                shape: (1, 1000, 4)          min:    0.00000  max:    1.00000  float32

14.3. 解析

15. Cell 15

15.1. 代码

# Show top anchors by score (before refinement)
limit = 100
sorted_anchor_ids = np.argsort(rpn['rpn_class'][:,:,1].flatten())[::-1]
visualize.draw_boxes(image, boxes=model.anchors[sorted_anchor_ids[:limit]], ax=get_ax())

15.2. 输出

15.3. 解析

16.Cell 16

16.1. 代码

# Show top anchors with refinement. Then with clipping to image boundaries
limit = 50
ax = get_ax(1, 2)
pre_nms_anchors = utils.denorm_boxes(rpn["pre_nms_anchors"][0], image.shape[:2])
refined_anchors = utils.denorm_boxes(rpn["refined_anchors"][0], image.shape[:2])
refined_anchors_clipped = utils.denorm_boxes(rpn["refined_anchors_clipped"][0], image.shape[:2])
visualize.draw_boxes(image, boxes=pre_nms_anchors[:limit],
                     refined_boxes=refined_anchors[:limit], ax=ax[0])
visualize.draw_boxes(image, refined_boxes=refined_anchors_clipped[:limit], ax=ax[1])

16.2. 输出
在这里插入图片描述

16.3. 解析

17. Cell 17

17.1. 代码

# Show refined anchors after non-max suppression
limit = 50
ixs = rpn["post_nms_anchor_ix"][:limit]
visualize.draw_boxes(image, refined_boxes=refined_anchors_clipped[ixs], ax=get_ax())

17.2. 输出

17.3. 解析

18. Cell 18

18.1. 代码

# Show final proposals
# These are the same as the previous step (refined anchors 
# after NMS) but with coordinates normalized to [0, 1] range.
limit = 50
# Convert back to image coordinates for display
h, w = config.IMAGE_SHAPE[:2]
proposals = rpn['proposals'][0, :limit] * np.array([h, w, h, w])
visualize.draw_boxes(image, refined_boxes=proposals, ax=get_ax())

18.2. 输出
在这里插入图片描述

18.3. 解析

19. Cell 19

19.1. 代码

# Measure the RPN recall (percent of objects covered by anchors)
# Here we measure recall for 3 different methods:
# - All anchors
# - All refined anchors
# - Refined anchors after NMS
iou_threshold = 0.7

recall, positive_anchor_ids = utils.compute_recall(model.anchors, gt_bbox, iou_threshold)
print("All Anchors ({:5})       Recall: {:.3f}  Positive anchors: {}".format(
    model.anchors.shape[0], recall, len(positive_anchor_ids)))

recall, positive_anchor_ids = utils.compute_recall(rpn['refined_anchors'][0], gt_bbox, iou_threshold)
print("Refined Anchors ({:5})   Recall: {:.3f}  Positive anchors: {}".format(
    rpn['refined_anchors'].shape[1], recall, len(positive_anchor_ids)))

recall, positive_anchor_ids = utils.compute_recall(proposals, gt_bbox, iou_threshold)
print("Post NMS Anchors ({:5})  Recall: {:.3f}  Positive anchors: {}".format(
    proposals.shape[0], recall, len(positive_anchor_ids)))

19.2. 输出

All Anchors (261888)       Recall: 0.429  Positive anchors: 5
Refined Anchors ( 6000)   Recall: 0.000  Positive anchors: 0
Post NMS Anchors (   50)  Recall: 1.000  Positive anchors: 11

19.3. 解析

20. Cell 20

20.1. 代码

# Get input and output to classifier and mask heads.
mrcnn = model.run_graph([image], [
    ("proposals", model.keras_model.get_layer("ROI").output),
    ("probs", model.keras_model.get_layer("mrcnn_class").output),
    ("deltas", model.keras_model.get_layer("mrcnn_bbox").output),
    ("masks", model.keras_model.get_layer("mrcnn_mask").output),
    ("detections", model.keras_model.get_layer("mrcnn_detection").output),
])

20.2. 输出

proposals                shape: (1, 1000, 4)          min:    0.00000  max:    1.00000  float32
probs                    shape: (1, 1000, 81)         min:    0.00000  max:    0.99996  float32
deltas                   shape: (1, 1000, 81, 4)      min:   -3.57599  max:    4.06808  float32
masks                    shape: (1, 100, 28, 28, 81)  min:    0.00000  max:    0.99997  float32
detections               shape: (1, 100, 6)           min:    0.00000  max:    6.00000  float32

20.3. 解析

21. Cell 21

21.1. 代码

# Get detection class IDs. Trim zero padding.
det_class_ids = mrcnn['detections'][0, :, 4].astype(np.int32)
det_count = np.where(det_class_ids == 0)[0][0]
det_class_ids = det_class_ids[:det_count]
detections = mrcnn['detections'][0, :det_count]

print("{} detections: {}".format(
    det_count, np.array(dataset.class_names)[det_class_ids]))

captions = ["{} {:.3f}".format(dataset.class_names[int(c)], s) if c > 0 else ""
            for c, s in zip(detections[:, 4], detections[:, 5])]
visualize.draw_boxes(
    image, 
    refined_boxes=utils.denorm_boxes(detections[:, :4], image.shape[:2]),
    visibilities=[2] * len(detections),
    captions=captions, title="Detections",
    ax=get_ax())

21.2. 输出

21.3. 解析

22. Cell 22

22.1. 代码

# Proposals are in normalized coordinates. Scale them
# to image coordinates.
h, w = config.IMAGE_SHAPE[:2]
proposals = np.around(mrcnn["proposals"][0] * np.array([h, w, h, w])).astype(np.int32)

# Class ID, score, and mask per proposal
roi_class_ids = np.argmax(mrcnn["probs"][0], axis=1)
roi_scores = mrcnn["probs"][0, np.arange(roi_class_ids.shape[0]), roi_class_ids]
roi_class_names = np.array(dataset.class_names)[roi_class_ids]
roi_positive_ixs = np.where(roi_class_ids > 0)[0]

# How many ROIs vs empty rows?
print("{} Valid proposals out of {}".format(np.sum(np.any(proposals, axis=1)), proposals.shape[0]))
print("{} Positive ROIs".format(len(roi_positive_ixs)))

# Class counts
print(list(zip(*np.unique(roi_class_names, return_counts=True))))

22.2. 输出

1000 Valid proposals out of 1000
81 Positive ROIs
[('BG', 919), ('backpack', 10), ('bus', 8), ('car', 7), ('handbag', 2), ('person', 54)]

22.3. 解析

23. Cell 23

23.1. 代码


# Display a random sample of proposals.
# Proposals classified as background are dotted, and
# the rest show their class and confidence score.
limit = 200
ixs = np.random.randint(0, proposals.shape[0], limit)
captions = ["{} {:.3f}".format(dataset.class_names[c], s) if c > 0 else ""
            for c, s in zip(roi_class_ids[ixs], roi_scores[ixs])]
visualize.draw_boxes(image, boxes=proposals[ixs],
                     visibilities=np.where(roi_class_ids[ixs] > 0, 2, 1),
                     captions=captions, title="ROIs Before Refinement",
                     ax=get_ax())

23.2. 输出
在这里插入图片描述

23.3. 解析

24. Cell 24

24.1. 代码

# Class-specific bounding box shifts.
roi_bbox_specific = mrcnn["deltas"][0, np.arange(proposals.shape[0]), roi_class_ids]
log("roi_bbox_specific", roi_bbox_specific)

# Apply bounding box transformations
# Shape: [N, (y1, x1, y2, x2)]
refined_proposals = utils.apply_box_deltas(
    proposals, roi_bbox_specific * config.BBOX_STD_DEV).astype(np.int32)
log("refined_proposals", refined_proposals)

# Show positive proposals
# ids = np.arange(roi_boxes.shape[0])  # Display all
limit = 5
ids = np.random.randint(0, len(roi_positive_ixs), limit)  # Display random sample
captions = ["{} {:.3f}".format(dataset.class_names[c], s) if c > 0 else ""
            for c, s in zip(roi_class_ids[roi_positive_ixs][ids], roi_scores[roi_positive_ixs][ids])]
visualize.draw_boxes(image, boxes=proposals[roi_positive_ixs][ids],
                     refined_boxes=refined_proposals[roi_positive_ixs][ids],
                     visibilities=np.where(roi_class_ids[roi_positive_ixs][ids] > 0, 1, 0),
                     captions=captions, title="ROIs After Refinement",
                     ax=get_ax())

24.2. 输出

roi_bbox_specific        shape: (1000, 4)             min:   -3.23401  max:    3.45557  float32
refined_proposals        shape: (1000, 4)             min:    0.00000  max: 1083.00000  int32

24.3. 解析

25. Cell 25

25.1. 代码

# Remove boxes classified as background
keep = np.where(roi_class_ids > 0)[0]
print("Keep {} detections:\n{}".format(keep.shape[0], keep))

25.2. 输出

Keep 81 detections:
[  0   1   2   3   4   5   7   8   9  11  13  14  15  16  17  18  19  23
  25  27  28  30  32  38  41  42  50  59  68  69  72  75  78  79  81  82
  89  90 102 105 109 115 131 132 144 152 153 158 173 176 192 204 296 326
 334 377 402 413 435 437 454 487 542 546 577 594 674 697 699 701 732 761
 776 782 857 876 906 936 944 956 977]

25.3. 解析

26. Cell 26

26.1. 代码

# Remove low confidence detections
keep = np.intersect1d(keep, np.where(roi_scores >= config.DETECTION_MIN_CONFIDENCE)[0])
print("Remove boxes below {} confidence. Keep {}:\n{}".format(
    config.DETECTION_MIN_CONFIDENCE, keep.shape[0], keep))

26.2. 输出

Remove boxes below 0.7 confidence. Keep 55:
[  0   1   3   4   5   8   9  11  13  14  15  23  27  28  32  38  41  50
  59  68  72  75  78  79  82  90 102 105 109 131 132 144 152 173 176 204
 296 326 334 402 413 435 437 487 542 546 577 594 697 732 782 876 906 956
 977]

26.3. 解析

27. Cell 27

27.1. 代码

# Apply per-class non-max suppression
pre_nms_boxes = refined_proposals[keep]
pre_nms_scores = roi_scores[keep]
pre_nms_class_ids = roi_class_ids[keep]

nms_keep = []
for class_id in np.unique(pre_nms_class_ids):
    # Pick detections of this class
    ixs = np.where(pre_nms_class_ids == class_id)[0]
    # Apply NMS
    class_keep = utils.non_max_suppression(pre_nms_boxes[ixs], 
                                            pre_nms_scores[ixs],
                                            config.DETECTION_NMS_THRESHOLD)
    # Map indicies
    class_keep = keep[ixs[class_keep]]
    nms_keep = np.union1d(nms_keep, class_keep)
    print("{:22}: {} -> {}".format(dataset.class_names[class_id][:20], 
                                   keep[ixs], class_keep))

keep = np.intersect1d(keep, nms_keep).astype(np.int32)
print("\nKept after per-class NMS: {}\n{}".format(keep.shape[0], keep))

27.2. 输出

person                : [  1   3   5   8   9  11  15  23  27  32  41  68  72  75  78  79  82 102
 105 109 144 152 173 176 204 296 326 334 402 413 435 437 487 542 546 577
 594 697 732 782 876 906 956] -> [3 1 9]
car                   : [  4  14 131 132] -> [14]
bus                   : [  0  13  28  38  50  59  90 977] -> [0]

Kept after per-class NMS: 5
[ 0  1  3  9 14]

27.3. 解析

28. Cell 28

28.1. 代码

# Show final detections
ixs = np.arange(len(keep))  # Display all
# ixs = np.random.randint(0, len(keep), 10)  # Display random sample
captions = ["{} {:.3f}".format(dataset.class_names[c], s) if c > 0 else ""
            for c, s in zip(roi_class_ids[keep][ixs], roi_scores[keep][ixs])]
visualize.draw_boxes(
    image, boxes=proposals[keep][ixs],
    refined_boxes=refined_proposals[keep][ixs],
    visibilities=np.where(roi_class_ids[keep][ixs] > 0, 1, 0),
    captions=captions, title="Detections after NMS",
    ax=get_ax())

28.2. 输出

28.3. 解析

29. Cell 29

29.1. 代码

display_images(np.transpose(gt_mask, [2, 0, 1]), cmap="Blues")

29.2. 输出

29.3. 解析

30. Cell 30

30.1. 代码

# Get predictions of mask head
mrcnn = model.run_graph([image], [
    ("detections", model.keras_model.get_layer("mrcnn_detection").output),
    ("masks", model.keras_model.get_layer("mrcnn_mask").output),
])

# Get detection class IDs. Trim zero padding.
det_class_ids = mrcnn['detections'][0, :, 4].astype(np.int32)
det_count = np.where(det_class_ids == 0)[0][0]
det_class_ids = det_class_ids[:det_count]

print("{} detections: {}".format(
    det_count, np.array(dataset.class_names)[det_class_ids]))

30.2. 输出

detections               shape: (1, 100, 6)           min:    0.00000  max:    6.00000  float32
masks                    shape: (1, 100, 28, 28, 81)  min:    0.00000  max:    0.99997  float32
5 detections: ['bus' 'person' 'person' 'person' 'car']

30.3. 解析

31. Cell 31

31.1. 代码

# Masks
det_boxes = utils.denorm_boxes(mrcnn["detections"][0, :, :4], image.shape[:2])
det_mask_specific = np.array([mrcnn["masks"][0, i, :, :, c] 
                              for i, c in enumerate(det_class_ids)])
det_masks = np.array([utils.unmold_mask(m, det_boxes[i], image.shape)
                      for i, m in enumerate(det_mask_specific)])
log("det_mask_specific", det_mask_specific)
log("det_masks", det_masks)

31.2. 输出

det_mask_specific        shape: (5, 28, 28)           min:    0.00007  max:    0.99992  float32
det_masks                shape: (5, 1024, 1024)       min:    0.00000  max:    1.00000  bool

31.3. 解析

32. Cell 32

32.1. 代码

display_images(det_mask_specific[:4] * 255, cmap="Blues", interpolation="none")

32.2. 输出

32.3. 解析

33. Cell 33

33.1. 代码

display_images(det_masks[:4] * 255, cmap="Blues", interpolation="none")

33.2. 输出

33.3. 解析

34. Cell 34

34.1. 代码

# Get activations of a few sample layers
activations = model.run_graph([image], [
    ("input_image",        tf.identity(model.keras_model.get_layer("input_image").output)),
    ("res4w_out",          model.keras_model.get_layer("res4w_out").output),  # for resnet100
    ("rpn_bbox",           model.keras_model.get_layer("rpn_bbox").output),
    ("roi",                model.keras_model.get_layer("ROI").output),
])

34.2. 输出

input_image              shape: (1, 1024, 1024, 3)    min: -123.70000  max:  151.10001  float32
res4w_out                shape: (1, 64, 64, 1024)     min:    0.00000  max:   67.11374  float32
rpn_bbox                 shape: (1, 261888, 4)        min:  -12.46734  max:   64.23416  float32
roi                      shape: (1, 1000, 4)          min:    0.00000  max:    1.00000  float32

34.3. 解析

35. Cell 35

35.1. 代码

# Input image (normalized)
_ = plt.imshow(modellib.unmold_image(activations["input_image"][0],config))

35.2. 输出

35.3. 解析

36. Cell 36

36.1. 代码

# Backbone feature map
display_images(np.transpose(activations["res4w_out"][0,:,:,:4], [2, 0, 1]))

36.2. 输出
在这里插入图片描述

36.3. 解析

37. Cell 37

37.1. 代码

# Histograms of RPN bounding box deltas
plt.figure(figsize=(12, 3))
plt.subplot(1, 4, 1)
plt.title("dy")
_ = plt.hist(activations["rpn_bbox"][0,:,0], 50)
plt.subplot(1, 4, 2)
plt.title("dx")
_ = plt.hist(activations["rpn_bbox"][0,:,1], 50)
plt.subplot(1, 4, 3)
plt.title("dw")
_ = plt.hist(activations["rpn_bbox"][0,:,2], 50)
plt.subplot(1, 4, 4)
plt.title("dh")
_ = plt.hist(activations["rpn_bbox"][0,:,3], 50)

37.2. 输出

37.3. 解析

38. Cell 38

38.1. 代码

# Distribution of y, x coordinates of generated proposals
plt.figure(figsize=(10, 5)
plt.subplot(1, 2, 1)
plt.title("y1, x1")
plt.scatter(activations["roi"][0,:,0], activations["roi"][0,:,1])
plt.subplot(1, 2, 2)
plt.title("y2, x2")
plt.scatter(activations["roi"][0,:,2], activations["roi"][0,:,3])
plt.show()

38.2. 输出

38.3. 解析

二、运行方式2——Pycharm

1. 下载inspect_model.py

2. 放入Mask_RCNN-master\samples\coco\文件夹

3. 运行inspect_model.py

改动代码4：

屏蔽掉get_ipython().magic('matplotlib inline')

#get_ipython().magic('matplotlib inline')

成功运行：

💛💛注意：jupyter notebook中更改的所有地方都要在inspect_model.py中进行更改。

WaitFoF
关注
专栏目录
maskrcnn
weixin_41571247的博客
01-07 159
import os import sys import random import math import re import time import numpy as np import cv2 import matplotlib import matplotlib.pyplot as plt from config import Config import utils import model...
matterport_MaskRCNN（4）——测试模型
chicken3wings的博客
12-18 760
注意事项： github的该项目在运行setup.py之后，会生成一个.egg文件，在我的mask虚拟环境中路径是D:\Anaconda\envs\mask\Lib\site-packages\mask_rcnn-2.1-py3.7.egg，当我们要修改一些文件的源码时，需要从该文件修改，并重启编译器才能生效。.egg文件其实是一个压缩包形式，所以我这里右键，使用好压打开，然后使用内部查看器可以需...
matterport_MaskRCNN（3）——训练模型
chicken3wings的博客
12-17 645
训练代码如下，模型文件会保存在该文件的上级目录的logs文件夹中，每个epoch结束保存一次 # -*- coding: utf-8 -*- """ Created on Wed Nov 6 19:06:46 2019 @author: Lenovo """ # -*- coding: utf-8 -*- """ Created on Tue Sep 17 15:55:49 2019 @a...
Mask R-CNN的代码实现
myGFZ的博客
05-06 7977
导言 这篇博客主要记录了我实现Mask R-CNN中demo的过程，及遇到问题的解决办法。源码下载地址稍后给出。 实现Mask R-CNN的demo一共需要下载以下两个文件： 1、Mask R-CNN源码 2、训练好的权重mask_rcnn_coco.h5 上面两个文件可以从网盘下载： 链接: https://pan.baidu.com/s/1qC5Rguvoyrppp4lZsa1...
Mask R-CNN用于目标检测和分割代码实现
吴建明wujianming_110117
07-14 1017
Mask R-CNN用于目标检测和分割代码实现 Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow 代码链接：https://github.com/matterport/Mask_RCNN 这是基于Python 3，Keras和TensorFlow 的Mask R-CNN的实现。该模型为图像中对象的每个实例生成边界框和分割masks。基于功能金字塔网络Feature Pyramid Network
[代码解析]Mask R-CNN介绍与实现(转)
急流勇进
11-20 1068
[代码解析]Mask R-CNN介绍与实现(转) 文章来源 DFann 版权声明：如果你觉得写的还可以，可以考虑打赏一下。转载请联系。 https://blog.csdn.net/u011974639/arti...
mask rcnn 召回率_[代码解析]Mask R-CNN介绍与实现(转)
weixin_42510012的博客
01-16 1022
简介代码源于matterport的工作组，可以在github上fork它们组的工作。软件必备复现的Mask R-CNN是基于Python3，Keras，TensorFlow。Python 3.4+TensorFlow 1.3+Keras 2.0.8+Jupyter NotebookNumpy, skimage, scipy建议配置一个高版本的Anaconda3+TensorFlow-GPU版本。M...
气球的mask-rcnn转coco的数据集，可直接使用最新的mmdetection框架去训练，测试效果好
最新发布
04-26
这个数据集是将“气球”相关的Mask R-CNN数据转换为COCO格式，以便能直接用于训练MMDetection框架。下面将详细解释相关知识点。 1. **Mask R-CNN**: Mask R-CNN是Faster R-CNN的扩展，由Facebook AI Research（FAIR...
COCO数据集提取自己想要的类
qq_33536992的博客
08-15 7777
文章目录前言一、COCO数据集二、安装COCO-PythonAPI(pycocotools)三、类提取+Json转xml四、参考 前言 COCO数据集是一个很大的数据集，包括了语以分割、实例分割、目标检测等，因此它的标签就对应了几种不同的标注方式。最新的COCO2017对应了总共90个类别，但有时侯我们在做任务不需要这么多的类，只需要其中的一类或几类。本篇以提取“person”这一类的目标检测数据集为例，并将标签转化为我们通常使用的xml格式。 一、COCO数据集 COCO数据集官方下载，选择如下图所示进行
Win10下Mask RCNN环境配置+踩坑说明，成功运行demo.py
qq_41721951的博客
07-24 3133
Win10环境下配置Mask RCNN,通过修改model.py文件调通源码，成功运行demo.ipynb Win10下Mask RCNN环境配置，笔者配置了整整一周，期间从tensorflow1.3版本换到2.5版本都没有成功，最后通过一定的代码修改终于跑通demo.py，于是泪水中带着喜悦写下这篇博客，希望大家少走点弯路，该配置流程在tensorflow1.3-2.5版本下均可，具体过程如下 1、版本信息 自己调试了tensorflow1.0和2.0版本，均已实现，版本影响不大，一些博客说，tensor
TensorFlow实战：Chapter-8上(Mask R-CNN介绍与实现)
热门推荐
DFan的NoteBook
11-09 6万+
简介 论文地址：Mask R-CNN 源代码：matterport - github 代码源于matterport的工作组，可以在github上fork它们组的工作。 软件必备 复现的Mask R-CNN是基于Python3，Keras，TensorFlow。 Python 3.4+ TensorFlow 1.3+ Keras 2.0.8+ Jupyter Noteboo...
mask-rcnn训练识别纸箱模型
聂小星
03-26 4613
源码来源：matterport/Mask_RCNN：https://github.com/matterport/Mask_RCNN 一、开发环境及工具 1.开发环境：anaconda3、python3.6、jupyter、pycharm 2.样本标注工具：labelme 3.python包：numpy1.16.1、scipy1.2.1、Pillow5.4.1、cython0.29.6、sc...
mask-CRNN笔记 —— inspect_model
Diana_Z的博客
10-31 806
1、下载coco数据 https://mp.csdn.net/postedit/83576598 2、新建一个*.py文件将*.ipynb的代码复制过去 3、数据路径修改： 修改代码中下图红框处COCO_DIR变量的值为数据文件夹dataset的路径。 运行 1、读dataset 2、加载模型 3、测试模型                  ...
mask-RCNN笔记——inspect_data使用
Diana_Z的博客
10-31 857
mask-RCNN的使用，其中代码inspect_data的使用，用于dateset的读取 1、下载coco数据库，安装coco工具包pycocotools https://blog.csdn.net/Diana_Z/article/details/83576598 2、新建一个*.py文件将*。ipynb的代码复制过去 3、数据路径修改： 修改代码中下图红框处COCO_DIR变量的...
语义分割　Maskrcnn 训练COCO格式数据集
Andrwin的博客
06-21 5574
Maskrcnn 训练COCO格式数据集 使用coco-annotator标注工具 coco.py改为 """ Mask R-CNN Configurations and data loading code for MS COCO. Copyright (c) 2017 Matterport, Inc. Licensed under the MIT License (see LICENS...
Mask_RCNN翻译和详解笔记一（原文翻译+源代码+代码使用说明）
wyx100的专栏
06-01 2万+
原文：https://github.com/matterport/Mask_RCNN 欢迎指正！ 名词表 Mask R-CNN（Mask Region-based Convolutional Neural Network），掩膜基于区域的卷积神经网络 RCNNs：region-based convolutional neural networks，基于区域的卷积神经网络 FPN（Feat...
python遍历文件夹下的文件
qq_40525008的博客
05-16 137
在读文件的时候往往需要遍历文件夹，python的os.path包含了很多文件、文件夹操作的方法。下面列出：os.path.abspath(path) #返回绝对路径os.path.basename(path) #返回文件名os.path.commonprefix(list) #返回多个路径中，所有path共有的最长的路径。os.path.dirname(path) #返回文件路径os.path.e...
解释这段代码 hist = pd.DataFrame(history.history) hist['epoch'] = history.epoch plt.figure() plt.xlabel('Epoch') plt.ylabel('Loss [MSE]') plt.plot(hist['epoch'], hist['mse'],label='mse') plt.legend() plt.show()
03-25
这段代码是用来画出训练模型的历史损失变化曲线。首先创建了一个空的Pandas DataFrame（名称为hist），然后将模型的历史损失数据存入该DataFrame中。 接着在DataFrame中新增了一个名为“epoch”的列，用于记录每次迭代的轮数（即epoch数）。 然后创建一个Matplotlib图表，横轴为epoch数，纵轴为MSE（均方误差） loss损失值，通过plt.plot()方法绘制出mse loss在训练过程中的变化曲线。最后加入图例和标签，并展示图表。
评论 3
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值