maskrcnn_benchmark理解记录——由demo\predictor.py引入推断过程

最新推荐文章于 2024-11-20 00:13:18 发布

业精于勤荒于嬉-行成于思而毁于随

最新推荐文章于 2024-11-20 00:13:18 发布

阅读量2.1k

点赞数 3

分类专栏： maskrcnn理解记录

本文链接：https://blog.csdn.net/m0_37644085/article/details/88688943

版权

主要看下run_on_opencv_image(self, image)。大概是这样几步。逐步记录
    predictions = self.compute_prediction(image)          #np.ndarray): an image as returned by OpenCV to tensor
    top_predictions = self.select_top_predictions(predictions)
    result = image.copy()
    result = self.overlay_boxes(result, top_predictions)
    if self.cfg.MODEL.KEYPOINT_ON:
        result = self.overlay_keypoints(result, top_predictions)
    result = self.overlay_class_names(result, top_predictions)
    return result

compute_prediction(self, original_image)

1.先预处理 image = self.transforms(original_image)

->输入original_image=[480,640,3]，int整型数据;
->经过变换后image=[3,800,1066],数据torch.float32;

2.转换为ImageList，填充，使其可被cfg.DATALOADER.SIZE_DIVISIBILITY 除尽，并放到cuda上。

3计算predictions

predictions = self.model(image_list)  #self.model = build_detection_model(cfg)

这里就会跳入 generalized_rcnn.py文件进行模型处理等。更具体的可点链接查看。那这部分就很重要了，有restnet+fpn骨架的建立，rpn进行proposals的生成，和结果预测等。分为三部分：

self.backbone = build_backbone(cfg) 
self.rpn = build_rpn(cfg, self.backbone.out_channels)  ##256 * 4=1024
self.roi_heads = build_roi_heads(cfg, self.backbone.out_channels)

1)modeling\backbone\backbone.py 提取各个stage的特征图;然后使用feature map进行RPN及ROI pooling操作;根据输入图片预处理后。

features = self.backbone(images.tensors)
可以得到P2~P6层参数如下：
ipdb> p features[0].size()
torch.Size([1, 256, 200, 272])
ipdb> p features[1].size()
torch.Size([1, 256, 100, 136])
ipdb> p features[2].size()
torch.Size([1, 256, 50, 68])
ipdb> p features[3].size()
torch.Size([1, 256, 25, 34])
ipdb> p features[4].size()
torch.Size([1, 256, 13, 17])
2)modeling\rpn\rpn.py->经过rpn网络得到候选框，每一层都有1000 proposals。

proposals, proposal_losses = self.rpn(images, features, targets)
proposals格式为：[BoxList(num_boxes=1000, image_width=1066, image_height=800, mode=xyxy)]

3)modeling\roi_heads\roi_heads.py。这里是经过fast rcnn网络，

x, result, detector_losses = self.roi_heads(features, proposals, targets)
由两分支组成：检测分支和分割分支组成;

roi_heads.append(("keypoint", build_roi_keypoint_head(cfg, in_channels)))
roi_heads = CombinedROIHeads(cfg, roi_heads)

在roi_heads.py文件的forward()中：x, detections, loss_box = self.box(features, proposals, targets)得到检测结果，

# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
import cv2
import torch
from torchvision import transforms as T

from maskrcnn_benchmark.modeling.detector import build_detection_model
from maskrcnn_benchmark.utils.checkpoint import DetectronCheckpointer
from maskrcnn_benchmark.structures.image_list import to_image_list
from maskrcnn_benchmark.modeling.roi_heads.mask_head.inference import Masker
from maskrcnn_benchmark import layers as L
from maskrcnn_benchmark.utils import cv2_util
from tensorboardX import Su