Faster RCNN roidb.py

最新推荐文章于 2021-04-07 22:05:24 发布
最新推荐文章于 2021-04-07 22:05:24 发布
阅读量4.4k 收藏 7
点赞数 2
分类专栏： Deep Learning Caffe学习 Faster RCNN Computer Vision
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/iamzhangzhuping/article/details/51345344
版权

1. self.roidb返回一个列表，列表元素为字典，每张图片对应一个字典（包括flipped的图像）

2. prepare_roidb 函数

def prepare_roidb(imdb):
    """Enrich the imdb's roidb by adding some derived quantities that
    are useful for training. This function precomputes the maximum
    overlap, taken over ground-truth boxes, between each ROI and
    each ground-truth box. The class with maximum overlap is also
    recorded.
    该函数主要是用来准备imdb的roidb，主要工作是给roidb中的字典添加一些属性，比如‘image’。
    简单地来说，imdb就是图像数据库，roidb就是图像中的Region Of Intrest 数据库
    """
    sizes = [PIL.Image.open(imdb.image_path_at(i)).size
             for i in xrange(imdb.num_images)]
    # 当在‘Stage 2 Fast R-CNN, init from stage 2 RPN R-CNN model’阶段中，roidb由rpn_roidb（）
    # 方法生成，其中的没一张图像的box不仅仅只有gtbox，还包括rpn_file里面的box。
    roidb = imdb.roidb 

    # print '~~~~~~~~~~~~~~imdb.image_index: {}'.format(imdb.image_index)
    # print '~~~~~~~~~~~~~~imdb.image_index: {} {}'.format(imdb.image_index[0], imdb.image_index[5011]) # 000005 000005 说明image_index是重复的两端合在一起的，即验证了append_flipped_images()中的self._image_index = self._image_index * 2
    # print '~~~~~~roidb[0]: {}'.format(roidb[0])
    # print '~~~~~~len(imdb.image_index): {}'.format(len(imdb.image_index)) #10022
    for i in xrange(len(imdb.image_index)):
        roidb[i]['image'] = imdb.image_path_at(i)
        roidb[i]['width'] = sizes[i][0]
        roidb[i]['height'] = sizes[i][1]
        # need gt_overlaps as a dense array for argmax
        gt_overlaps = roidb[i]['gt_overlaps'].toarray()
        # max overlap with gt over classes (columns)
        max_overlaps = gt_overlaps.max(axis=1)
        # gt class that had the max overlap 返回最大值的aixs=1轴上的坐标，在这里它同时代表某一类
        max_classes = gt_overlaps.argmax(axis=1)
        roidb[i]['max_classes'] = max_classes
        roidb[i]['max_overlaps'] = max_overlaps
        # sanity checks
        # max overlap of 0 => class should be zero (background)
        #检查背景类
        zero_inds = np.where(max_overlaps == 0)[0]
        assert all(max_classes[zero_inds] == 0)
        # max overlap > 0 => class should not be zero (must be a fg class)
        nonzero_inds = np.where(max_overlaps > 0)[0]
        assert all(max_classes[nonzero_inds] != 0)
        # print '~~~~~~~~~~~~~~~~zero_inds: {}'.format(zero_inds)
    # print '~~~~~~~~roidb[0]["gt_overlaps"]: {}'.format(roidb[0]['gt_overlaps'].toarray())
    # print '~~~~~~~~roidb[1]["gt_overlaps"]: {}'.format(roidb[1]['gt_overlaps'].toarray())
    # print '~~~~~~~~roidb[2]["gt_overlaps"]: {}'.format(roidb[2]['gt_overlaps'].toarray())
    # print '~~~~~~~~roidb[3]["gt_overlaps"]: {}'.format(roidb[3]['gt_overlaps'].toarray())

print ‘~~~~roidb[3][“gt_overlaps”]: {}’.format(roidb[3][‘gt_overlaps’].toarray())等测试代码在stage 1 时的输出如下：

{'boxes': array([[262, 210, 323, 338],
                [164, 263, 252, 371],
                [240, 193, 294, 298]], dtype=uint16), 'gt_classes': array([9, 9, 9], dtype=int32), 'gt_overlaps': <3x21 sparse matrix of type '<type 'numpy.float32'>'
    with 3 stored elements in Compressed Sparse Row format>, 'seg_areas': array([ 7998.,  9701.,  5830.], dtype=float32), 'flipped': False}
## 'gt_overlaps': <3x21 sparse matrix of type '<type 'numpy.float32'>' with 3 stored elements in Compressed Sparse Row format> 21代表21类，
#with 3 stored elements in Compressed Sparse Row format> 说明该sparse matrix里面只存储了3个非0值


#roidb[0]
{'gt_classes': array([9, 9, 9], dtype=int32), 'max_classes': array([9, 9, 9]), 'image': '/home/sam/WORKSPACE/py-faster-rcnn/data/VOCdevkit2007/VOC2007/JPEGImages/000005.jpg', 'flipped': False, 'width': 500, 
'boxes': array([[262, 210, 323, 338],
                [164, 263, 252, 371],
                [240, 193, 294, 298]], dtype=uint16), 'max_overlaps': array([ 1.,  1.,  1.], dtype=float32), 'height': 375, 'seg_areas': array([ 7998.,  9701.,  5830.], dtype=float32), 'gt_overlaps': <3x21 sparse matrix of type '<type 'numpy.float32'>'
    with 3 stored elements in Compressed Sparse Row format>}


#roidb[1]
{'gt_classes': array([7], dtype=int32), 'max_classes': array([7]), 'image': '/home/sam/WORKSPACE/py-faster-rcnn/data/VOCdevkit2007/VOC2007/JPEGImages/000007.jpg', 'flipped': False, 'width': 500, 
    'boxes': array([[140,  49, 499, 329]], dtype=uint16), 'max_overlaps': array([ 1.], dtype=float32), 'height': 333, 'seg_areas': array([ 101160.], dtype=float32), 'gt_overlaps': <1x21 sparse matrix of type '<type 'numpy.float32'>'
    with 1 stored elements in Compressed Sparse Row format>}


#roidb[2]
{'gt_classes': array([13, 15, 15, 15], dtype=int32), 'max_classes': array([13, 15, 15, 15]), 'image': '/home/sam/WORKSPACE/py-faster-rcnn/data/VOCdevkit2007/VOC2007/JPEGImages/000009.jpg', 'flipped': False, 'width': 500, 
'boxes': array([[ 68, 171, 269, 329],
                [149, 140, 228, 283],
                [284, 200, 326, 330],
                [257, 197, 296, 328]], dtype=uint16), 'max_overlaps': array([ 1.,  1.,  1.,  1.], dtype=float32), 'height': 375, 'seg_areas': array([ 32118.,  11520.,   5633.,   5280.], dtype=float32), 'gt_overlaps': <4x21 sparse matrix of type '<type 'numpy.float32'>'
    with 4 stored elements in Compressed Sparse Row format>}


#roidb[3]
{'gt_classes': array([7], dtype=int32), 'max_classes': array([7]), 'image': '/home/sam/WORKSPACE/py-faster-rcnn/data/VOCdevkit2007/VOC2007/JPEGImages/000012.jpg', 'flipped': False, 'width': 500, 
'boxes': array([[155,  96, 350, 269]], dtype=uint16), 'max_overlaps': array([ 1.], dtype=float32), 'height': 333, 'seg_areas': array([ 34104.], dtype=float32), 'gt_overlaps': <1x21 sparse matrix of type '<type 'numpy.float32'>'
    with 1 stored elements in Compressed Sparse Row format>}

#roidb[0]["gt_overlaps"]: 
[[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0. 0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0. 0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0. 0.  0.  0.]]

#roidb[1]["gt_overlaps"]: 
[[ 0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0. 0.  0.  0.]]

#roidb[2]["gt_overlaps"]: 
[[ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.]
 [ 0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.]]

#roidb[3]["gt_overlaps"]: 
[[ 0.  0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]]

def add_bbox_regression_targets(roidb): 添加bbox的回归目标（给roidb里面的字典添加了’bbox_targets’属性），同时根据cfg文件来处理bbox_targets的mean stds 问题

def add_bbox_regression_targets(roidb):
    """Add information needed to train bounding-box regressors."""
    assert len(roidb) > 0
    assert 'max_classes' in roidb[0], 'Did you call prepare_roidb first?'

    num_images = len(roidb)
    # Infer number of classes from the number of columns in gt_overlaps
    num_classes = roidb[0]['gt_overlaps'].shape[1]
    for im_i in xrange(num_images):
        rois = roidb[im_i]['boxes']

        # 当在‘Stage 2 Fast R-CNN, init from stage 2 RPN R-CNN model’阶段中，roidb由rpn_roidb（）
        # 方法生成，其中的没一张图像的box不仅仅只有gtbox，还包括rpn_file里面的box。
        max_overlaps = roidb[im_i]['max_overlaps']
        max_classes = roidb[im_i]['max_classes']
        roidb[im_i]['bbox_targets'] = \
                _compute_targets(rois, max_overlaps, max_classes)

    if cfg.TRAIN.BBOX_NORMALIZE_TARGETS_PRECOMPUTED:
        # Use fixed / precomputed "means" and "stds" instead of empirical values
        means = np.tile(
                np.array(cfg.TRAIN.BBOX_NORMALIZE_MEANS), (num_classes, 1))
        stds = np.tile(
                np.array(cfg.TRAIN.BBOX_NORMALIZE_STDS), (num_classes, 1))
    else:
        # Compute values needed for means and stds
        # var(x) = E(x^2) - E(x)^2
        # 针对每个类计算mean 和 std， 因为需要训练class-specific regressors
        class_counts = np.zeros((num_classes, 1)) + cfg.EPS
        sums = np.zeros((num_classes, 4))
        squared_sums = np.zeros((num_classes, 4))
        for im_i in xrange(num_images):
            targets = roidb[im_i]['bbox_targets']
            for cls in xrange(1, num_classes):
                cls_inds = np.where(targets[:, 0] == cls)[0]
                if cls_inds.size > 0:
                    class_counts[cls] += cls_inds.size
                    sums[cls, :] += targets[cls_inds, 1:].sum(axis=0)
                    squared_sums[cls, :] += \
                            (targets[cls_inds, 1:] ** 2).sum(axis=0)

        means = sums / class_counts
        stds = np.sqrt(squared_sums / class_counts - means ** 2)

    print 'bbox target means:'
    print means # 输出21类的bbox_targets的4个offset的均值
    print means[1:, :].mean(axis=0) # ignore bg class
    print 'bbox target stdevs:'
    print stds # 输出21类的bbox_targets的4个offset的std，每一类为1行
    print stds[1:, :].mean(axis=0) # ignore bg class

    # Normalize targets
    if cfg.TRAIN.BBOX_NORMALIZE_TARGETS:
        print "Normalizing targets"
        for im_i in xrange(num_images):
            targets = roidb[im_i]['bbox_targets']
            for cls in xrange(1, num_classes):
                cls_inds = np.where(targets[:, 0] == cls)[0]
                roidb[im_i]['bbox_targets'][cls_inds, 1:] -= means[cls, :]
                roidb[im_i]['bbox_targets'][cls_inds, 1:] /= stds[cls, :]
    else:
        print "NOT normalizing targets"

    # These values will be needed for making predictions
    # (the predicts will need to be unnormalized and uncentered)
    return means.ravel(), stds.ravel()

def _compute_targets(rois, overlaps, labels)
roidb中 键‘boxes’所对应的值为左上、右下两点的坐标，其形式为（w1, h1, w2, h2）

def _compute_targets(rois, overlaps, labels):
    """Compute bounding-box regression targets for an image."""
    # Indices of ground-truth ROIs
    gt_inds = np.where(overlaps == 1)[0]
    if len(gt_inds) == 0:
        # Bail if the image has no ground-truth ROIs
        return np.zeros((rois.shape[0], 5), dtype=np.float32)
    # Indices of examples for which we try to make predictions
    ex_inds = np.where(overlaps >= cfg.TRAIN.BBOX_THRESH)[0]

    # Get IoU overlap between each ex ROI and gt ROI
    # 调用bbox.pyx文件中的bbox_overlaps函数计算RPN Proposals 与 Ground-Truth Box的IOU
    # overlap;IOU overlap为Proposal与gt box的面积交集 除以 面积的并集。
    ex_gt_overlaps = bbox_overlaps(
        np.ascontiguousarray(rois[ex_inds, :], dtype=np.float),
        np.ascontiguousarray(rois[gt_inds, :], dtype=np.float))

    # Find which gt ROI each ex ROI has max overlap with:
    # this will be the ex ROI's gt target
    gt_assignment = ex_gt_overlaps.argmax(axis=1)

    # 这里gt_assignment的size比gt_inds的size大，所以返回gt_rois的size和ex_rois的size一样大，
    #即为ex_rois中的每个roi都assign了一个gt，可见python的灵活强大
    gt_rois = rois[gt_inds[gt_assignment], :]
    ex_rois = rois[ex_inds, :]

    # 调用bbox_transform.py文件中的bbox_transform函数。target返回图片中每个box的结果，
    #被选为（P，G）pair的proposals包含其中，那些不满足overlaps >= cfg.TRAIN.BBOX_THRESH 
    #的proposal也包含其中，只不过相应的值为0.bbox_transform的计算参考论文RCNN
    targets = np.zeros((rois.shape[0], 5), dtype=np.float32)
    targets[ex_inds, 0] = labels[ex_inds]
    targets[ex_inds, 1:] = bbox_transform(ex_rois, gt_rois)
    return targets

bbox.pyx文件中的 def bbox_overlaps(np.ndarray[DTYPE_t, ndim=2] boxes,np.ndarray[DTYPE_t, ndim=2] query_boxes):
返回 ： overlaps: (N, K) ndarray of overlap between boxes and query_boxes

def bbox_overlaps(
        np.ndarray[DTYPE_t, ndim=2] boxes,
        np.ndarray[DTYPE_t, ndim=2] query_boxes):
    """
    Parameters
    ----------
    boxes: (N, 4) ndarray of float
    query_boxes: (K, 4) ndarray of float
    Returns
    -------
    overlaps: (N, K) ndarray of overlap between boxes and query_boxes
    """
    cdef unsigned int N = boxes.shape[0]
    cdef unsigned int K = query_boxes.shape[0]
    cdef np.ndarray[DTYPE_t, ndim=2] overlaps = np.zeros((N, K), dtype=DTYPE)
    cdef DTYPE_t iw, ih, box_area
    cdef DTYPE_t ua
    cdef unsigned int k, n
    for k in range(K):
        box_area = (
            (query_boxes[k, 2] - query_boxes[k, 0] + 1) *
            (query_boxes[k, 3] - query_boxes[k, 1] + 1)
        )
        for n in range(N):
            iw = (
                min(boxes[n, 2], query_boxes[k, 2]) -
                max(boxes[n, 0], query_boxes[k, 0]) + 1
            )
            if iw > 0:
                ih = (
                    min(boxes[n, 3], query_boxes[k, 3]) -
                    max(boxes[n, 1], query_boxes[k, 1]) + 1
                )
                if ih > 0:
                    ua = float(
                        (boxes[n, 2] - boxes[n, 0] + 1) *
                        (boxes[n, 3] - boxes[n, 1] + 1) +
                        box_area - iw * ih
                    )
                    overlaps[n, k] = iw * ih / ua
    return overlaps
iamzhangzhuping
关注
  • 2
    点赞
  • 踩
  • 7
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
训练Faster RCNN出现的错误集合
bingbingxie1的博客
01-03 1987
问题（1）：Appending horizontally-flipped training examples... voc_2007_trainval gt roidb loaded from /home/xiebingbing/py-faster-rcnn/data/cache/voc_2007_trainval_gt_roidb.pkl done Preparing training data
Detectron源码解读-roidb数据结构
ZeroZone零域的博客
10-28 1519
roidb数据结构 roidb的类型是list, 其中的每个元素的数据类型都是dict, roidb列表的长度为数据集的数量(即图片的数量), roidb中每个元素的详细情况如下表所示: for entry in roidb 数据类型 详细说明 entry['id'] int 代表了当前image的img_id entry['file_name'] string 表示当前图片的...
roidb.py
bestrivern的博客
10-04 398
roidb是比较复杂的数据结构，存放了数据集的roi信息。原始的roidb来自数据集，在trian.py的get_training_roidb(imdb)函数进行了水平翻转扩充数量，然后prepare_roidb(imdb)【定义在roidb.py】为roidb添加了一些说明性的属性。 roidb是由字典组成的list，roidb[img_index]包含了该图片索引所包含到roi信息，下面以r...
roidb的结构
qq_34809033的博客
10-20 3080
roidb是由字典组成的list，每张图片对应一个字典，roidb[img_index]包含了该图片索引所包含的roi信息，下面以roidb[img_index]为例说明： box_num：一幅图片中所包含的box数量 class_num：分类所包含的种类数量 roidb[img_index]包含的key value boxes box位置信息，box_num*4的np.ar...
Fast R-CNN roidb数据准备
yang_502的博客
10-10 4785
在Faster R-CNN上项目代码上运行Fast R-CNN。关于初始的roidb数据，主要的几个相关文件有pascal_voc.py,imdb.py,roidb.py等。 （1）运行脚本是 fast_rcnn.sh # ./experiments/scripts/fast_rcnn.sh 0 VGG_CNN_M_1024 pascal_voc --set EXP_DIR foobar RN
FasterRcnn2.m
08-25
matlab中实现Faster-RCNN网路的训练，并可以通过图像文件进行测试，显示测试文件内容，并在其上标注检测目标位置。
虫子识别fasterrcnn模型.tar
06-28
虫子识别fasterrcnn模型.tar 可以用于虫子的fasterrcnn模型，解压直接可以用。
FasterRCNN算法.pptx
03-15
在Faster R-CNN的发展历程中，先有RCNN（Region-based Convolutional Neural Network）的出现，它是首次将卷积神经网络引入物体检测。RCNN首先通过Selective Search等方法生成候选区域，然后对每个区域进行CNN特征...
fasterRCNN.rar
02-10
在“fasterRCNN.rar”压缩包中，包含了基于TensorFlow 2.0实现的Faster R-CNN源码。这个实现可能包括以下部分： 1. 数据预处理模块：负责读取数据集，进行图像预处理，如归一化、缩放等。 2. 模型定义：包括RPN和...
faster-rcnn骨架.zip
02-22
faster-rcnn骨架
cython_bbox.so
11-12
Faster-RCNN在linux tensorflow python3.5下运行时需要的make生成文件。
bbox_overlaps python
jacke121的专栏
04-07 3341
pytorch: import torch def bbox_overlaps(bboxes1, bboxes2, mode='iou', is_aligned=False): """Calculate overlap between two set of bboxes. If ``is_aligned`` is ``False``, then calculate the ious between each bbox of bboxes1 and bboxes2, ot...
Faster-RCNN_TF代码解读16：roi_data_layer/roidb.py
一呆飞仙的博客
09-19 2412
# -------------------------------------------------------- # Fast R-CNN # Copyright (c) 2015 Microsoft # Licensed under The MIT License [see LICENSE for details] # Written by Ross Girshick # ----------
Faster RCNN 训练自己的检测模型
热门推荐
marshb11的专栏
01-03 1万+
一、准备自己的训练数据根据pascal VOC 2007的训练数据集基本架构，第一步，当然是要准备自己的训练图片集，本文直接将自己的准备的图片集（.jpg）扔到如下文件夹下： $(py-faster-rcnn)/data/VOCdevkit2007/VOC2007/JPEGImages第二步，根据上述自己的要训练检测的物体图片集，标注相应的.xml文件(我是自己写了一个简单的矩形框标注工具，生成相应
faster_rcnn(5)代码解析
l_ml_m_lm_m的博客
07-30 445
lib/roi_data_layer/roidb.py # -------------------------------------------------------- # Fast R-CNN # Copyright (c) 2015 Microsoft # Licensed under The MIT License [see LICENSE for details] # Writt...
faster-rcnn系列assert (boxes[:, 2] >= boxes[:, 0]).all()和loss偶尔为nan的问题
Sundrops的专栏
11-16 3427
问题综述这些问题的根源都是faster-rcnn系列在处理生成pascal voc数据集时，为了使像素以0为起点，每个bbox的左上右下坐标都减1,如果你的数据里有坐标为0，一般是x1或y1,这时x1 = 0-1 = 65535，就会出现下面的问题1，2。
1.3argmax axis参数详细解析及理解
qq_40761869的博客
10-10 438
argmax()函数的axis参数特别迷，很奇怪，而且往往就算花了大功夫搞明白之后也还是似懂非懂 下面简要介绍下这个函数 官方介绍 argmax(a, axis=None, out=None) Returns the indices of the maximum values along an axis. Parameters ---------- a : ...
Faster Rcnn源码阅读分析（TF+python版）
Irving.She Blog
07-22 3551
Faster Rcnn源码阅读分析———源码地址：https://github.com/CharlesShang/TFFRCNN1）直接运行已经训练好的模型The demo performs detection using a VGG16 network trained for detection on PASCAL VOC 2007.python demo.py --model /models/V
error: \faster-rcnn.pytorch-master\lib\cythonfnS.pyx
最新发布
06-07
这个错误通常表示您在使用 `faster-rcnn.pytorch` 库时遇到了问题。这个库是用来进行目标检测的。 在出现这个错误之前，您做了什么操作？您是否安装了所有相关的依赖项？请确保您已经按照 `faster-rcnn.pytorch` 的...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值