faster rcnn 训练自己数据集记录

Caffe各种Bug修正
./tools/test_net.py --gpu 0 --def models/pascal_voc/VGG16/faster_rcnn_end2end/test.prototxt --net /home/drl/project2/py-faster-rcnn/output/faster_rcnn_end2end/voc_2007_trainval/vgg16_faster_rcnn_iter_220.caffemodel --imdb voc_2007_test --cfg experiments/cfgs/faster_rcnn_end2end.yml

 

./experiments/scripts/faster_rcnn_end2end.sh 0 VGG16 pascal_voc


./tools/train_net.py --gpu 0 --solver models/pascal_voc/VGG16/faster_rcnn_end2end/solver.prototxt   --imdb voc_2007_trainval --iters 50 --cfg experiments/cfgs/faster_rcnn_end2end.yml

 ./experiments/scripts/faster_rcnn_end2end.sh 0 VGG16 pascal_voc

./tools/train_net.py --gpu 0 --solver models/pascal_voc/VGG16/faster_rcnn_end2end/solver.prototxt --imdb voc_2007_trainval --iters 70000 --cfg experiments/cfgs/faster_rcnn_end2end.yml --weights /home/drl/project2/py-faster-rcnn/data/faster_rcnn_models/VGG16.v2.caffemodel

./tools/demo.py

1. AttributeError: 'module' object has no attribute 'text_format'


2. bbox_targets[ind, start:end] = bbox_target_data[ind, 1:]
TypeError: slice indices must be integers or None or have an __index__ method

An easier solution is to add the following lines to lib/proposal_target_layer.py
After line 126,

start=int(start)
end=int(end)

After line 166,

fg_rois_per_this_image=int(fg_rois_per_this_image)


3.  BB = BB[sorted_ind, :]
IndexError: too many indices for array

4. 修改test.protxt和train.protxt的cls_score和bbox_pred 为_1
修改test.py bbox_pred为bbox_pred_1

修改voc_eval.py int变成float

5. VGG16.V2.CAFFEMODEL

6. assert (boxes[:, 2] >= boxes[:, 0]).all()
AssertionError


2、自己制作了VOC或者coco数据集格式

如果你自己制作了voc pascal或者coco数据集格式,那么你需要注意,看看是否有类似下面的报错

RuntimeWarning: invalid value encountered in log targets_dw = np.log(gt_widths / ex_widths)

这种报错说明数据集的数据有一些问题,多出现在没有控制好边界的情况,首先,打开lib/database/pascal_voc.py文件,找到208行,将208行至211行每一行后面的-1删除,如下所示:

x1 = float(bbox.find(‘xmin’).text)
y1 = float(bbox.find(‘ymin’).text)
x2 = float(bbox.find(‘xmax’).text)
y2 = float(bbox.find(‘ymax’).text)
原因是因为我们制作的xml文件中有些框的坐标是从左上角开始的,也就是(0,0)如果再减一就会出现log(-1)的情况

如果这样之后还是出现类似的报错,那么说明依然有-1或者其他负数的情况出现。解决方法是打开./lib/model/config.py文件,找到flipp选项,将其置为False
__C.TRAIN.USE_FLIPPED = False

如果这样以后还是报类似的错误,就一定检查一下自己制作数据集的过程,看看是否哪里没有考虑清
---------------------
作者:ZeroZone零域
来源:CSDN
原文:https://blog.csdn.net/ksws0292756/article/details/80702704
版权声明:本文为博主原创文章,转载请附上博文链接!


一. 修改VGG_CNN_M_1024模型配置文件

1)train.prototxt文件

      input-data层的num_class数值由21改为2;

      roi-data层的num_class数值由21改为2;

      cls_score层的num_output数值由21改为2(1+1);

      bbox_pred层的num_output数值由84改为8(2*4);

2)test.prototxt文件(c++dll调用的.prototxt也要改)

cls_score层的num_output数值由21改为2(1+1);

bbox_pred层的num_output数值由84改为8(2*4);

3)lib/datasets/pascal_voc.py文件

       修改self._classes = ('__background__',  '训练的数据类别')

 

三. cache问题

在重新训练新的数据之前将cache删除

1) py-faster-rcnn/output
2) py-faster-rcnn/data/cache

 

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
Faster R-CNN 是一种基于深度学习的目标检测算法,它可以用于训练自己的数据集。下面是 Faster R-CNN 训练自己数据集的代码示例: 1. 准备训练数据集 首先需要准备训练数据集,包括图像和标注文件。标注文件可以是 VOC 格式或 COCO 格式。 2. 安装依赖库和下载代码 需要安装 TensorFlow 和 Keras,以及下载 Faster R-CNN 的代码。 3. 修改配置文件 修改 Faster R-CNN 的配置文件,包括训练和测试的参数、数据集路径以及模型保存路径等。 4. 训练模型 运行训练代码,使用准备好的数据集进行训练,直到模型收敛或达到预设的训练轮数。 5. 测试模型 使用测试数据集训练好的模型进行测试,评估模型的准确率和召回率等指标。 6. 模型优化 根据测试结果对模型进行优化,包括调整参数、增加数据集大小等。 参考代码: 以下是 Faster R-CNN 训练自己数据集的代码示例。这里以 TensorFlow 和 Keras 为例,代码中的数据集为 VOC 格式。 ```python # 导入依赖库 import tensorflow as tf from keras import backend as K from keras.layers import Input from keras.models import Model from keras.optimizers import Adam from keras.utils import plot_model from keras.callbacks import TensorBoard, ModelCheckpoint from keras_frcnn import config from keras_frcnn import data_generators from keras_frcnn import losses as losses_fn from keras_frcnn import roi_helpers from keras_frcnn import resnet as nn from keras_frcnn import visualize # 设置配置文件 config_output_filename = 'config.pickle' network = 'resnet50' num_epochs = 1000 output_weight_path = './model_frcnn.hdf5' input_weight_path = './resnet50_weights_tf_dim_ordering_tf_kernels.h5' tensorboard_dir = './logs' train_path = './train.txt' test_path = './test.txt' num_rois = 32 horizontal_flips = True vertical_flips = True rot_90 = True output_weight_path = './model_frcnn.hdf5' # 加载配置文件 config = config.Config() config_output_filename = 'config.pickle' # 加载数据集 all_imgs, classes_count, class_mapping = data_generators.get_data(train_path) test_imgs, _, _ = data_generators.get_data(test_path) # 计算平均像素值 if 'bg' not in classes_count: classes_count['bg'] = 0 class_mapping['bg'] = len(class_mapping) config.class_mapping = class_mapping # 计算平均像素值 C = config.num_channels mean_pixel = [103.939, 116.779, 123.68] img_size = (config.im_size, config.im_size) # 组装模型 input_shape_img = (None, None, C) img_input = Input(shape=input_shape_img) roi_input = Input(shape=(num_rois, 4)) shared_layers = nn.nn_base(img_input, trainable=True) # RPN 网络 num_anchors = len(config.anchor_box_scales) * len(config.anchor_box_ratios) rpn_layers = nn.rpn(shared_layers, num_anchors) # RoI 网络 classifier = nn.classifier(shared_layers, roi_input, num_rois, nb_classes=len(classes_count), trainable=True) model_rpn = Model(img_input, rpn_layers) model_classifier = Model([img_input, roi_input], classifier) # 加载权重 model_rpn.load_weights(input_weight_path, by_name=True) model_classifier.load_weights(input_weight_path, by_name=True) # 生成训练数据 data_gen_train = data_generators.get_anchor_gt(all_imgs, classes_count, C, K.image_dim_ordering(), mode='train', \ img_size=img_size, \ num_rois=num_rois, \ horizontal_flips=horizontal_flips, \ vertical_flips=vertical_flips, \ rot_90=rot_90) # 编译模型 optimizer = Adam(lr=1e-5) model_rpn.compile(optimizer=optimizer, loss=[losses_fn.rpn_loss_cls(num_anchors), losses_fn.rpn_loss_regr(num_anchors)]) model_classifier.compile(optimizer=optimizer, loss=[losses_fn.class_loss_cls, losses_fn.class_loss_regr(len(classes_count) - 1)], metrics={'dense_class_{}'.format(len(classes_count)): 'accuracy'}) # 训练模型 epoch_length = 1000 num_epochs = int(num_epochs) iter_num = 0 losses = np.zeros((epoch_length, 5)) rpn_accuracy_rpn_monitor = [] rpn_accuracy_for_epoch = [] start_time = time.time() best_loss = np.Inf class_mapping_inv = {v: k for k, v in class_mapping.items()} print('Starting training') for epoch_num in range(num_epochs): progbar = generic_utils.Progbar(epoch_length) print('Epoch {}/{}'.format(epoch_num + 1, num_epochs)) while True: try: if len(rpn_accuracy_rpn_monitor) == epoch_length and C.verbose: mean_overlapping_bboxes = float(sum(rpn_accuracy_rpn_monitor)) / len(rpn_accuracy_rpn_monitor) rpn_accuracy_rpn_monitor = [] print('Average number of overlapping bounding boxes from RPN = {} for {} previous iterations'.format(mean_overlapping_bboxes, epoch_length)) if mean_overlapping_bboxes == 0: print('RPN is not producing bounding boxes that overlap the ground truth boxes. Check RPN settings or keep training.') X, Y, img_data = next(data_gen_train) loss_rpn = model_rpn.train_on_batch(X, Y) P_rpn = model_rpn.predict_on_batch(X) R = roi_helpers.rpn_to_roi(P_rpn[0], P_rpn[1], C.image_dim_ordering(), use_regr=True, overlap_thresh=0.7, max_boxes=300) X2, Y1, Y2, IouS = roi_helpers.calc_iou(R, img_data, C, class_mapping) if X2 is None: rpn_accuracy_rpn_monitor.append(0) rpn_accuracy_for_epoch.append(0) continue # sampling positive/negative samples neg_samples = np.where(Y1[0, :, -1] == 1) pos_samples = np.where(Y1[0, :, -1] == 0) if len(neg_samples) > 0: neg_samples = neg_samples[0] else: neg_samples = [] if len(pos_samples) > 0: pos_samples = pos_samples[0] else: pos_samples = [] rpn_accuracy_rpn_monitor.append(len(pos_samples)) rpn_accuracy_for_epoch.append((len(pos_samples))) if C.num_rois > 1: if len(pos_samples) < C.num_rois // 2: selected_pos_samples = pos_samples.tolist() else: selected_pos_samples = np.random.choice(pos_samples, C.num_rois // 2, replace=False).tolist() try: selected_neg_samples = np.random.choice(neg_samples, C.num_rois - len(selected_pos_samples), replace=False).tolist() except: selected_neg_samples = np.random.choice(neg_samples, C.num_rois - len(selected_pos_samples), replace=True).tolist() sel_samples = selected_pos_samples + selected_neg_samples else: # in the extreme case where num_rois = 1, we pick a random pos or neg sample selected_pos_samples = pos_samples.tolist() selected_neg_samples = neg_samples.tolist() if np.random.randint(0, 2): sel_samples = random.choice(neg_samples) else: sel_samples = random.choice(pos_samples) loss_class = model_classifier.train_on_batch([X, X2[:, sel_samples, :]], [Y1[:, sel_samples, :], Y2[:, sel_samples, :]]) losses[iter_num, 0] = loss_rpn[1] losses[iter_num, 1] = loss_rpn[2] losses[iter_num, 2] = loss_class[1] losses[iter_num, 3] = loss_class[2] losses[iter_num, 4] = loss_class[3] iter_num += 1 progbar.update(iter_num, [('rpn_cls', np.mean(losses[:iter_num, 0])), ('rpn_regr', np.mean(losses[:iter_num, 1])), ('detector_cls', np.mean(losses[:iter_num, 2])), ('detector_regr', np.mean(losses[:iter_num, 3])), ('mean_overlapping_bboxes', float(sum(rpn_accuracy_for_epoch)) / len(rpn_accuracy_for_epoch))]) if iter_num == epoch_length: loss_rpn_cls = np.mean(losses[:, 0]) loss_rpn_regr = np.mean(losses[:, 1]) loss_class_cls = np.mean(losses[:, 2]) loss_class_regr = np.mean(losses[:, 3]) class_acc = np.mean(losses[:, 4]) mean_overlapping_bboxes = float(sum(rpn_accuracy_for_epoch)) / len(rpn_accuracy_for_epoch) rpn_accuracy_for_epoch = [] if C.verbose: print('Mean number of bounding boxes from RPN overlapping ground truth boxes: {}'.format(mean_overlapping_bboxes)) print('Classifier accuracy for bounding boxes from RPN: {}'.format(class_acc)) print('Loss RPN classifier: {}'.format(loss_rpn_cls)) print('Loss RPN regression: {}'.format(loss_rpn_regr)) print('Loss Detector classifier: {}'.format(loss_class_cls)) print('Loss Detector regression: {}'.format(loss_class_regr)) print('Elapsed time: {}'.format(time.time() - start_time)) curr_loss = loss_rpn_cls + loss_rpn_regr + loss_class_cls + loss_class_regr iter_num = 0 start_time = time.time() if curr_loss < best_loss: if C.verbose: print('Total loss decreased from {} to {}, saving weights'.format(best_loss, curr_loss)) best_loss = curr_loss model_rpn.save_weights(output_weight_path) model_classifier.save_weights(output_weight_path) break except Exception as e: print('Exception: {}'.format(e)) continue print('Training complete, exiting.') ``` 这是一个简单的 Faster R-CNN 训练自己数据集的示例代码,可以根据自己的数据集和需求进行修改和优化。

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值