基于mmdetection训练自定义数据集踩坑记录（一）

2024.2.29记录

---

前言

基于mmdetection，训练自定义数据集，包含目标共3类，分别为：cross、box、ship。主要是先调通代码，没有过多的内容。

---

一、复现过程踩的坑

1 新建配置文件

新建配置文件：faster-rcnn_r50_fpn_1x_coco_custom_dataset.py，放在文件mmdetection/configs/__mytest/下，文件代码内容如下：

_base_ = '../faster_rcnn/faster-rcnn_r50_fpn_1x_coco.py'

data_root = 'data/coco/'
metainfo = {
        'classes': ('cross', 'box', 'ship'),
        'palette': [(220, 20, 60), (220, 20, 60), (220, 20, 60),]
}

model = dict(
    roi_head=dict(
        bbox_head=dict(num_classes=3)))

2 复现过程遇到的问题

2.1 训练

在Teriminal终端执行代码：

python tools/train.py configs/__mytest/faster-rcnn_r50_fpn_1x_coco_custom_dataset.py

报错如下：

Traceback (most recent call last):
  File "tools/train.py", line 121, in <module>
    main()
  File "tools/train.py", line 117, in main
    runner.train()
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\runner\runner.py", line 1728, in train
    self._train_loop = self.build_train_loop(
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\runner\runner.py", line 1520, in build_train_loop
    loop = LOOPS.build(
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\registry\registry.py", line 570, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\registry\build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\runner\loops.py", line 44, in __init__
    super().__init__(runner, dataloader)
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\runner\base_loop.py", line 26, in __init__
    self.dataloader = runner.build_dataloader(
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\runner\runner.py", line 1370, in build_dataloader
    dataset = DATASETS.build(dataset_cfg)
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\registry\registry.py", line 570, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\registry\build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmdet-3.3.0-py3.8.egg\mmdet\datasets\base_det_dataset.py", line 51, in __init__
    super().__init__(*args, **kwargs)
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\dataset\base_dataset.py", line 247, in __init__
    self.full_init()
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmdet-3.3.0-py3.8.egg\mmdet\datasets\base_det_dataset.py", line 89, in full_init
    self.data_bytes, self.data_address = self._serialize_data()
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\dataset\base_dataset.py", line 768, in _serialize_data
    data_bytes = np.concatenate(data_list)
  File "<__array_function__ internals>", line 200, in concatenate
ValueError: need at least one array to concatenate

解决办法：

step 1：修改faster-rcnn_r50_fpn_1x_coco_custom_dataset.py文件中代码为：

_base_ = '../faster_rcnn/faster-rcnn_r50_fpn_1x_coco.py'

model = dict(
    roi_head=dict(
        bbox_head=dict(num_classes=3)))

step 2：在tools/train.py文件中添加以下代码。

from mmdet.datasets.coco import CocoDataset
CocoDataset.METAINFO = {
        'classes': ('cross', 'box', 'ship'),
        'palette': [(220, 20, 60), (220, 20, 60), (220, 20, 60),]
}

代码成功运行，并在mmdetection文件路径下生成work_dirs文件夹。

2.2 测试，得到pkl文件

在Teriminal终端执行代码：

python tools/test.py configs/__mytest/faster-rcnn_r50_fpn_1x_coco_custom_dataset.py work_dirs/faster-rcnn_r50_fpn_1x_coco_custom_dataset/epoch_12.pth --out work_dirs/faster-rcnn_r50_fpn_1x_coco_custom_dataset/result.pkl

报错如下：

Traceback (most recent call last):
  File "tools/test.py", line 149, in <module>
    main()
  File "tools/test.py", line 145, in main
    runner.test()
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\runner\runner.py", line 1823, in test
    metrics = self.test_loop.run()  # type: ignore
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\runner\loops.py", line 446, in run
    metrics = self.evaluator.evaluate(len(self.dataloader.dataset))
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\evaluator\evaluator.py", line 79, in evaluate
    _results = metric.evaluate(size)
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmengine\evaluator\metric.py", line 133, in evaluate
    _metrics = self.compute_metrics(results)  # type: ignore
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmdet-3.3.0-py3.8.egg\mmdet\evaluation\metrics\coco_metric.py", line 423, in compute_metrics
    result_files = self.results2json(preds, outfile_prefix)
  File "D:\Anaconda\envs\openmmlab\lib\site-packages\mmdet-3.3.0-py3.8.egg\mmdet\evaluation\metrics\coco_metric.py", line 243, in results2json
    data['category_id'] = self.cat_ids[label]
IndexError: list index out of range

解决办法：

step 1：在mmdetection/mmdet/datasets/coco.py文件中，修改classer类别，如图所示：

step 2：在mmdetection文件夹路径下，再次编译环境。实现方式为：在Teriminal终端执行以下代码：

python setup.py install

完成以上修改后，再次执行代码，不再报错。

---

！！以上是踩坑的详细过程，若想跳过，直接参考下面内容即可。

---

二、踩坑后的复现记录

1 修改coco.py文件

在mmdetection/mmdet/datasets/coco.py文件中，根据自定义数据集的详细情况，修改classer类别。

2 编译环境

在Teriminal终端执行代码：

python setup.py install

3 训练

python tools/train.py configs/__mytest/faster-rcnn_r50_fpn_1x_coco_custom_dataset.py

4 测试

result.pkl的保存路径可自行定义

python tools/test.py configs/__mytest/faster-rcnn_r50_fpn_1x_coco_custom_dataset.py work_dirs/faster-rcnn_r50_fpn_1x_coco_custom_dataset/epoch_12.pth --out work_dirs/faster-rcnn_r50_fpn_1x_coco_custom_dataset/result.pkl

总结

以上仅为我的解决办法，水平有限，欢迎朋友们留言指正！

特别感谢：
https://github.com/open-mmlab/mmdetection/issues/2781
https://www.codenong.com/cs107067843/

碎碎念

在代码复现过程中，遇到的问题一个接一个，总是改完这个，还有下一个bug出现，太痛苦了555555555。

坚强！要坚强！

要学会在官网的issues中找问题的解决办法，慢慢来，一定可以的！加油！加油！