CV之detectron2:detectron2的简介、安装、使用方法之详细攻略
目录
2、Build and Install Detectron2
3、Detectron2 Model Zoo and Baselines
COCO Object Detection Baselines
COCO Instance Segmentation Baselines with Mask R-CNN
COCO Person Keypoint Detection Baselines with Keypoint R-CNN
COCO Panoptic Segmentation Baselines with Panoptic FPN
LVIS Instance Segmentation Baselines with Mask R-CNN
Cityscapes & Pascal VOC Baselines
detectron2的简介
Detectron is Facebook AI Research's software system that implements state-of-the-art object detection algorithms, including Mask R-CNN. It is written in Python and powered by the Caffe2 deep learning framework. At FAIR, Detectron has enabled numerous research projects, including: Feature Pyramid Networks for Object Detection, Mask R-CNN, Detecting and Recognizing Human-Object Interactions, Focal Loss for Dense Object Detection, Non-local Neural Networks, Learning to Segment Every Thing, Data Distillation: Towards Omni-Supervised Learning, DensePose: Dense Human Pose Estimation In The Wild, and Group Normalization. | Detectron是Facebook人工智能研究的软件系统,它实现了最先进的目标检测算法,包括Mask R-CNN。它是用Python编写的,由Caffe2深度学习框架提供支持。 在Facebook人工智能研究中,Detectron已经启动了许多研究项目,包括:用于物体检测的特征金字塔网络、掩模R-CNN、检测和识别人类与物体的相互作用、用于密集物体检测的焦距损失、非局部神经网络、学习分割每件事物、数据蒸馏:朝向全监督学习,DensePose:在野外进行密集的人体姿势估计,并进行组规范化。 |
Detectron2 is Facebook AI Research's next generation software system that implements state-of-the-art object detection algorithms. It is a ground-up rewrite of the previous version, Detectron, and it originates from maskrcnn-benchmark. | Detectron2是Facebook人工智能研究的下一代软件系统,实现了最先进的目标检测算法。它是对先前版本Detectron的一次彻底重写,它源于maskrcnn基准测试。 |
GitHub:
Detectron,https://github.com/facebookresearch/Detectron/
Detectron2,https://github.com/facebookresearch/detectron2
Detectron的目的是为目标检测研究提供高质量、高性能的codebase。它的设计是灵活的,以支持快速实施和评估的新研究。Detectron包括以下对象检测算法的实现:
- Mask R-CNN -- Marr Prize at ICCV 2017
- RetinaNet -- Best Student Paper Award at ICCV 2017
- Faster R-CNN
- RPN
- Fast R-CNN
- R-FCN
采用下列主干网络架构:
- ResNeXt{50,101,152}
- ResNet{50,101,152}
- Feature Pyramid Networks (with ResNet/ResNeXt)
- VGG16
附加的主干架构可能很容易实现。有关这些模型的详细信息,请参阅下面的参考资料。
1、Detectron2—What's New
- It is powered by the PyTorch deep learning framework.
- Includes more features such as panoptic segmentation, densepose, Cascade R-CNN, rotated bounding boxes, etc.
- Can be used as a library to support different projects on top of it. We'll open source more research projects in this way.
- It trains much faster.
detectron2的安装
1、Requirements
- Linux or macOS
- Python ≥ 3.6
- PyTorch ≥ 1.3
- torchvision that matches the PyTorch installation. You can install them together at pytorch.org to make sure of this.
- OpenCV, needed by demo and visualization
- pycocotools:
pip install cython; pip install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
- GCC ≥ 4.9
2、Build and Install Detectron2
1、官方安装
git clone https://github.com/facebookresearch/detectron2.git
cd detectron2
pip install -e .
# (add --user if you don't have permission)
# or if you are on macOS
# MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ pip install -e .
2、Windows下安装
git clone https://github.com/facebookresearch/detectron2.git
cd detectron2
python setup.py build develop
相关文章:CV之detectron2:detectron2安装过程记录
3、Detectron2 Model Zoo and Baselines
COCO Object Detection Baselines
Faster R-CNN:
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | model id | download |
---|---|---|---|---|---|---|---|
R50-C4 | 1x | 0.551 | 0.110 | 4.8 | 35.7 | 137257644 | model | metrics |
R50-DC5 | 1x | 0.380 | 0.068 | 5.0 | 37.3 | 137847829 | model | metrics |
R50-FPN | 1x | 0.210 | 0.055 | 3.0 | 37.9 | 137257794 | model | metrics |
R50-C4 | 3x | 0.543 | 0.110 | 4.8 | 38.4 | 137849393 | model | metrics |
R50-DC5 | 3x | 0.378 | 0.073 | 5.0 | 39.0 | 137849425 | model | metrics |
R50-FPN | 3x | 0.209 | 0.047 | 3.0 | 40.2 | 137849458 | model | metrics |
R101-C4 | 3x | 0.619 | 0.149 | 5.9 | 41.1 | 138204752 | model | metrics |
R101-DC5 | 3x | 0.452 | 0.082 | 6.1 | 40.6 | 138204841 | model | metrics |
R101-FPN | 3x | 0.286 | 0.063 | 4.1 | 42.0 | 137851257 | model | metrics |
X101-FPN | 3x | 0.638 | 0.120 | 6.7 | 43.0 | 139173657 | model | metrics |
RetinaNet:
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | model id | download |
---|---|---|---|---|---|---|---|
R50 | 1x | 0.200 | 0.062 | 3.9 | 36.5 | 137593951 | model | metrics |
R50 | 3x | 0.201 | 0.063 | 3.9 | 37.9 | 137849486 | model | metrics |
R101 | 3x | 0.280 | 0.080 | 5.1 | 39.9 | 138363263 | model | metrics |
RPN & Fast R-CNN:
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | prop. AR | model id | download |
---|---|---|---|---|---|---|---|---|
RPN R50-C4 | 1x | 0.130 | 0.051 | 1.5 | 51.6 | 137258005 | model | metrics | |
RPN R50-FPN | 1x | 0.186 | 0.045 | 2.7 | 58.0 | 137258492 | model | metrics | |
Fast R-CNN R50-FPN | 1x | 0.140 | 0.035 | 2.6 | 37.8 | 137635226 | model | metrics |
COCO Instance Segmentation Baselines with Mask R-CNN
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | mask AP | model id | download |
---|---|---|---|---|---|---|---|---|
R50-C4 | 1x | 0.584 | 0.117 | 5.2 | 36.8 | 32.2 | 137259246 | model | metrics |
R50-DC5 | 1x | 0.471 | 0.074 | 6.5 | 38.3 | 34.2 | 137260150 | model | metrics |
R50-FPN | 1x | 0.261 | 0.053 | 3.4 | 38.6 | 35.2 | 137260431 | model | metrics |
R50-C4 | 3x | 0.575 | 0.118 | 5.2 | 39.8 | 34.4 | 137849525 | model | metrics |
R50-DC5 | 3x | 0.470 | 0.075 | 6.5 | 40.0 | 35.9 | 137849551 | model | metrics |
R50-FPN | 3x | 0.261 | 0.055 | 3.4 | 41.0 | 37.2 | 137849600 | model | metrics |
R101-C4 | 3x | 0.652 | 0.155 | 6.3 | 42.6 | 36.7 | 138363239 | model | metrics |
R101-DC5 | 3x | 0.545 | 0.155 | 7.6 | 41.9 | 37.3 | 138363294 | model | metrics |
R101-FPN | 3x | 0.340 | 0.070 | 4.6 | 42.9 | 38.6 | 138205316 | model | metrics |
X101-FPN | 3x | 0.690 | 0.129 | 7.2 | 44.3 | 39.5 | 139653917 | model | metrics |
COCO Person Keypoint Detection Baselines with Keypoint R-CNN
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | kp. AP | model id | download |
---|---|---|---|---|---|---|---|---|
R50-FPN | 1x | 0.315 | 0.083 | 5.0 | 53.6 | 64.0 | 137261548 | model | metrics |
R50-FPN | 3x | 0.316 | 0.076 | 5.0 | 55.4 | 65.5 | 137849621 | model | metrics |
R101-FPN | 3x | 0.390 | 0.090 | 6.1 | 56.4 | 66.1 | 138363331 | model | metrics |
X101-FPN | 3x | 0.738 | 0.142 | 8.7 | 57.3 | 66.0 | 139686956 | model | metrics |
COCO Panoptic Segmentation Baselines with Panoptic FPN
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | mask AP | PQ | model id | download |
---|---|---|---|---|---|---|---|---|---|
R50-FPN | 1x | 0.304 | 0.063 | 4.8 | 37.6 | 34.7 | 39.4 | 139514544 | model | metrics |
R50-FPN | 3x | 0.302 | 0.063 | 4.8 | 40.0 | 36.5 | 41.5 | 139514569 | model | metrics |
R101-FPN | 3x | 0.392 | 0.078 | 6.0 | 42.4 | 38.5 | 43.0 | 139514519 | model | metrics |
LVIS Instance Segmentation Baselines with Mask R-CNN
Mask R-CNN baselines on the LVIS dataset, v0.5. These baselines are described in Table 3(c) of the LVIS paper.
NOTE: the 1x schedule here has the same amount of iterations as the COCO 1x baselines. They are roughly 24 epochs of LVISv0.5 data. The final results of these configs have large variance across different runs.
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | mask AP | model id | download |
---|---|---|---|---|---|---|---|---|
R50-FPN | 1x | 0.292 | 0.127 | 7.1 | 23.6 | 24.4 | 144219072 | model | metrics |
R101-FPN | 1x | 0.371 | 0.124 | 7.8 | 25.6 | 25.9 | 144219035 | model | metrics |
X101-FPN | 1x | 0.712 | 0.166 | 10.2 | 26.7 | 27.1 | 144219108 | model | metrics |
Cityscapes & Pascal VOC Baselines
Simple baselines for
- Mask R-CNN on Cityscapes instance segmentation (initialized from COCO pre-training, then trained on Cityscapes fine annotations only)
- Faster R-CNN on PASCAL VOC object detection (trained on VOC 2007 train+val + VOC 2012 train+val, tested on VOC 2007 using 11-point interpolated AP)
Name | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | box AP50 | mask AP | model id | download |
---|---|---|---|---|---|---|---|---|
R50-FPN, Cityscapes | 0.240 | 0.092 | 4.4 | 36.5 | 142423278 | model | metrics | ||
R50-C4, VOC | 0.537 | 0.086 | 4.8 | 51.9 | 80.3 | 142202221 | model | metrics |
Other Settings
Ablations for Deformable Conv and Cascade R-CNN:
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | mask AP | model id | download |
---|---|---|---|---|---|---|---|---|
Baseline R50-FPN | 1x | 0.261 | 0.053 | 3.4 | 38.6 | 35.2 | 137260431 | model | metrics |
Deformable Conv | 1x | 0.342 | 0.061 | 3.5 | 41.5 | 37.5 | 138602867 | model | metrics |
Cascade R-CNN | 1x | 0.317 | 0.066 | 4.0 | 42.1 | 36.4 | 138602847 | model | metrics |
Baseline R50-FPN | 3x | 0.261 | 0.055 | 3.4 | 41.0 | 37.2 | 137849600 | model | metrics |
Deformable Conv | 3x | 0.349 | 0.066 | 3.5 | 42.7 | 38.5 | 144998336 | model | metrics |
Cascade R-CNN | 3x | 0.328 | 0.075 | 4.0 | 44.3 | 38.5 | 144998488 | model | metrics |
Ablations for normalization methods: (Note: The baseline uses 2fc
head while the others use 4conv1fc
head. According to the GroupNorm paper, the change in head does not improve the baseline by much)
Name | lr sched | train time (s/iter) | inference time (s/im) | train mem (GB) | box AP | mask AP | model id | download |
---|---|---|---|---|---|---|---|---|
Baseline R50-FPN | 3x | 0.261 | 0.055 | 3.4 | 41.0 | 37.2 | 137849600 | model | metrics |
SyncBN | 3x | 0.464 | 0.063 | 5.6 | 42.0 | 37.8 | 143915318 | model | metrics |
GN | 3x | 0.356 | 0.077 | 7.3 | 42.6 | 38.6 | 138602888 | model | metrics |
GN (scratch) | 3x | 0.400 | 0.077 | 9.8 | 39.9 | 36.6 | 138602908 | model | metrics |
A few very large models trained for a long time, for demo purposes:
Name | inference time (s/im) | train mem (GB) | box AP | mask AP | PQ | model id | download |
---|---|---|---|---|---|---|---|
Panoptic FPN R101 | 0.123 | 11.4 | 47.4 | 41.3 | 46.1 | 139797668 | model | metrics |
Mask R-CNN X152 | 0.281 | 15.1 | 50.2 | 44.0 | 18131413 | model | metrics | |
above + test-time aug. | 51.9 | 45.9 |
detectron2的使用方法
1、demo测试
python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml --input input1.jpg input2.jpg [--other-options] --opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl