在detectron用自己的数据训练Faster-Rcnn+FPN

我的系统ubuntu16.04

代码放在github上

https://github.com/withyou1771/Detectron_FocalLoss

欢迎大家点亮星星~~~

安装caffe2

1、
Required Dependencies
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      git \
      libgoogle-glog-dev \
      libprotobuf-dev \
      protobuf-compiler \
      python-dev \
      python-pip                          

sudo pip install numpy protobuf

2、Optional Dependencies


sudo apt-get install -y --no-install-recommends libgflags-dev
sudo apt-get install -y --no-install-recommends \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      openmpi-bin \
      openmpi-doc \
      python-pydot
sudo pip install \
      flask \
      future \
      graphviz \
      hypothesis \
      jupyter \
      matplotlib \
      pydot python-nvd3 \
      pyyaml \
      requests \
      scikit-image \
      scipy \
      setuptools \
      six \
      tornado

3、
git clone --recursive https://github.com/caffe2/caffe2.git && cd caffe2
make && cd build && sudo make install

python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

4、
python -m caffe2.python.operator_test.relu_op_test


5、
sudo vim ~/.bashrc

export PYTHONPATH=/usr/local:$PYTHONPATH
export PYTHONPATH=$PYTHONPATH:/home/ubuntu/caffe2/build
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

source ~/.bashrc

可以打开python试试能不能import caffe2

安装detectron

pip install numpy pyyaml matplotlib opencv-python>=3.0 setuptools Cython mock
# DETECTRON=/path/to/clone/detectron
git clone https://github.com/withyou1771/Detectron_FocalLoss.git
cd $Detectron_FocalLoss/lib && make

python2 $DETECTRON/tests/test_spatial_narrow_as_op.py

安装python依赖:
pip install numpy pyyaml matplotlib opencv-python>=3.0 setuptools Cython mock

安装 COCO API:
# COCOAPI=/path/to/clone/cocoapi
git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
cd $COCOAPI/PythonAPI
# Install into global site-packages
make install
# Alternatively, if you do not have permissions or prefer
# not to install the COCO API into global site-packages
python2 setup.py install --user

Creating Symlinks for COCO
ln -s /path/to/coco $DETECTRON/lib/datasets/data/coco
mkdir -p $DETECTRON/lib/datasets/data/coco
ln -s /path/to/coco_train2014 $DETECTRON/lib/datasets/data/coco/
ln -s /path/to/coco_val2014 $DETECTRON/lib/datasets/data/coco/

ln -s /path/to/json/annotations $DETECTRON/lib/datasets/data/coco/annotations

测试detectron：

python2 tools/infer_simple.py \
    --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \
    --output-dir /tmp/detectron-visualizations \
    --image-ext jpg \
    --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl \
    demo

Detectron应自动从--wts参数指定的URL下载模型

训练自己的数据

1、voc的xml格式转为coco的json

python tools/xml_to_json.py

需要修改 ：

xml_path = ''

json_file = ''

在lib/datasets/dataset_catalog.py 文件中添加你的数据集

2、下载model

我用的R-50.pkl

3、experiments文件夹下的修改yaml

需要修改的内容

NUM_CLASSES:
STEPS: 
WEIGHTS:
DATASETS:

不加FPN和FocalLoss

cd run_train

sh train_faster.sh

加FPN和不加FocalLoss

cd run_train

sh train_faster_fpn.sh

分析训练的Loss

训练输入的日志都存在了run_train下的log文件中

python tools/draw_loss_one.py

需要修改

log_path =''

img_path = ''

测试训练的模型

在对应的ymal文件中添加自己的测试数据集，注意测试集的名字不能带‘test’

测试单个模型

CUDA_VISIBLE_DEVICES=0 python2 tools/test_net.py --cfg experiments/faster_rcnn_R-50-FPN.yaml TEST.WEIGHTS output/train/yourdata/generalized_rcnn/model_final.pkl NUM_GPUS 1

批量测试模型

我设置了每迭代4000次保存一次模型，可以对保存的所有模型进行批量测试，结果保存在result.txt，选取效果最好的

python tools/test_list.py --model_root /path/to/model --yaml_path experiments/faster_rcnn_R-50-FPN.yaml --res_path /path/to/result.txt