(训练一)王朗自然保护区---目标检测数据集介绍 & balancap/ssd训练介绍

１数据集准备

1 王朗DIY

2 这个数据集要放在ssd目录下：
/home/hp/z../Tensorflow/Deeplearning/SSD-Tensorflow-master
我们使用的tf框架下的ssd是github上面balancap/SSD-Tensorflow版本: https://github.com/balancap/SSD-Tensorflow

3 关于训练集的结构如下
Anotations:      自己制作xml
ImageSets:     里面有三个文件夹Layout(空文件夹), Main（自己制作Ｍain）, Segmentation(空文件夹)
　　　　　　 自己制作Ｍain/test.tst, Ｍain/train.txt, Ｍain/trianval.txt, Ｍain/val.txt
JEPGImages: 文件夹里面存放有原图片文件.jpg
SegmentationClass: 空文件夹
SegmentationObject: 空文件夹

1)其中Annotations文件里面存储的是xml文件
(使用labelme工具进行标注得到，可以参考https://blog.csdn.net/zjc910997316/article/details/82993420但是这个写的并不好）

2)ImageSet文件夹里面三个文件夹 Layout(空文件夹), Main（自己制作Ｍain）, Segmentation(空文件夹)

Main文件里面四个.txt文件Ｍain/test.tst, Ｍain/train.txt, Ｍain/trianval.txt, Ｍain/val.txt

3)JEPGImages文件夹里面存放有原图片文件.jpg

4)SegmentationClass: 空文件夹

5)SegmentationObject: 空文件夹

２用于训练

 

Training

The script train_ssd_network.py  is in charged of training the network.
脚本 train_ssd_network.py 负责网络培训。
Similarly to TF-Slim models, one can pass numerous options to the training process
(dataset, optimiser, hyper-parameters, model, ...).
与TF-Slim模型类似，可以在训练过程中传递许多选项(数据集，优化器，超参数，模型，…)
In particular, it is possible to provide a checkpoint file which can be use as starting point in order to fine-tune a network.
特别是，可以提供检查点文件，可以将其用作微调网络的起点。

Fine-tuning existing SSD checkpoints

The easiest way to fine the SSD model is to use as pre-trained SSD network (VGG-300 or VGG-512).
For instance, one can fine a model starting from the former as following:

微调现有的SSD检查点
优化SSD模型最简单的方法是使用预先训练的SSD网络(VGG-300或VGG-512)。
例如，可以从前者开始细化模型如下:

DATASET_DIR=./tfrecords
TRAIN_DIR=./logs/
CHECKPOINT_PATH=./checkpoints/ssd_300_vgg.ckpt
python train_ssd_network.py \
    --train_dir=${TRAIN_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2012 \
    --dataset_split_name=train \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${CHECKPOINT_PATH} \
    --save_summaries_secs=60 \
    --save_interval_secs=600 \
    --weight_decay=0.0005 \
    --optimizer=adam \
    --learning_rate=0.001 \
    --batch_size=32

 

Note that in addition to the training script flags, one may also want to experiment with data augmentation parameters
(random cropping, resolution, ...) in ssd_vgg_preprocessing.py
or/and network parameters (feature layers, anchors boxes, ...) in ssd_vgg_300/512.py

Furthermore, the training script can be combined with the evaluation routine in order to monitor the performance of saved checkpoints on a validation dataset.
For that purpose, one can pass to training and validation scripts a GPU memory upper limit such that both can run in parallel on the same device.
If some GPU memory is available for the evaluation script, the former can be run in parallel as follows:

在ssd_vgg_preprocessing.py , 注意，除了训练脚本标志之外，还可能需要试验数据增强参数 (随机裁剪，分辨率，…)
或者/和 ssd_vgg_300/512.py 中的网络参数(特性层、锚框等)

此外，可以将训练脚本与评估例程结合使用，以监视在验证数据集中保存的检查点的性能。
为此，可以将GPU内存上限传递给训练和验证脚本，这样两者就可以在同一设备上并行运行。
如果计算脚本有GPU内存可用，则可以并行运行GPU内存:

EVAL_DIR=${TRAIN_DIR}/eval
python eval_ssd_network.py \
    --eval_dir=${EVAL_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2007 \
    --dataset_split_name=test \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${TRAIN_DIR} \
    --wait_for_checkpoints=True \
    --batch_size=1 \
    --max_num_batches=500

Fine-tuning a network trained on ImageNet

One can also try to build a new SSD model based on standard architecture (VGG, ResNet, Inception, ...) and set up on top of it the multibox layers (with specific anchors, ratios, ...).
For that purpose, you can fine-tune a network by only loading the weights of the original architecture, and initialize randomly the rest of network.
For instance, in the case of the VGG-16 architecture, one can train a new model as following:

微调一个基于ImageNet的网络

您还可以尝试基于标准体系结构(VGG、ResNet、Inception，……)构建一个新的SSD模型，并在其之上设置多盒层(使用特定的锚、比率，……)。
为此，您可以通过只加载原始体系结构的权重来微调网络，并随机初始化网络的其余部分。
例如，在VGG-16架构的情况下，可以训练一个新的模型如下:

DATASET_DIR=./tfrecords
TRAIN_DIR=./log/
CHECKPOINT_PATH=./checkpoints/vgg_16.ckpt
python train_ssd_network.py \
    --train_dir=${TRAIN_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2007 \
    --dataset_split_name=train \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${CHECKPOINT_PATH} \
    --checkpoint_model_scope=vgg_16 \
    --checkpoint_exclude_scopes=ssd_300_vgg/conv6,ssd_300_vgg/conv7,ssd_300_vgg/block8,ssd_300_vgg/block9,ssd_300_vgg/block10,ssd_300_vgg/block11,ssd_300_vgg/block4_box,ssd_300_vgg/block7_box,ssd_300_vgg/block8_box,ssd_300_vgg/block9_box,ssd_300_vgg/block10_box,ssd_300_vgg/block11_box \
    --trainable_scopes=ssd_300_vgg/conv6,ssd_300_vgg/conv7,ssd_300_vgg/block8,ssd_300_vgg/block9,ssd_300_vgg/block10,ssd_300_vgg/block11,ssd_300_vgg/block4_box,ssd_300_vgg/block7_box,ssd_300_vgg/block8_box,ssd_300_vgg/block9_box,ssd_300_vgg/block10_box,ssd_300_vgg/block11_box \
    --save_summaries_secs=60 \
    --save_interval_secs=600 \
    --weight_decay=0.0005 \
    --optimizer=adam \
    --learning_rate=0.001 \
    --learning_rate_decay_factor=0.94 \
    --batch_size=32

Hence, in the former command, the training script randomly initializes the weights belonging to the checkpoint_exclude_scopes and load from the checkpoint file vgg_16.ckpt the remaining part of the network.

因此，在前一个命令中，训练脚本随机初始化属于checkpoint_exclude_scope的权重，并从检查点文件vgg_16.ckpt加载网络的其余部分。

Note that we also specify with the trainable_scopes parameter to first only train the new SSD components and left the rest of VGG network unchanged.

注意，我们还使用 trainable_scope 参数指定，首先只训练新的SSD组件，其余的VGG网络保持不变。
--------------------------------------------------------------------------------------------------------------------------------------------

Once the network has converged to a good first result (~0.5 mAP for instance), you can fine-tuned the complete network as following:

一旦网络收敛到良好的第一个结果(例如~0.5 mAP)，您可以对整个网络进行微调，如下所示:

DATASET_DIR=./tfrecords
TRAIN_DIR=./log_finetune/
CHECKPOINT_PATH=./log/model.ckpt-N
python train_ssd_network.py \
    --train_dir=${TRAIN_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2007 \
    --dataset_split_name=train \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${CHECKPOINT_PATH} \
    --checkpoint_model_scope=vgg_16 \
    --save_summaries_secs=60 \
    --save_interval_secs=600 \
    --weight_decay=0.0005 \
    --optimizer=adam \
    --learning_rate=0.00001 \
    --learning_rate_decay_factor=0.94 \
    --batch_size=32

A number of pre-trained weights of popular deep architectures can be found on TF-Slim models page.

在TF-Slim模型页面上可以找到一些流行的深度架构的预训练权重。