python图像分割 框架_Keras-FCN

Keras-FCN

Fully convolutional networks and semantic segmentation with Keras.

Models

Models are found in models.py, and include ResNet and DenseNet based models. AtrousFCN_Resnet50_16s is the current best performer, with pixel mean Intersection over Union mIoU 0.661076, and pixel accuracy around 0.9 on the augmented Pascal VOC2012 dataset detailed below.

Install

Useful setup scripts for Ubuntu 14.04 and 16.04 can be found in the robotics_setup repository. First use that to install CUDA, TensorFlow,

mkdir -p ~/src

cd ~/src

# install dependencies

pip install pillow keras sacred

# fork of keras-contrib necessary for DenseNet based models

git clone git@github.com:ahundt/keras-contrib.git -b densenet-atrous

cd keras-contrib

sudo python setup.py install

# Install python coco tools

cd ~/src

git clone https://github.com/pdollar/coco.git

cd coco

sudo python setup.py install

cd ~/src

git clone https://github.com/aurora95/Keras-FCN.git

Datasets

Datasets can be downloaded and configured in an automated fashion via the ahundt-keras branch on a fork of the tf_image_segmentation repository.

For simplicity, the instructions below assume all repositories are in ~/src/, and datasets are downloaded to ~/.keras/ by default.

cd ~/src

git clone git@github.com:ahundt/tf-image-segmentation.git -b Keras-FCN

Pascal VOC + Berkeley Data Augmentation

Pascal VOC 2012 augmented with Berkeley Semantic Contours is the primary dataset used for training Keras-FCN. Note that the default configuration maximizes the size of the dataset, and will not in a form that can be submitted to the pascal VOC2012 segmentation results leader board, details are below.

# Automated Pascal VOC Setup (recommended)

export PYTHONPATH=$PYTHONPATH:~/src/tf-image-segmentation

cd path/to/tf-image-segmentation/tf_image_segmentation/recipes/pascal_voc/

python data_pascal_voc.py pascal_voc_setup

This downloads and configures image/annotation filenames pairs train/val splits from combined Pascal VOC with train and validation split respectively that has

image full filename/ annotation full filename pairs in each of the that were derived

from PASCAL and PASCAL Berkeley Augmented dataset.

The datasets can be downloaded manually as follows:

# Manual Pascal VOC Download (not required)

# original PASCAL VOC 2012

wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar # 2 GB

# berkeley augmented Pascal VOC

wget http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/semantic_contours/benchmark.tgz # 1.3 GB

The setup utility has three type of train/val splits(credit matconvnet-fcn):

Let BT, BV, PT, PV, and PX be the Berkeley training and validation

sets and PASCAL segmentation challenge training, validation, and

test sets. Let T, V, X the final trainig, validation, and test

sets.

Mode 1::

V = PV (same validation set as PASCAL)

Mode 2:: (default))

V = PV \ BT (PASCAL val set that is not a Berkeley training

image)

Mode 3::

V = PV \ (BV + BT)

In all cases:

S = PT + PV + BT + BV

X = PX (the test set is uncahgend)

T = (S \ V) \ X (the rest is training material)

MS COCO

MS COCO support is very experimental, contributions would be highly appreciated.

Note that there any pixel can have multiple classes, for example a pixel which is point on a cup on a table will be classified as both cup and table, but sometimes the z-ordering is wrong in the dataset. This means saving the classes as an image will result in very poor performance.

export PYTHONPATH=$PYTHONPATH:~/src/tf-image-segmentation

cd ~/src/tf-image-segmentation/tf_image_segmentation/recipes/mscoco

# Initial download is 13 GB

# Extracted 91 class segmentation encoding

# npy matrix files may require up to 1TB

python data_coco.py coco_setup

python data_coco.py coco_to_pascal_voc_imageset_txt

python data_coco.py coco_image_segmentation_stats

# Train on coco

cd ~/src/Keras-FCN

python train_coco.py

Training and testing

The default configuration trains and evaluates AtrousFCN_Resnet50_16s on pascal voc 2012 with berkeley data augmentation.

cd ~/src/Keras-FCN

cd utils

# Generate pretrained weights

python transfer_FCN.py

cd ~/src/Keras-FCN

# Run training

python train.py

# Evaluate the performance of the network

python evaluate.py

Model weights will be in ~/src/Keras-FCN/Models, along with saved image segmentation results from the validation dataset.

Key files

model.py

contains model definitions, you can use existing models or you can define your own one.

train.py

The training script. Most parameters are set in the main function, and data augmentation parameters are where SegDataGenerator is initialized, you may change them according to your needs.

inference.py

Used for infering segmentation results. It can be directly run and it's also called in evaluate.py

evaluate.py

Used for evaluating perforance. It will save all segmentation results as images and calculate IOU. Outputs are not perfectly formatted so you may need to look into the code to see the meaning.

Most parameters of train.py, inference.py, and evaluate.py are set in the main function.