Caffe用Python增加新层——数据读入层

有时需要根据自己的问题，为caffe添加新的数据层或损失层，那么可以用caffe中的python层来解决。

用Python为caffe增加新层，可分为两步：    

1. 用python定义新层的实现。
2. 在网络模型proto文件中写好对应的参数配置。

下面将举例介绍：

• 参考caffe-root\examples\pycaffe\layer\pascal_multilabel_datalayers.py中的例子。该文件中了定义一个数据层类PascalMultilabelDataLayerSync，用于读取Pascal VOC的多标签数据。这是pascal_multilabel_datalayers.py：

# imports
import json
import time
import pickle
import scipy.misc
import skimage.io
import caffe

import numpy as np
import os.path as osp

from xml.dom import minidom
from random import shuffle
from threading import Thread
from PIL import Image

from tools import SimpleTransformer


class PascalMultilabelDataLayerSync(caffe.Layer):

    """
    This is a simple synchronous datalayer for training a multilabel model on
    PASCAL.
    """

    def setup(self, bottom, top):

        self.top_names = ['data', 'label']

        # === Read input parameters ===

        # params is a python dictionary with layer parameters.
        params = eval(self.param_str)

        # Check the parameters for validity.
        check_params(params)

        # store input as class variables
        self.batch_size = params['batch_size']

        # Create a batch loader to load the images.
        self.batch_loader = BatchLoader(params, None)

        # === reshape tops ===
        # since we use a fixed input image size, we can shape the data layer
        # once. Else, we'd have to do it in the reshape call.
        top[0].reshape(
            self.batch_size, 3, params['im_shape'][0], params['im_shape'][1])
        # Note the 20 channels (because PASCAL has 20 classes.)
        top[1].reshape(self.batch_size, 20)

        print_info("PascalMultilabelDataLayerSync", params)

    def forward(self, bottom, top):
        """
        Load data.
        """
        for itt in range(self.batch_size):
            # Use the batch loader to load the next image.
            im, multilabel = self.batch_loader.load_next_image()

            # Add directly to the caffe data layer
            top[0].data[itt, ...] = im
            top[1].data[itt, ...] = multilabel

    def reshape(self, bottom, top):
        """
        There is no need to reshape the data, since the input is of fixed size
        (rows and columns)
        """
        pass

    def backward(self, top, propagate_down, bottom):
        """
        These layers does not back propagate
        """
        pass


class BatchLoader(object):

    """
    This class abstracts away the loading of images.
    Images can either be loaded singly, or in a batch. The latter is used for
    the asyncronous data layer to preload batches while other processing is
    performed.
    """

    def __init__(self, params, result):
        self.result = result
        self.batch_size = params['batch_size']
        self.pascal_root = params['pascal_root']
        self.im_shape = params['im_shape']
        # get list of image indexes.
        list_file = params['split'] + '.txt'
        self.indexlist = [line.rstrip('\n') for line in open(
            osp.join(self.pascal_root, 'ImageSets/Main', list_file))]
        self._cur = 0  # current image
        # this class does some simple data-manipulations
        self.transformer = SimpleTransformer()

        print "BatchLoader initialized with {} images".format(
            len(self.indexlist))

    def load_next_image(self):
        """
        Load the next image in a batch.
        """
        # Did we finish an epoch?
        if self._cur == len(self.indexlist):
            self._cur = 0
            shuffle(self.indexlist)

        # Load an image
        index = self.indexlist[self._cur]  # Get the image index
        image_file_name = index + '.jpg'
        im = np.asarray(Image.open(
            osp.join(self.pascal_root, 'JPEGImages', image_file_name)))
        im = scipy.misc.imresize(im, self.im_shape)  # resize

        # do a simple horizontal flip as data augmentation
        flip = np.random.choice(2)*2-1
        im = im[:, ::flip, :]

        # Load and prepare ground truth
        multilabel = np.zeros(20).astype(np.float32)
        anns = load_pascal_annotation(index, self.pascal_root)
        for label in anns['gt_classes']:
            # in the multilabel problem we don't care how MANY instances
            # there are of each class. Only if they are present.
            # The "-1" is b/c we are not interested in the background
            # class.
            multilabel[label - 1] = 1

        self._cur += 1
        return self.transformer.preprocess(im), multilabel


def load_pascal_annotation(index, pascal_root):
    """
    This code is borrowed from Ross Girshick's FAST-RCNN code
    (https://github.com/rbgirshick/fast-rcnn).
    It parses the PASCAL .xml metadata files.
    See publication for further details: (http://arxiv.org/abs/1504.08083).

    Thanks Ross!

    """
    classes = ('__background__',  # always index 0
               'aeroplane', 'bicycle', 'bird', 'boat',
               'bottle', 'bus', 'car', 'cat', 'chair',
                         'cow', 'diningtable', 'dog', 'horse',
                         'motorbike', 'person', 'pottedplant',
                         'sheep', 'sofa', 'train', 'tvmonitor')
    class_to_ind = dict(zip(classes, xrange(21)))

    filename = osp.join(pascal_root, 'Annotations', index + '.xml')
    # print 'Loading: {}'.format(filename)

    def get_data_from_tag(node, tag):
        return node.getElementsByTagName(tag)[0].childNodes[0].data

    with open(filename) as f:
        data = minidom.parseString(f.read())

    objs = data.getElementsByTagName('object')
    num_objs = len(objs)

    boxes = np.zeros((num_objs, 4), dtype=np.uint16)
    gt_classes = np.zeros((num_objs), dtype=np.int32)
    overlaps = np.zeros((num_objs, 21), dtype=np.float32)

    # Load object bounding boxes into a data frame.
    for ix, obj in enumerate(objs):
        # Make pixel indexes 0-based
        x1 = float(get_data_from_tag(obj, 'xmin')) - 1
        y1 = float(get_data_from_tag(obj, 'ymin')) - 1
        x2 = float(get_data_from_tag(obj, 'xmax')) - 1
        y2 = float(get_data_from_tag(obj, 'ymax')) - 1
        cls = class_to_ind[
            str(get_data_from_tag(obj, "name")).lower().strip()]
        boxes[ix, :] = [x1, y1, x2, y2]
        gt_classes[ix] = cls
        overlaps[ix, cls] = 1.0

    overlaps = scipy.sparse.csr_matrix(overlaps)

    return {'boxes': boxes,
            'gt_classes': gt_classes,
            'gt_overlaps': overlaps,
            'flipped': False,
            'index': index}


def check_params(params):
    """
    A utility function to check the parameters for the data layers.
    """
    assert 'split' in params.keys(
    ), 'Params must include split (train, val, or test).'

    required = ['batch_size', 'pascal_root', 'im_shape']
    for r in required:
        assert r in params.keys(), 'Params must include {}'.format(r)


def print_info(name, params):
    """
    Output some info regarding the class
    """
    print "{} initialized for split: {}, with bs: {}, im_shape: {}.".format(
        name,
        params['split'],
        params['batch_size'],
        params['im_shape'])

• 相应地，我们需要定义test.prototxt文件：（由于这里只是验证该方法的有效性，因此网络中只有一个data层）

name: "TEST-Net"
layer {
  name: "data"
  type: "Python"
  top: "data"
  top: "label"
  python_param{
    module: "pascal_multilabel_datalayers"
    layer: "PascalMultilabelDataLayerSync"
    param_str:'{"pascal_root": "path/to/your/pascal_datasets", "batch_size": 64, "split": "train", "im_shape": [227,227]}'
  }
}

这里应注意，层的type为“Python”；python_param中的module是定义该数据层实现的类所在的模块名称，也就是该py文件的文件名；python_param中的layer是新层的类名；param_str是在类的定义中会用到的一些参数（注意其书写格式）;其中的pascal_root应是你的pascal数据集的根目录。

• 接下来写一个脚本用于测试该网络是否可用，代码如下：

import caffe  

caffe.set_device(0)  
caffe.set_mode_gpu()  
  
print ("=================================")
net = caffe.Net('test.prototxt',caffe.TEST)  
net.forward()  

        使用TEST模式，直接运行。

• 打印信息如下，证明该过程成功：

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0622 23:05:16.634856  8284 common.cpp:36] System entropy source not available, using fallback algorithm to generate seed instead.
=================================
I0622 23:05:16.638833  8284 net.cpp:51] Initializing net from parameters:
name: "TEST-Net"
state {
  phase: TEST
  level: 0
}
layer {
  name: "data"
  type: "Python"
  top: "data"
  top: "label"
  python_param {
    module: "pascal_multilabel_datalayers"
    layer: "PascalMultilabelDataLayerSync"
    param_str: "{\"pascal_root\": \"D:/Kong_fei/Data/ObjectDetection/voc/VOCdevkit/VOC2007\", \"batch_size\": 64, \"split\": \"train\", \"im_shape\": [227,227]}"
  }
}
I0622 23:05:16.639876  8284 layer_factory.cpp:58] Creating layer data
I0622 23:05:16.646919  8284 net.cpp:84] Creating Layer data
I0622 23:05:16.648903  8284 net.cpp:380] data -> data
I0622 23:05:16.649906  8284 net.cpp:380] data -> label
BatchLoader initialized with 2501 images
PascalMultilabelDataLayerSync initialized for split: train, with bs: 64, im_shape: [227, 227].
I0622 23:05:16.688024  8284 net.cpp:122] Setting up data
I0622 23:05:16.690033  8284 net.cpp:129] Top shape: 64 3 227 227 (9893568)
I0622 23:05:16.691071  8284 net.cpp:129] Top shape: 64 20 (1280)
I0622 23:05:16.692045  8284 net.cpp:137] Memory required for data: 39579392
I0622 23:05:16.692045  8284 net.cpp:200] data does not need backward computation.
I0622 23:05:16.693953  8284 net.cpp:242] This network produces output data
I0622 23:05:16.693953  8284 net.cpp:242] This network produces output label
I0622 23:05:16.694959  8284 net.cpp:255] Network initialization done.

• 当然，还可以用相同的方法，定义loss层，参考caffe-root\examples\pycaffe\layer\pyloss.py。

参考博客：

https://blog.csdn.net/m0_37477175/article/details/78295072

https://www.cnblogs.com/fanhaha/p/7247839.html

https://blog.csdn.net/langb2014/article/details/53309618