Caffe在Cifar10上复现ResNet

转载地址：http://blog.csdn.net/a_1937/article/details/51507545

ResNet在2015年的ImageNet竞赛上的识别率达到了非常高的水平，这里我将使用Caffe在Cifar10上复现论文4.2节的Cifar实验。

• ResNet的基本模块
• Caffe实现
• Cifar10上的实验结果及说明

---

ResNet的基本模块

本文参照Torch7在Cifar10上复现ResNet的实验，使用Caffe复现之。ResNet的基本模块可以如下Python代码:

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-decorator" style="color: rgb(0, 102, 102); box-sizing: border-box;">@requires_authorization</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> __future__ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> print_function
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> caffe <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> layers <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> L, params <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> P, to_proto
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> caffe.proto <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> caffe_pb2
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> caffe

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># helper function for building ResNet block structures </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># The function below does computations: bottom--->conv--->BatchNorm</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">conv_factory</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(bottom, ks, n_out, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, pad=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)</span>:</span>
    conv = L.Convolution(bottom, kernel_size=ks, stride=stride, num_output=n_out, pad=pad,
                         param = [dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)],
                         bias_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'constant'</span>, value=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'gaussian'</span>, std=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span>))
    batch_norm = L.BatchNorm(conv, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>,
                             param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)])
    scale = L.Scale(batch_norm, bias_term=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> scale

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># bottom--->conv--->BatchNorm--->ReLU</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">conv_factory_relu</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(bottom, ks, n_out, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, pad=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)</span>:</span>
    conv = L.Convolution(bottom, kernel_size=ks, stride=stride, num_output=n_out, pad=pad,
                         param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)],
                         bias_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'constant'</span>, value=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'gaussian'</span>, std=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span>))
    batch_norm = L.BatchNorm(conv, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>)])
    scale = L.Scale(batch_norm, bias_term=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
    relu = L.ReLU(scale, in_place=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> relu

 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  Residual building block! Implements option (A) from Section 3.3. The input</span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  is passed through two 3x3 convolution layers. Currently this block only supports </span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  stride == 1 or stride == 2. When stride is 2, the block actually does pooling.</span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  Instead of simply doing pooling which may cause representational bottlneck as</span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  described in inception v3, here we use 2 parallel branches P && C and add them</span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  together. Note pooling branch may has less channels than convolution branch so we</span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  need to do zero-padding along channel dimension. And to the best knowledge of</span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  ours, we haven't found current caffe implementation that supports this operation. </span>
 <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  So later I'll give implementation in C++ and CUDA.</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">residual_block</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(bottom, num_filters, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)</span>:</span>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> stride == <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>:
        conv1 = conv_factory_relu(bottom, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
        conv2 = conv_factory(conv1, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
        add = L.Eltwise(bottom, conv2, operation=P.Eltwise.SUM)
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> add
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">elif</span> stride == <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>:
        conv1 = conv_factory_relu(bottom, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
        conv2 = conv_factory(conv1, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_filters, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
        pool = L.Pooling(bottom, pool=P.Pooling.AVE, kernel_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, stride=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
        pad = L.PadChannel(pool, num_channels_to_pad=num_filters / <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
        add = L.Eltwise(conv2, pad, operation=P.Eltwise.SUM)
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> add
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span>:
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">raise</span> Exception(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'Currently, stride must be either 1 or 2.'</span>)

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Generate resnet cifar10 train && test prototxt. n_size control number of layers.</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># The total number of layers is  6 * n_size + 2. Here I don't know any of implementation </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># which can contain simultaneously TRAIN && TEST phase. </span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ==========================Note here==============================</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># !!! SO YOU have to include TRAIN && TEST by your own AFTER you use the script to generate the prototxt !!!</span>
<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">resnet_cifar</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(train_lmdb, test_lmdb, mean_file, batch_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>, n_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>)</span>:</span>
    data, label = L.Data(source=test_lmdb, backend=P.Data.LMDB, batch_size=batch_size, ntop=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>,
        transform_param=dict(mean_file=mean_file, crop_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">28</span>), include=dict(phase=getattr(caffe_pb2, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'TEST'</span>)))
    residual = conv_factory_relu(data, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># --------------> 16, 32, 32    1st group</span>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> xrange(n_size):
        residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>)

    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># --------------> 32, 16, 16    2nd group</span>
    residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> xrange(n_size - <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>):
        residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span>)

    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># --------------> 64, 8, 8      3rd group</span>
    residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> i <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> xrange(n_size - <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>):
        residual = residual_block(residual, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span>)

    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># -------------> end of residual</span>
    global_pool = L.Pooling(residual, pool=P.Pooling.AVE, global_pooling=<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">True</span>)
    fc = L.InnerProduct(global_pool, param=[dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>), dict(lr_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>, decay_mult=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)],num_output=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>,
                        bias_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'constant'</span>, value=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'gaussian'</span>, std=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.01</span>))
    loss = L.SoftmaxWithLoss(fc, label)
    acc = L.Accuracy(fc, label, include=dict(phase=getattr(caffe_pb2, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'TEST'</span>)))
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> to_proto(loss, acc)

<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">make_net</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(tgt_file)</span>:</span>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">with</span> open(tgt_file, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'w'</span>) <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> f:
        print(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'name: "resnet_cifar10"'</span>, file=f)
        print(resnet_cifar(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'dataset/cifar10_train_lmdb'</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'dataset/cifar10_test_lmdb'</span>,
                           <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'dataset/mean.proto'</span>, n_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span>), file=f)

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> __name__ == <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'__main__'</span>:
    tgt_file=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'D:/VSProjects/caffe/models/ucas_resnet_cifar10/res56_cifar_train_test.prototxt'</span>
    make_net(tgt_file)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li></ul>

Caffe实现

首先所有的卷积核采用的都是3x3大小，当stride=1时在ResNet一个block里面通过padding那么feature map的尺寸不变。这是输入可以和经过两个卷积的分支相加。当stride=2时，通过两个卷积的分支feature map的尺寸减半，输入可以通过pooling减半，变为和经过两个卷积分支的尺寸一样，但是可能会导致pooling分支的channel数目少于卷积分支的数目。有两种方式解决这个问题，如原文所描述的那样：1、通过1x1的卷积线性投影，不接任何激活函数，然后直接将卷积核个数设置到跟卷积分支输出的channels个数相等； 2：通过对channel维度进行加零填充，即zero-padding。这里我采用后者方式，下面给出后者的Caffe实现代码（也就是上面python代码中的L.PadChannel对应的C++代码）：

<code class="language-C++ hljs cpp has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#ifndef CAFFE_PAD_CHANNEL_LAYER_HPP_</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#define CAFFE_PAD_CHANNEL_LAYER_HPP_</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/blob.hpp"</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/layer.hpp"</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/proto/caffe.pb.h"</span>

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">namespace</span> caffe {

    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/*
    * @brief zero-padding channel to extend number of channels
    *
    * Note: Back-propagate just drop the pad derivatives
    */</span>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> PadChannelLayer : <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> Layer<Dtype> {
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span>:
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">explicit</span> PadChannelLayer(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> LayerParameter& param)
            : Layer<Dtype>(param) {}
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> LayerSetUp(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Reshape(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);

        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">char</span>* type() <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"PadChannel"</span>; }
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> ExactNumBottomBlobs() <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>; }
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">inline</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> ExactNumTopBlobs() <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>; }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span>:
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Forward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Backward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top,
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom);
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Forward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top);
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">virtual</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> Backward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top,
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom);
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num_channels_to_pad_;
    };

}  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// namespace caffe</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#endif  <span class="hljs-comment" style="box-sizing: border-box;">// CAFFE_PAD_CHANNEL_LAYER_HPP_</span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li></ul>

pad_channel_layer.cpp

<code class="hljs cpp has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/layers/pad_channel_layer.hpp"</span>

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">namespace</span> caffe {

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::LayerSetUp(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {
        CHECK_NE(top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>], bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]) << <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>->type() << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" Layer does not "</span>
            <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"allow in-place computation."</span>;
        num_channels_to_pad_ = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>->layer_param_.pad_channel_param().num_channels_to_pad();
        CHECK_GT(num_channels_to_pad_, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>) << <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"num channels to pad must greater than 0!"</span>;
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Reshape(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {

        <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span>></span> top_shape = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->shape();
        top_shape[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>] += num_channels_to_pad_;
        top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->Reshape(top_shape);
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Forward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* bottom_data = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->cpu_data();
        Dtype* top_data = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_cpu_data();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->num();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channel_by_dim = channels * dim;
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> n = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>; n < num; n++){
            caffe_copy(channel_by_dim, bottom_data, top_data);
            bottom_data += channel_by_dim;
            top_data += channel_by_dim;
            caffe_set(num_channels_to_pad_ * dim, Dtype(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>), top_data);
            top_data += num_channels_to_pad_ * dim;
        }
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Backward_cpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom, 
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top) {
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* top_diff = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->cpu_diff();
        Dtype* bottom_diff = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_cpu_diff();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->num();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> channel_by_dim = channels * dim;
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> n = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>; n < num; n++){ <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// just drop the padding derivatives part.</span>
            caffe_copy(channel_by_dim, top_diff, bottom_diff);
            top_diff += (channels + num_channels_to_pad_) * dim;
            bottom_diff += channel_by_dim;
        }
    }
    INSTANTIATE_CLASS(PadChannelLayer);
    REGISTER_LAYER_CLASS(PadChannel);

}  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// namespace caffe</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li></ul>

pad_channel_layer.cu

<code class="hljs cpp has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#include "caffe/layers/pad_channel_layer.hpp"</span>

<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">namespace</span> caffe {

    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// Copy (one line per thread) from one array to another, with arbitrary</span>
    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// strides in the last two dimensions.</span>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    __global__ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> pad_forward_kernel(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_count, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> src_channels, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_channels,
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim,  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* src, Dtype* dst) 
    {
        CUDA_KERNEL_LOOP(index, dst_count)
        {
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = index / (dim * dst_channels);
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_c = index / dim % dst_channels;
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> pixel_pos = index % dim;
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (dst_c < src_channels)
                dst[index] = src[num * src_channels * dim + dst_c * dim + pixel_pos];
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span>
                dst[index] = Dtype(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>);
        }
    }


    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Forward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom,
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top){
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* bottom_data = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->gpu_data();
        Dtype* top_data = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_gpu_data();

        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> src_channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_channels = src_channels + num_channels_to_pad_;
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dst_count = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->count();
        pad_forward_kernel<Dtype> << <CAFFE_GET_BLOCKS(dst_count), CAFFE_CUDA_NUM_THREADS >> >(
            dst_count, src_channels, dst_channels, dim, bottom_data, top_data);
        CUDA_POST_KERNEL_CHECK;
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    __global__ <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> pad_backward_kernel(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_count, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_channels, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> top_channels,
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* top, Dtype* bottom)
    {
        CUDA_KERNEL_LOOP(index, bottom_count)
        {
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> num = index / (dim * bottom_channels);
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_c = index / dim % bottom_channels;
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> pixel_pos = index % dim;
            bottom[index] = top[num * top_channels * dim + bottom_c * dim + pixel_pos];
        }
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">template</span> <<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">typename</span> Dtype>
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> PadChannelLayer<Dtype>::Backward_gpu(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& top,
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">bool</span>></span>& propagate_down, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> <span class="hljs-stl_container" style="box-sizing: border-box;"><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">vector</span><Blob<Dtype></span>*>& bottom) {
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">const</span> Dtype* top_diff = top[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->gpu_diff();
        Dtype* bottom_diff = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->mutable_gpu_diff();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_count = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->count();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> bottom_channels = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->channels();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> dim = bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->height() * bottom[<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]->width();
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> top_channels = bottom_channels + num_channels_to_pad_;
        pad_backward_kernel<Dtype> << <CAFFE_GET_BLOCKS(bottom_count), CAFFE_CUDA_NUM_THREADS >> >(
            bottom_count, bottom_channels, top_channels, dim, top_diff, bottom_diff);
        CUDA_POST_KERNEL_CHECK;
    }

    INSTANTIATE_LAYER_GPU_FUNCS(PadChannelLayer);

}  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// namespace caffe</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li></ul>

实验结果

1、 使用上述脚本针对Cifar10生成20层的resnet。网络定义太长，这里就不给出来了，但要注意的是因为原图是32x32大小的，这里随机crop 28x28的，并镜像一下。不同于原论文所述的在图片各边补4个像素再随机裁剪32x32的。网络定义的输入部分以及solver.prototxt：

<code class="hljs css has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">name</span>: "<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">resnet_cifar10</span>"
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">layer</span> <span class="hljs-rules" style="box-sizing: border-box;">{
  <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">name</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"input"</span>
  type: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data"</span>
  top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data1"</span>
  top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"label"</span>
  include {
    phase: TRAIN
  </span></span></span>}
  <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">transform_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
    <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">mean_file</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/mean.proto"</span>
    crop_size: <span class="hljs-number" style="box-sizing: border-box;">28</span>
    mirror: true
  </span></span></span>}
  <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">data_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
    <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">source</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/cifar10_train_lmdb"</span>
    batch_size: <span class="hljs-number" style="box-sizing: border-box;">100</span>
    backend: LMDB
  </span></span></span>}
}
<span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">layer</span> <span class="hljs-rules" style="box-sizing: border-box;">{
  <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">name</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"input"</span>
  type: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data"</span>
  top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Data1"</span>
  top: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"label"</span>
  include {
    phase: TEST
  </span></span></span>}
  <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">transform_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
    <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">mean_file</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/mean.proto"</span>
    crop_size: <span class="hljs-number" style="box-sizing: border-box;">28</span>
  </span></span></span>}
  <span class="hljs-tag" style="color: rgb(0, 0, 0); box-sizing: border-box;">data_param</span> <span class="hljs-rules" style="box-sizing: border-box;">{
    <span class="hljs-rule" style="box-sizing: border-box;"><span class="hljs-attribute" style="box-sizing: border-box;">source</span>:<span class="hljs-value" style="box-sizing: border-box; color: rgb(0, 102, 102);"> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"dataset/cifar10_test_lmdb"</span>
    batch_size: <span class="hljs-number" style="box-sizing: border-box;">100</span>
    backend: LMDB
  </span></span></span>}
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li></ul>

solver.prototxt如下

<code class="hljs http has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-attribute" style="box-sizing: border-box;">net</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"res20_cifar_train_test.prototxt"</span>
<span class="hljs-attribute" style="box-sizing: border-box;">test_iter</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100 # conver the whole test set. 100 * 100 = 10000 images.</span>
<span class="hljs-attribute" style="box-sizing: border-box;">test_interval</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">500 # Each 500 is one epoch, test after each epoch</span>
<span class="hljs-attribute" style="box-sizing: border-box;">base_lr</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.1</span>
<span class="hljs-attribute" style="box-sizing: border-box;">momentum</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.9</span>
<span class="hljs-attribute" style="box-sizing: border-box;">weight_decay</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.0001</span>
<span class="hljs-attribute" style="box-sizing: border-box;">average_loss</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100</span>
<span class="hljs-attribute" style="box-sizing: border-box;">lr_policy</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"multistep"</span>
<span class="hljs-attribute" style="box-sizing: border-box;">stepvalue</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">40000</span>
<span class="hljs-attribute" style="box-sizing: border-box;">stepvalue</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">80000</span>
<span class="hljs-attribute" style="box-sizing: border-box;">gamma</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">0.1</span>
<span class="hljs-attribute" style="box-sizing: border-box;">display</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100</span>
<span class="hljs-attribute" style="box-sizing: border-box;">max_iter</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">100000 # 100000 iteration is 200 epochs</span>
<span class="hljs-attribute" style="box-sizing: border-box;">snapshot</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">500</span>
<span class="hljs-attribute" style="box-sizing: border-box;">snapshot_prefix</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"trained_model/cifar"</span>
<span class="hljs-attribute" style="box-sizing: border-box;">solver_mode</span>: <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">GPU</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

2、 网络结构可视化 
由于网络结构太大，这里使用了Netscope对网络进行可视化，并截取部分网络结构图如下: 

 
3、 Cifar10上20层和56层的实验结果准确率对比

网络层数	本文复现结果	原论文结果
20	91.19%	91.25%
56	92.54%	93.03%

4、 Cifar10 20层的训练曲线以及测试曲线 
训练曲线图： 

测试曲线图： 

5、实验结论 
从第3届的表格可以看出，本文使用Caffe复现的resnet在Cifar10上的识别率非常接近论文结果。但比论文结果稍低，可能的原因有，本实验使用的是28x28的切图，原文使用的是32x32的切图, 另外，初始化也有所不同。但总的来说，可以认为复现了原文的结果。