GoogLeNet论文《Going deeper with convolutions》总结

最新推荐文章于 2024-05-06 16:06:08 发布
最新推荐文章于 2024-05-06 16:06:08 发布
阅读量537 收藏 7
点赞数
分类专栏： 深度学习用于目标检测论文
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/zj15939317693/article/details/77306086
版权

论文中文翻译：http://blog.csdn.net/motianchi/article/details/50970136

一、文章意义

       这是Google公司Inception系列的开山之作，在这篇文章中首次提出了Inception模块，后面的Inception v2&v3、Inception v4也是在这篇文章的基础上改进的。GoogLeNet是2014年ILSVRC挑战赛冠军，将Top5 的错误率降低到6.67%，是一个22层的深度网络。此外，作者团队的阵容也是吊炸天。

 

二、文章思路

        作者在文章的第三部分“Motivation and High Consideration”部分中提到了文章的写作动机及高层次的考虑设计。作者认为获得高质量模型的最直接方法是增加网络的深度（层数）和宽度（层核或神经元数）（“The most straightforward way of improving the performance of deep neural network is by increasing theri size”）,但是这样的设计思路会带来两个缺陷：
（1）“Bigger size typically means a large number of parameters,which makes the enlarged network more prone to overfitting”,随着层的增加和神经元个数的增加，参数数量肯定会增加，随之而来的就是很容易过拟合。
（2）“Another drawback of uniformly increased network size is the dramatically increased use of computational resources”,即层数和神经元个数的增加会带来更高计算资源的要求。
       针对这两个问题，作者认为根本的解决办法是把全连接层改成稀疏结构，甚至卷积层也是。但是，当涉及到大量非统一的稀疏的数据结构的计算时，现在的计算设施是很低效的。作者为了将网络向稀疏结构设计，同时又兼顾计算性能，作者根据“稀疏矩阵聚类为较为密集的子矩阵来提高计算性能”的理论，据此设计了名为Inception结构。
      注：文章的第三部分读的晕晕乎乎的啊，自己对“稀疏性”了解真的少，网上搜了一些，有人说是1*1卷积降维体现稀疏性，但个人感觉不仅仅是这个，希望了解的同学能给一些参考，谢谢！

 

三、文章结构

       分为两个部分：Inception结构和整体的GoogLeNet网络结构。

 

1、Inception结构

 

         图（b）在图（a）的基础上引入了1*1卷积降维（这就是Inception结构的稀疏性的体现？）。GoogLeNet并没有采用AlexNet的结构，而是用多尺度卷积（网络宽度提高）+1*1卷积降维+Pooling组成基本模块，整个网络也是Inception模块的堆叠，这是GoogLeNet网络的一大特色。

        卷积尺度包括5*5,3*3,1*1，通过多个不同的尺寸的卷积核可以提取图像中的不同特征，即增加了网络的宽度。通过不同的padding使不同尺寸的卷积核最终输出的特征图大小一致。最顶层的 concatenation层是来自不同卷积核的输出特征图的叠加。

2、GoogLeNet网络结构

       几点说明：

（1）每一个卷积层后面都跟着一个ReLUs层，图上省略掉了；

（2）在Inception（4a）和Inception（4d）的后面都跟着一个额外的Softmax层，用以减小“梯度消失”现象。这两个softmax的loss值最终以一个          0.3的权重连接到最后的softmax的loss值上，也有多模型融合的意思在里面，具体的链接为：Average Pooling+1*1卷积+ReLU+FC+ReLU+

         Dropout+FC+SoftmaxWithLoss，Dropout的系数为0.7，FC层据说是为了方便微调用的；

（3）在Inception（4a）和Inception（4d）后都接着一个Average Pooling层；

（4）最顶端的FC层有1024个单元；

       整个网络包含22层（不算池化层），是由Inception模块堆叠而成，其中的ReLU和Dropout图中省略了。除了最顶端的softmax层以外还有两个额外的softmax层，网络中穿插着Max Pooling和Average Pooling，细节网络可参考caffe里的GoogLeNet。

四、Caffe中的实现

       

name: "GoogleNet"
layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    mirror: true
    crop_size: 224
    mean_value: 104
    mean_value: 117
    mean_value: 123
  }
  data_param {
    source: "examples/imagenet/ilsvrc12_train_lmdb"
    batch_size: 32
    backend: LMDB
  }
}
layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TEST
  }
  transform_param {
    mirror: false
    crop_size: 224
    mean_value: 104
    mean_value: 117
    mean_value: 123
  }
  data_param {
    source: "examples/imagenet/ilsvrc12_val_lmdb"
    batch_size: 50
    backend: LMDB
  }
}
layer {
  name: "conv1/7x7_s2"
  type: "Convolution"
  bottom: "data"
  top: "conv1/7x7_s2"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    pad: 3
    kernel_size: 7
    stride: 2
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "conv1/relu_7x7"
  type: "ReLU"
  bottom: "conv1/7x7_s2"
  top: "conv1/7x7_s2"
}
layer {
  name: "pool1/3x3_s2"
  type: "Pooling"
  bottom: "conv1/7x7_s2"
  top: "pool1/3x3_s2"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "pool1/norm1"
  type: "LRN"
  bottom: "pool1/3x3_s2"
  top: "pool1/norm1"
  lrn_param {
    local_size: 5
    alpha: 0.0001
    beta: 0.75
  }
}
layer {
  name: "conv2/3x3_reduce"
  type: "Convolution"
  bottom: "pool1/norm1"
  top: "conv2/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "conv2/relu_3x3_reduce"
  type: "ReLU"
  bottom: "conv2/3x3_reduce"
  top: "conv2/3x3_reduce"
}
layer {
  name: "conv2/3x3"
  type: "Convolution"
  bottom: "conv2/3x3_reduce"
  top: "conv2/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 192
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "conv2/relu_3x3"
  type: "ReLU"
  bottom: "conv2/3x3"
  top: "conv2/3x3"
}
layer {
  name: "conv2/norm2"
  type: "LRN"
  bottom: "conv2/3x3"
  top: "conv2/norm2"
  lrn_param {
    local_size: 5
    alpha: 0.0001
    beta: 0.75
  }
}
layer {
  name: "pool2/3x3_s2"
  type: "Pooling"
  bottom: "conv2/norm2"
  top: "pool2/3x3_s2"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "inception_3a/1x1"
  type: "Convolution"
  bottom: "pool2/3x3_s2"
  top: "inception_3a/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3a/relu_1x1"
  type: "ReLU"
  bottom: "inception_3a/1x1"
  top: "inception_3a/1x1"
}
layer {
  name: "inception_3a/3x3_reduce"
  type: "Convolution"
  bottom: "pool2/3x3_s2"
  top: "inception_3a/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 96
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3a/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_3a/3x3_reduce"
  top: "inception_3a/3x3_reduce"
}
layer {
  name: "inception_3a/3x3"
  type: "Convolution"
  bottom: "inception_3a/3x3_reduce"
  top: "inception_3a/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3a/relu_3x3"
  type: "ReLU"
  bottom: "inception_3a/3x3"
  top: "inception_3a/3x3"
}
layer {
  name: "inception_3a/5x5_reduce"
  type: "Convolution"
  bottom: "pool2/3x3_s2"
  top: "inception_3a/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 16
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3a/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_3a/5x5_reduce"
  top: "inception_3a/5x5_reduce"
}
layer {
  name: "inception_3a/5x5"
  type: "Convolution"
  bottom: "inception_3a/5x5_reduce"
  top: "inception_3a/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 32
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3a/relu_5x5"
  type: "ReLU"
  bottom: "inception_3a/5x5"
  top: "inception_3a/5x5"
}
layer {
  name: "inception_3a/pool"
  type: "Pooling"
  bottom: "pool2/3x3_s2"
  top: "inception_3a/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_3a/pool_proj"
  type: "Convolution"
  bottom: "inception_3a/pool"
  top: "inception_3a/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 32
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3a/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_3a/pool_proj"
  top: "inception_3a/pool_proj"
}
layer {
  name: "inception_3a/output"
  type: "Concat"
  bottom: "inception_3a/1x1"
  bottom: "inception_3a/3x3"
  bottom: "inception_3a/5x5"
  bottom: "inception_3a/pool_proj"
  top: "inception_3a/output"
}
layer {
  name: "inception_3b/1x1"
  type: "Convolution"
  bottom: "inception_3a/output"
  top: "inception_3b/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3b/relu_1x1"
  type: "ReLU"
  bottom: "inception_3b/1x1"
  top: "inception_3b/1x1"
}
layer {
  name: "inception_3b/3x3_reduce"
  type: "Convolution"
  bottom: "inception_3a/output"
  top: "inception_3b/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3b/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_3b/3x3_reduce"
  top: "inception_3b/3x3_reduce"
}
layer {
  name: "inception_3b/3x3"
  type: "Convolution"
  bottom: "inception_3b/3x3_reduce"
  top: "inception_3b/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 192
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3b/relu_3x3"
  type: "ReLU"
  bottom: "inception_3b/3x3"
  top: "inception_3b/3x3"
}
layer {
  name: "inception_3b/5x5_reduce"
  type: "Convolution"
  bottom: "inception_3a/output"
  top: "inception_3b/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 32
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3b/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_3b/5x5_reduce"
  top: "inception_3b/5x5_reduce"
}
layer {
  name: "inception_3b/5x5"
  type: "Convolution"
  bottom: "inception_3b/5x5_reduce"
  top: "inception_3b/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 96
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3b/relu_5x5"
  type: "ReLU"
  bottom: "inception_3b/5x5"
  top: "inception_3b/5x5"
}
layer {
  name: "inception_3b/pool"
  type: "Pooling"
  bottom: "inception_3a/output"
  top: "inception_3b/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_3b/pool_proj"
  type: "Convolution"
  bottom: "inception_3b/pool"
  top: "inception_3b/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_3b/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_3b/pool_proj"
  top: "inception_3b/pool_proj"
}
layer {
  name: "inception_3b/output"
  type: "Concat"
  bottom: "inception_3b/1x1"
  bottom: "inception_3b/3x3"
  bottom: "inception_3b/5x5"
  bottom: "inception_3b/pool_proj"
  top: "inception_3b/output"
}
layer {
  name: "pool3/3x3_s2"
  type: "Pooling"
  bottom: "inception_3b/output"
  top: "pool3/3x3_s2"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "inception_4a/1x1"
  type: "Convolution"
  bottom: "pool3/3x3_s2"
  top: "inception_4a/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 192
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4a/relu_1x1"
  type: "ReLU"
  bottom: "inception_4a/1x1"
  top: "inception_4a/1x1"
}
layer {
  name: "inception_4a/3x3_reduce"
  type: "Convolution"
  bottom: "pool3/3x3_s2"
  top: "inception_4a/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 96
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4a/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_4a/3x3_reduce"
  top: "inception_4a/3x3_reduce"
}
layer {
  name: "inception_4a/3x3"
  type: "Convolution"
  bottom: "inception_4a/3x3_reduce"
  top: "inception_4a/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 208
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4a/relu_3x3"
  type: "ReLU"
  bottom: "inception_4a/3x3"
  top: "inception_4a/3x3"
}
layer {
  name: "inception_4a/5x5_reduce"
  type: "Convolution"
  bottom: "pool3/3x3_s2"
  top: "inception_4a/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 16
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4a/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_4a/5x5_reduce"
  top: "inception_4a/5x5_reduce"
}
layer {
  name: "inception_4a/5x5"
  type: "Convolution"
  bottom: "inception_4a/5x5_reduce"
  top: "inception_4a/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 48
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4a/relu_5x5"
  type: "ReLU"
  bottom: "inception_4a/5x5"
  top: "inception_4a/5x5"
}
layer {
  name: "inception_4a/pool"
  type: "Pooling"
  bottom: "pool3/3x3_s2"
  top: "inception_4a/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_4a/pool_proj"
  type: "Convolution"
  bottom: "inception_4a/pool"
  top: "inception_4a/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4a/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_4a/pool_proj"
  top: "inception_4a/pool_proj"
}
layer {
  name: "inception_4a/output"
  type: "Concat"
  bottom: "inception_4a/1x1"
  bottom: "inception_4a/3x3"
  bottom: "inception_4a/5x5"
  bottom: "inception_4a/pool_proj"
  top: "inception_4a/output"
}
layer {
  name: "loss1/ave_pool"
  type: "Pooling"
  bottom: "inception_4a/output"
  top: "loss1/ave_pool"
  pooling_param {
    pool: AVE
    kernel_size: 5
    stride: 3
  }
}
layer {
  name: "loss1/conv"
  type: "Convolution"
  bottom: "loss1/ave_pool"
  top: "loss1/conv"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "loss1/relu_conv"
  type: "ReLU"
  bottom: "loss1/conv"
  top: "loss1/conv"
}
layer {
  name: "loss1/fc"
  type: "InnerProduct"
  bottom: "loss1/conv"
  top: "loss1/fc"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 1024
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "loss1/relu_fc"
  type: "ReLU"
  bottom: "loss1/fc"
  top: "loss1/fc"
}
layer {
  name: "loss1/drop_fc"
  type: "Dropout"
  bottom: "loss1/fc"
  top: "loss1/fc"
  dropout_param {
    dropout_ratio: 0.7
  }
}
layer {
  name: "loss1/classifier"
  type: "InnerProduct"
  bottom: "loss1/fc"
  top: "loss1/classifier"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 1000
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "loss1/loss"
  type: "SoftmaxWithLoss"
  bottom: "loss1/classifier"
  bottom: "label"
  top: "loss1/loss1"
  loss_weight: 0.3
}
layer {
  name: "loss1/top-1"
  type: "Accuracy"
  bottom: "loss1/classifier"
  bottom: "label"
  top: "loss1/top-1"
  include {
    phase: TEST
  }
}
layer {
  name: "loss1/top-5"
  type: "Accuracy"
  bottom: "loss1/classifier"
  bottom: "label"
  top: "loss1/top-5"
  include {
    phase: TEST
  }
  accuracy_param {
    top_k: 5
  }
}
layer {
  name: "inception_4b/1x1"
  type: "Convolution"
  bottom: "inception_4a/output"
  top: "inception_4b/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 160
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4b/relu_1x1"
  type: "ReLU"
  bottom: "inception_4b/1x1"
  top: "inception_4b/1x1"
}
layer {
  name: "inception_4b/3x3_reduce"
  type: "Convolution"
  bottom: "inception_4a/output"
  top: "inception_4b/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 112
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4b/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_4b/3x3_reduce"
  top: "inception_4b/3x3_reduce"
}
layer {
  name: "inception_4b/3x3"
  type: "Convolution"
  bottom: "inception_4b/3x3_reduce"
  top: "inception_4b/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 224
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4b/relu_3x3"
  type: "ReLU"
  bottom: "inception_4b/3x3"
  top: "inception_4b/3x3"
}
layer {
  name: "inception_4b/5x5_reduce"
  type: "Convolution"
  bottom: "inception_4a/output"
  top: "inception_4b/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 24
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4b/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_4b/5x5_reduce"
  top: "inception_4b/5x5_reduce"
}
layer {
  name: "inception_4b/5x5"
  type: "Convolution"
  bottom: "inception_4b/5x5_reduce"
  top: "inception_4b/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4b/relu_5x5"
  type: "ReLU"
  bottom: "inception_4b/5x5"
  top: "inception_4b/5x5"
}
layer {
  name: "inception_4b/pool"
  type: "Pooling"
  bottom: "inception_4a/output"
  top: "inception_4b/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_4b/pool_proj"
  type: "Convolution"
  bottom: "inception_4b/pool"
  top: "inception_4b/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4b/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_4b/pool_proj"
  top: "inception_4b/pool_proj"
}
layer {
  name: "inception_4b/output"
  type: "Concat"
  bottom: "inception_4b/1x1"
  bottom: "inception_4b/3x3"
  bottom: "inception_4b/5x5"
  bottom: "inception_4b/pool_proj"
  top: "inception_4b/output"
}
layer {
  name: "inception_4c/1x1"
  type: "Convolution"
  bottom: "inception_4b/output"
  top: "inception_4c/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4c/relu_1x1"
  type: "ReLU"
  bottom: "inception_4c/1x1"
  top: "inception_4c/1x1"
}
layer {
  name: "inception_4c/3x3_reduce"
  type: "Convolution"
  bottom: "inception_4b/output"
  top: "inception_4c/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4c/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_4c/3x3_reduce"
  top: "inception_4c/3x3_reduce"
}
layer {
  name: "inception_4c/3x3"
  type: "Convolution"
  bottom: "inception_4c/3x3_reduce"
  top: "inception_4c/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 256
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4c/relu_3x3"
  type: "ReLU"
  bottom: "inception_4c/3x3"
  top: "inception_4c/3x3"
}
layer {
  name: "inception_4c/5x5_reduce"
  type: "Convolution"
  bottom: "inception_4b/output"
  top: "inception_4c/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 24
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4c/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_4c/5x5_reduce"
  top: "inception_4c/5x5_reduce"
}
layer {
  name: "inception_4c/5x5"
  type: "Convolution"
  bottom: "inception_4c/5x5_reduce"
  top: "inception_4c/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4c/relu_5x5"
  type: "ReLU"
  bottom: "inception_4c/5x5"
  top: "inception_4c/5x5"
}
layer {
  name: "inception_4c/pool"
  type: "Pooling"
  bottom: "inception_4b/output"
  top: "inception_4c/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_4c/pool_proj"
  type: "Convolution"
  bottom: "inception_4c/pool"
  top: "inception_4c/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4c/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_4c/pool_proj"
  top: "inception_4c/pool_proj"
}
layer {
  name: "inception_4c/output"
  type: "Concat"
  bottom: "inception_4c/1x1"
  bottom: "inception_4c/3x3"
  bottom: "inception_4c/5x5"
  bottom: "inception_4c/pool_proj"
  top: "inception_4c/output"
}
layer {
  name: "inception_4d/1x1"
  type: "Convolution"
  bottom: "inception_4c/output"
  top: "inception_4d/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 112
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4d/relu_1x1"
  type: "ReLU"
  bottom: "inception_4d/1x1"
  top: "inception_4d/1x1"
}
layer {
  name: "inception_4d/3x3_reduce"
  type: "Convolution"
  bottom: "inception_4c/output"
  top: "inception_4d/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 144
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4d/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_4d/3x3_reduce"
  top: "inception_4d/3x3_reduce"
}
layer {
  name: "inception_4d/3x3"
  type: "Convolution"
  bottom: "inception_4d/3x3_reduce"
  top: "inception_4d/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 288
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4d/relu_3x3"
  type: "ReLU"
  bottom: "inception_4d/3x3"
  top: "inception_4d/3x3"
}
layer {
  name: "inception_4d/5x5_reduce"
  type: "Convolution"
  bottom: "inception_4c/output"
  top: "inception_4d/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 32
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4d/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_4d/5x5_reduce"
  top: "inception_4d/5x5_reduce"
}
layer {
  name: "inception_4d/5x5"
  type: "Convolution"
  bottom: "inception_4d/5x5_reduce"
  top: "inception_4d/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4d/relu_5x5"
  type: "ReLU"
  bottom: "inception_4d/5x5"
  top: "inception_4d/5x5"
}
layer {
  name: "inception_4d/pool"
  type: "Pooling"
  bottom: "inception_4c/output"
  top: "inception_4d/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_4d/pool_proj"
  type: "Convolution"
  bottom: "inception_4d/pool"
  top: "inception_4d/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 64
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4d/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_4d/pool_proj"
  top: "inception_4d/pool_proj"
}
layer {
  name: "inception_4d/output"
  type: "Concat"
  bottom: "inception_4d/1x1"
  bottom: "inception_4d/3x3"
  bottom: "inception_4d/5x5"
  bottom: "inception_4d/pool_proj"
  top: "inception_4d/output"
}
layer {
  name: "loss2/ave_pool"
  type: "Pooling"
  bottom: "inception_4d/output"
  top: "loss2/ave_pool"
  pooling_param {
    pool: AVE
    kernel_size: 5
    stride: 3
  }
}
layer {
  name: "loss2/conv"
  type: "Convolution"
  bottom: "loss2/ave_pool"
  top: "loss2/conv"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "loss2/relu_conv"
  type: "ReLU"
  bottom: "loss2/conv"
  top: "loss2/conv"
}
layer {
  name: "loss2/fc"
  type: "InnerProduct"
  bottom: "loss2/conv"
  top: "loss2/fc"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 1024
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "loss2/relu_fc"
  type: "ReLU"
  bottom: "loss2/fc"
  top: "loss2/fc"
}
layer {
  name: "loss2/drop_fc"
  type: "Dropout"
  bottom: "loss2/fc"
  top: "loss2/fc"
  dropout_param {
    dropout_ratio: 0.7
  }
}
layer {
  name: "loss2/classifier"
  type: "InnerProduct"
  bottom: "loss2/fc"
  top: "loss2/classifier"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 1000
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "loss2/loss"
  type: "SoftmaxWithLoss"
  bottom: "loss2/classifier"
  bottom: "label"
  top: "loss2/loss1"
  loss_weight: 0.3
}
layer {
  name: "loss2/top-1"
  type: "Accuracy"
  bottom: "loss2/classifier"
  bottom: "label"
  top: "loss2/top-1"
  include {
    phase: TEST
  }
}
layer {
  name: "loss2/top-5"
  type: "Accuracy"
  bottom: "loss2/classifier"
  bottom: "label"
  top: "loss2/top-5"
  include {
    phase: TEST
  }
  accuracy_param {
    top_k: 5
  }
}
layer {
  name: "inception_4e/1x1"
  type: "Convolution"
  bottom: "inception_4d/output"
  top: "inception_4e/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 256
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4e/relu_1x1"
  type: "ReLU"
  bottom: "inception_4e/1x1"
  top: "inception_4e/1x1"
}
layer {
  name: "inception_4e/3x3_reduce"
  type: "Convolution"
  bottom: "inception_4d/output"
  top: "inception_4e/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 160
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4e/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_4e/3x3_reduce"
  top: "inception_4e/3x3_reduce"
}
layer {
  name: "inception_4e/3x3"
  type: "Convolution"
  bottom: "inception_4e/3x3_reduce"
  top: "inception_4e/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 320
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4e/relu_3x3"
  type: "ReLU"
  bottom: "inception_4e/3x3"
  top: "inception_4e/3x3"
}
layer {
  name: "inception_4e/5x5_reduce"
  type: "Convolution"
  bottom: "inception_4d/output"
  top: "inception_4e/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 32
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4e/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_4e/5x5_reduce"
  top: "inception_4e/5x5_reduce"
}
layer {
  name: "inception_4e/5x5"
  type: "Convolution"
  bottom: "inception_4e/5x5_reduce"
  top: "inception_4e/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4e/relu_5x5"
  type: "ReLU"
  bottom: "inception_4e/5x5"
  top: "inception_4e/5x5"
}
layer {
  name: "inception_4e/pool"
  type: "Pooling"
  bottom: "inception_4d/output"
  top: "inception_4e/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_4e/pool_proj"
  type: "Convolution"
  bottom: "inception_4e/pool"
  top: "inception_4e/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_4e/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_4e/pool_proj"
  top: "inception_4e/pool_proj"
}
layer {
  name: "inception_4e/output"
  type: "Concat"
  bottom: "inception_4e/1x1"
  bottom: "inception_4e/3x3"
  bottom: "inception_4e/5x5"
  bottom: "inception_4e/pool_proj"
  top: "inception_4e/output"
}
layer {
  name: "pool4/3x3_s2"
  type: "Pooling"
  bottom: "inception_4e/output"
  top: "pool4/3x3_s2"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "inception_5a/1x1"
  type: "Convolution"
  bottom: "pool4/3x3_s2"
  top: "inception_5a/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 256
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5a/relu_1x1"
  type: "ReLU"
  bottom: "inception_5a/1x1"
  top: "inception_5a/1x1"
}
layer {
  name: "inception_5a/3x3_reduce"
  type: "Convolution"
  bottom: "pool4/3x3_s2"
  top: "inception_5a/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 160
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5a/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_5a/3x3_reduce"
  top: "inception_5a/3x3_reduce"
}
layer {
  name: "inception_5a/3x3"
  type: "Convolution"
  bottom: "inception_5a/3x3_reduce"
  top: "inception_5a/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 320
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5a/relu_3x3"
  type: "ReLU"
  bottom: "inception_5a/3x3"
  top: "inception_5a/3x3"
}
layer {
  name: "inception_5a/5x5_reduce"
  type: "Convolution"
  bottom: "pool4/3x3_s2"
  top: "inception_5a/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 32
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5a/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_5a/5x5_reduce"
  top: "inception_5a/5x5_reduce"
}
layer {
  name: "inception_5a/5x5"
  type: "Convolution"
  bottom: "inception_5a/5x5_reduce"
  top: "inception_5a/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5a/relu_5x5"
  type: "ReLU"
  bottom: "inception_5a/5x5"
  top: "inception_5a/5x5"
}
layer {
  name: "inception_5a/pool"
  type: "Pooling"
  bottom: "pool4/3x3_s2"
  top: "inception_5a/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_5a/pool_proj"
  type: "Convolution"
  bottom: "inception_5a/pool"
  top: "inception_5a/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5a/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_5a/pool_proj"
  top: "inception_5a/pool_proj"
}
layer {
  name: "inception_5a/output"
  type: "Concat"
  bottom: "inception_5a/1x1"
  bottom: "inception_5a/3x3"
  bottom: "inception_5a/5x5"
  bottom: "inception_5a/pool_proj"
  top: "inception_5a/output"
}
layer {
  name: "inception_5b/1x1"
  type: "Convolution"
  bottom: "inception_5a/output"
  top: "inception_5b/1x1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 384
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5b/relu_1x1"
  type: "ReLU"
  bottom: "inception_5b/1x1"
  top: "inception_5b/1x1"
}
layer {
  name: "inception_5b/3x3_reduce"
  type: "Convolution"
  bottom: "inception_5a/output"
  top: "inception_5b/3x3_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 192
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5b/relu_3x3_reduce"
  type: "ReLU"
  bottom: "inception_5b/3x3_reduce"
  top: "inception_5b/3x3_reduce"
}
layer {
  name: "inception_5b/3x3"
  type: "Convolution"
  bottom: "inception_5b/3x3_reduce"
  top: "inception_5b/3x3"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 384
    pad: 1
    kernel_size: 3
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5b/relu_3x3"
  type: "ReLU"
  bottom: "inception_5b/3x3"
  top: "inception_5b/3x3"
}
layer {
  name: "inception_5b/5x5_reduce"
  type: "Convolution"
  bottom: "inception_5a/output"
  top: "inception_5b/5x5_reduce"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 48
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5b/relu_5x5_reduce"
  type: "ReLU"
  bottom: "inception_5b/5x5_reduce"
  top: "inception_5b/5x5_reduce"
}
layer {
  name: "inception_5b/5x5"
  type: "Convolution"
  bottom: "inception_5b/5x5_reduce"
  top: "inception_5b/5x5"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    pad: 2
    kernel_size: 5
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5b/relu_5x5"
  type: "ReLU"
  bottom: "inception_5b/5x5"
  top: "inception_5b/5x5"
}
layer {
  name: "inception_5b/pool"
  type: "Pooling"
  bottom: "inception_5a/output"
  top: "inception_5b/pool"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 1
    pad: 1
  }
}
layer {
  name: "inception_5b/pool_proj"
  type: "Convolution"
  bottom: "inception_5b/pool"
  top: "inception_5b/pool_proj"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 128
    kernel_size: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0.2
    }
  }
}
layer {
  name: "inception_5b/relu_pool_proj"
  type: "ReLU"
  bottom: "inception_5b/pool_proj"
  top: "inception_5b/pool_proj"
}
layer {
  name: "inception_5b/output"
  type: "Concat"
  bottom: "inception_5b/1x1"
  bottom: "inception_5b/3x3"
  bottom: "inception_5b/5x5"
  bottom: "inception_5b/pool_proj"
  top: "inception_5b/output"
}
layer {
  name: "pool5/7x7_s1"
  type: "Pooling"
  bottom: "inception_5b/output"
  top: "pool5/7x7_s1"
  pooling_param {
    pool: AVE
    kernel_size: 7
    stride: 1
  }
}
layer {
  name: "pool5/drop_7x7_s1"
  type: "Dropout"
  bottom: "pool5/7x7_s1"
  top: "pool5/7x7_s1"
  dropout_param {
    dropout_ratio: 0.4
  }
}
layer {
  name: "loss3/classifier"
  type: "InnerProduct"
  bottom: "pool5/7x7_s1"
  top: "loss3/classifier"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 1000
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "loss3/loss3"
  type: "SoftmaxWithLoss"
  bottom: "loss3/classifier"
  bottom: "label"
  top: "loss3/loss3"
  loss_weight: 1
}
layer {
  name: "loss3/top-1"
  type: "Accuracy"
  bottom: "loss3/classifier"
  bottom: "label"
  top: "loss3/top-1"
  include {
    phase: TEST
  }
}
layer {
  name: "loss3/top-5"
  type: "Accuracy"
  bottom: "loss3/classifier"
  bottom: "label"
  top: "loss3/top-5"
  include {
    phase: TEST
  }
  accuracy_param {
    top_k: 5
  }
}

 

五、训练测试过程

       训练是参照DisBelief【1】，使用异步随机梯度下降法，动量固定为0.9，学习率设置为每迭代8个epoches下降4%。图像采样方法在比赛中一直在变，没有详细的参照方法，但有一个小技巧被证实很有效：采样的图像块大小占原图的8%到100%，长宽比在3/4和4/3中随机选择，光尺度变化（photometric distortions）和若干差值方法也被使用。

        在最后的测试过程中，作者用了三个tricks:

（1）融合了7个GoogLeNet版本的模型。这7个模型的权值初始化方式和学习率设置都一样，不同的是采样方法和图片输入顺序；

（2）在测试阶段，将测试图片重采样为256*256, 288*288, 320*320, 352*352 4个尺寸，分别取他们的左、中、右三个正方形部分（肖像图取上中下          三部分）。每一个正方形部分取四个角和中间的224*224部分，并将该正方形图像块重采样为224*224的大小，并取他们的镜像。这样一副测试          图像就可以扩充为4*3*6*2=144张测试图像块；

（3）最后模型融合时采用softmax输出平均的方法

 

六、实验结果

        这里只列出分类的结果，检测的结果可以参考原文。

Table 3可以看出融合的模型越多，测试时裁剪的图像块越多，最后的测试结果越好。

 

参考论文：

【1】Ng A. Large Scale Distributed Deep Networks.[J]. Advances in Neural Information Processing Systems, 2012:1223-1231.

参考博客：

1、http://blog.csdn.net/shuzfan/article/details/50738394

2、http://blog.csdn.net/lynnandwei/article/details/44458033

3、http://www.cnblogs.com/Allen-rg/p/5833919.html

4、http://blog.csdn.net/whiteinblue/article/details/43635575/

5、http://www.eepw.com.cn/article/201705/359304.htm

 

博客中哪里写的不对的，欢迎批评指正。

Main Theme
关注
  • 0
    点赞
  • 踩
  • 7
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
经典神经网络论文超详细解读（三）——GoogLeNet InceptionV1学习笔记（翻译＋精读+代码复现）
路人贾的博客
12-11 6649
GoogLeNet InceptionV1论文（Going deeper with convolutions）超详细解读。翻译＋总结。文末有代码复现
论文整理：GoogLeNet--Going deeper with convolutions
qq_52053775的博客
04-30 2996
1.摘要 我们提出了一种代号为“GoogleNet”的深度卷积神经网络架构，它负责在2014年ImageNet大规模视觉识别挑战(ILSVRC14)中设置分类和检测的新技术状态。该体系结构的主要特点是提高了网络内部计算资源的利用率。这是通过一个精心设计的设计来实现的，它允许增加网络的深度和宽度，同时保持计算预算不变。为了优化质量，体系结构的决策是基于赫布原则和多尺度处理的直觉。在我们提交给ILSVRC14时使用的一个特殊化身是GoogLeNet，这是一个22层的深度网络，其质量......
GoogleNet论文翻译和解析
最新发布
l_1037947629的博客
05-06 592
我们设计了一种深度卷积网络结构，命名为Inception，它用来在2014 年 ILSVRC14 大赛中设置分类和检测的新技术状态。这个结构主要特点是提高了网络内部计算机资源的利用率。这一点通过一个巧妙地设计来实现，该设计允许提高网络深度和规模的同时保持计算预算不变。为了优化质量，这个结构基于Hebbian原理和多尺度处理的直觉感知。注：Hebbian原理：一个神经科学理论，解释了在学习的过程中脑中的神经元所发生的变化，不用过多了解。
GoogLeNet论文解读
David Lee的博客
02-24 4928
GoogLeNet论文翻译及解读
论文精读之--GoogLeNet
weixin_43584866的博客
01-12 1464
GoogLeNet是2014年Google团队提出的，其中L为大写，目的是为了致敬LeNet。论文的多处命名来自于电影盗梦空间。GoogLeNet主要的模型是Inception（中文翻译为盗梦空间）。论文题目为《Going deeper with convolutions》，来自于该图片：GoogLNet当年取得了ILSVRC 2014年分类冠军，检测冠军以及定位亚军。开启了多尺度，1*1卷积时代，以及衍生了GooLeNet v2 v3 v4系列。
GoogLeNet系列论文
qq_38156104的博客
07-21 2864
GoogLeNet Incepetion V1 这是GoogLeNet的最早版本，出现在2014年的《Going deeper with convolutions》。之所以名为“GoogLeNet”而非“GoogleNet”,文章说是为了向早期的LeNet致敬。 Motivation 深度学习以及神经网络快速发展，人们不再只关注更给力的硬件、更大的数据集、更大的模型，而是更在意新的idea、新的算法以及模型的改进。 一般来说，提升网络性能最直接的办法就是增加网络深度和宽度，即增加隐层和以及各层神经元
Going deeper with convolutions.pdf
02-03
GoogLeNet Inception V1了论文 《 Going deeper with convolutions 》
【6】Going deeper with convolutions.pdf
08-26
We propose a deep convolutional neural network architecture codenamed Inception ...GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.
Going Deeper with Convolutions
04-23
GoogLeNet是谷歌团队为了参加ILSVRC 2014比赛而精心准备的，为了达到最佳的性能，除了使用inception等网络结构外，还做了大量的辅助工作：包括训练多个model求平均、裁剪不同尺度的图像做多次验证等等。详细的这些...
1409.4842-GoogLeNet-inception-V1-Going deeper with convolutions.pdf
09-09
inception-V1 论文 1409.4842-GoogLeNet-inception-V1-Going deeper with convolutions.pdf
googLeNet 深度学习四篇论文
09-09
1. Inception-v1（1409.4842 - Going Deeper with Convolutions）：这是GoogLeNet系列的开创之作，由Szegedy等人在2014年提出。该模型的主要创新是引入了“ inception module”（又称Inception结构），它通过并行...
GoogLeNet论文原文.pdf
12-29
GoogLeNet论文原文下载
goolenet论文（4篇）
12-04
[v1] Going Deeper with Convolutions, 6.67% test error, http://arxiv.org/abs/1409.4842 [v2] Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, 4.8% test error, http://arxiv.org/abs/1502.03167 [v3] Rethinking the Inception Architecture for Computer Vision, 3.5% test error, http://arxiv.org/abs/1512.00567 [v4] Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning, 3.08% test error, http://arxiv.org/abs/1602.07261
GoogleNet经典论文4篇
11-26
欢迎想入门CNN和深度学习的朋友们阅读论文。 GoogleNet始于LeNet-5，一个有着标准的堆叠式卷积层冰带有一个或多个全连接层的结构的卷积神经网络。通常使用dropout来针对过拟合问题。  为了提出一个更深的网络，GoogLeNet做到了22层，利用inception结构，这个结构很好地利用了网络中的计算资源，并且在不增加计算负载的情况下，增加网络的宽度和深度。同时，为了优化网络质量，采用了Hebbian原理和多尺度处理。GoogLeNet在分类和检测上都取得了不错的效果。  最近深度学习的发展，大多来源于新的想法，算法以及网络结构的改善，而不是依赖于硬件，新的数据集，更深的网络，并且深度学习的研究不应该完全专注于精确度的问题上，而更应该关注与网络结构的改善方面的工作。
CNN经典网络模型之GoogleNet论文解读
qq_43649937的博客
08-09 2809
深度学习
【深度学习经典论文翻译2】GoogLeNet-Going Deeper with Convolutions全文翻译
热门推荐
可多的Academic笔记本
03-24 2万+
本文是经典深度学习论文GoogLeNet的全文翻译，论文原文和中英对照翻译欢迎点击：http://pan.baidu.com/s/1gedzApl 下载
Going Deeper with Convolutions学习笔记
MasicMcsu的博客
04-18 3333
摘要 本模型的主要特色是改进了对计算资源的利用，在同等计算负担下，增加了网络的深度和宽度。 在ILSVRC14数据集上进行测试的，采用了22层深度网络。 相关工作 对于像Imagenet大规模数据集，当前的趋势是增加神经网络的深度和宽度，使用dropout解决过拟合的问题。 Lin etal提出的Network-in-Network（M. Lin, Q. Chen,
《Going Deeper With Convolution》学习笔记
纸上君
11-30 3839
最近在学深度学习，所以把这篇论文简单翻译了一下，有些地方是结合自己的学习和体会进行翻译的，若有纰漏，望请指正。同时感谢袁冠达同学的校验。   摘要 我们在ILSVRC14上提交了一个代号为Inception的深度卷积神经网络架构。这个架构的主要特点是提高网络中计算资源的利用率。在保持计算量不变的前提下，通过细致的设计来提高网络的深度和广度。为了优化性能，这个架构是基于赫布学习规则（Hebbi
深度提升：GoogLeNet论文详解与深度学习优化策略
"《Going deeper with convolutions》论文PPT深入探讨了一种创新的深度学习网络结构，其核心在于如何在保持计算资源需求不变的前提下，通过改进网络的宽度和深度来提升性能。论文的主要贡献包括： 1. 减少参数量与...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值