Tensorflow ResnetV2代码解读

一，神经网络逐层加深有Degradiation问题，准确率先上升到饱和，再加深会下降，这不是过拟合，是测试集和训练集同时下降的。网络过深导致的梯度弥散或消失。为了解决深度网络的退化问题，何凯明等人提出了残差结构，这个结构解决了网络越深，训练误差反而提升的问题，使得网络理论上可以无限深。
二，残差网络的核心是bottleneck网络结构及恒等映射，注意Channel维度变化： ，宛如一个中间细两端粗的瓶颈，所以称为“bottleneck”。这种结构相比VGG，早已经被证明是非常效的，能够更好的提取图像特征。

@slim.add_arg_scope
def bottleneck(inputs, depth, depth_bottleneck, stride, rate=1,
               outputs_collections=None, scope=None):
  """Bottleneck residual unit variant with BN before convolutions.

  This is the full preactivation residual unit variant proposed in [2]. See
  Fig. 1(b) of [2] for its definition. Note that we use here the bottleneck
  variant which has an extra bottleneck layer.

  When putting together two consecutive ResNet blocks that use this unit, one
  should use stride = 2 in the last unit of the first block.

  Args:
    inputs: A tensor of size [batch, height, width, channels].
    depth: The depth of the ResNet unit output.
    depth_bottleneck: The depth of the bottleneck layers.
    stride: The ResNet unit's stride. Determines the amount of downsampling of
      the units output compared to its input.
    rate: An integer, rate for atrous convolution.
    outputs_collections: Collection to add the ResNet unit output.
    scope: Optional variable_scope.

  Returns:
    The ResNet unit's output.
  """
  """    
  核心残差学习单元
  输入tensor给出一个直连部分和残差部分加和的输出tensor
      :param inputs: 输入tensor
      :param depth: Block类参数，输出tensor通道
      :param depth_bottleneck: Block类参数，中间卷积层通道数 
      :param stride: Block类参数，降采样步长， 3个卷积只有中间层采用非1步长去降采样。    
      :param outputs_collections: 节点容器collection    
      :return: 输出tensor  
   """
  
  with tf.variable_scope(scope, 'bottleneck_v2', [inputs]) as sc:
    depth_in = slim.utils.last_dimension(inputs.get_shape(), min_rank=4)
    # 获取输入tensor的最后一个维度(通道)
    preact = slim.batch_norm(inputs, activation_fn=tf.nn.relu, scope='preact')
    # 对输入正则化处理，并激活，因在卷积之前，成为预激活
    
    if depth == depth_in:
      shortcut = resnet_utils.subsample(inputs, stride, 'shortcut')
      # 如果输入tensor通道数等于输出tensor通道数 
      # 降采样使输入tensor使之宽高等于输出tensor

    else:
      shortcut = slim.conv2d(preact, depth, [1, 1], stride=stride,
                             normalizer_fn=None, activation_fn=None,
                             scope='shortcut')
      # 否则，使用尺寸为1*1的卷积核改变其通道数,           
      # 同时调整宽高匹配输出tensor

    #'residual残差部分
    residual = slim.conv2d(preact, depth_bottleneck, [1, 1], stride=1,
                           scope='conv1')
    residual = resnet_utils.conv2d_same(residual, depth_bottleneck, 3, stride,
                                        rate=rate, scope='conv2')
    residual = slim.conv2d(residual, depth, [1, 1], stride=1,
                           normalizer_fn=None, activation_fn=None,
                           scope='conv3')

    output = shortcut + residual

    return slim.utils.collect_named_outputs(outputs_collections,
                                            sc.name,
                                            output)

总结：批正则化数据
shortcut分量处理：调整输入tensor使之和输出tensor深度一致，宽高一致
residual分量处理：11/1卷积->33/自定步长(所以上面需要调整shortcut宽高)卷积->1*1/1卷积
shortcut + residual 作为最终输出，注意是Add，不是concat。

class Block(collections.namedtuple('Block', ['scope', 'unit_fn', 'args'])):
  """A named tuple describing a ResNet block.

  Its parts are:
    scope: The scope of the `Block`.
    unit_fn: The ResNet unit function which takes as input a `Tensor` and
      returns another `Tensor` with the output of the ResNet unit.
    args: A list of length equal to the number of units in the `Block`. The list
      contains one (depth, depth_bottleneck, stride) tuple for each unit in the
      block to serve as argument to unit_fn.
  """
  """   
   使用collections.namedtuple设计ResNet基本模块组的name tuple，并用它创建Block的类    只包含数据结构，不包含具体方法。    
   定义一个典型的Block，需要输入三个参数：    
   scope：Block的名称    
   unit_fn：ResNet V2中的残差学习单元生成函数    
   args：Block的args（输出深度，瓶颈深度，瓶颈步长）    
  """




  def subsample(inputs, factor, scope=None):
  """Subsamples the input along the spatial dimensions.

  Args:
    inputs: A `Tensor` of size [batch, height_in