slim.variance_scaling_initializer()

参考  slim.l2_regularizer() - 云+社区 - 腾讯云

参考 slim.variance_scaling_initializer() - 云+社区 - 腾讯云

def xavier_initializer(uniform=True, seed=None, dtype=dtypes.float32):
  """Returns an initializer performing "Xavier" initialization for weights.

  This function implements the weight initialization from:

  Xavier Glorot and Yoshua Bengio (2010):
           [Understanding the difficulty of training deep feedforward neural
           networks. International conference on artificial intelligence and
           statistics.](
           http://www.jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf)

  This initializer is designed to keep the scale of the gradients roughly the
  same in all layers. In uniform distribution this ends up being the range:
  `x = sqrt(6. / (in + out)); [-x, x]` and for normal distribution a standard
  deviation of `sqrt(2. / (in + out))` is used.

  Args:
    uniform: Whether to use uniform or normal distributed random initialization.
    seed: A Python integer. Used to create random seeds. See
          `tf.set_random_seed` for behavior.
    dtype: The data type. Only floating point types are supported.
  Returns:
    An initializer for a weight matrix.
  Raises:
    ValueError: if `dtype` is not a floating point type.
    TypeError: if `mode` is not in ['FAN_IN', 'FAN_OUT', 'FAN_AVG'].
  """
  if not dtype.is_floating:
    raise TypeError('Cannot create initializer for non-floating point type.')
  if mode not in ['FAN_IN', 'FAN_OUT', 'FAN_AVG']:
    raise TypeError('Unknown mode %s [FAN_IN, FAN_OUT, FAN_AVG]', mode)

返回对权重执行“Xavier”初始化的初始化器。此函数实现权重初始化，从:

Xavier Glorot和yobengio(2010):[了解深度前馈神经网络训练的难点]。(http://www.jmlr.org/programedings/papers/v9/glorot10a/glorot10a.pdf)

这个初始化器的设计目的是在所有层中保持梯度的比例大致相同。在均匀分布中，这个范围是' x = sqrt(6。/ (in + out);正态分布的标准差为√2。/ (in + out))’。

参数：

• factor:浮动。一个乘法因素
• mode:字符串。“FAN_IN”、“FAN_OUT’,‘FAN_AVG’
• uniform:是否使用均匀或正态分布随机初始化
• seed:一个Python整数。用于创建随机种子。看到“特遣部队。set_random_seed”行为
• dtype:数据类型。只支持浮点类型

返回值：

• 生成单位方差张量的初始化器

可能产生的异常：

ValueError: if `dtype` is not a floating point type.
TypeError: if `mode` is not in ['FAN_IN', 'FAN_OUT', 'FAN_AVG'].