tf的初始化参数总结

来源：初始化参数

1. tf.glorot_normal_initializer

• 源码：

class GlorotNormal(VarianceScaling):
  """The Glorot normal initializer, also called Xavier normal initializer.

  It draws samples from a truncated normal distribution centered on 0
  with `stddev = sqrt(2 / (fan_in + fan_out))`
  where `fan_in` is the number of input units in the weight tensor
  and `fan_out` is the number of output units in the weight tensor.

  Args:
    seed: A Python integer. Used to create random seeds. See
      `tf.set_random_seed` for behavior.
    dtype: Default data type, used if no `dtype` argument is provided when
      calling the initializer. Only floating point types are supported.

  References:
      [Glorot et al., 2010](http://proceedings.mlr.press/v9/glorot10a.html)
      ([pdf](http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf))
  """

  def __init__(self, seed=None, dtype=dtypes.float32):
    super(GlorotNormal, self).__init__(
        scale=1.0,
        mode="fan_avg",
        distribution="truncated_normal",
        seed=seed,
        dtype=dtype)

  def get_config(self):
    return {"seed": self.seed, "dtype": self.dtype.name}

• 公式：

stddev = sqrt(2 / (fan_in + fan_out))

fan_in: 在权重张量中输入单元的个数，该层输入的节点数。

fan_out: 在权重张量中输出单元的个数，该层输出的节点数。

• 参数：

scale=1 : 放大倍数

mode=‘truncated_normal’ : 输入输出节点数的平均值

distribution='truncated_normal': 分布采用截断的正态分布，默认阶段为点为[-2σ,2σ]。数据出现在该部分的概率为95.4%。

seed=seed : 随机种子数，与python的随机种子数含义一致

dtype=dtype : 数据类型，只支持浮点型

to be continued...