tf.random_normal与tf.truncated_normal的区别
-
定义
The generated values follow a normal distribution with specified mean and standard deviation, except that values whose magnitude is more than 2 standard deviations from the mean are dropped and re-picked. -
与tf.random_normal的不同之处
tf.random_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)outputs random values from a normal distribution.
tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)outputs random values from a truncated normal distribution.
- 价值在于:
The point for using truncated normal is to overcome saturation of tome functions like sigmoid (where if the value is too big/small, the neuron stops learning).
即为了克服神经网络在利用某些激活函数学习时,例如sidmoid,会在值过大活过小时停止学习的缺点。