深度学习之:神经网络参数(权重)初始化

参数初始化

---

在我们真正开始训练网络之前，我们必须要知道如何去初始化神经网络的参数。

Training your neural network requires specifying an initial value of the weights.

A well chosen initialization method will help learning. 

 

 

我们先来看看三种不同的参数初始化方式:

1. Zero initialization : 全部参数都初始化为0
2. Random initialization : 初始化为随机数(没有控制大小)
3. He initialization 随机初始化了之后，乘以，这样就避免了参数的初始值过大或者过小，因此可以取得比较好的效果

[He initialization is named for the first author of He et al., 2015. If you have heard of "Xavier initialization", this is similar except Xavier initialization uses a scaling factor for the weights W[l]W[l] of sqrt(1./layers_dims[l-1]) where He initialization would use sqrt(2./layers_dims[l-1]). ]

 

三种初始化方式的代码如下:

def initialize_parameters_zeros(layers_dims):
    """
    Arguments:
    layer_dims -- python array (list) containing the size of each layer.
    
    Returns:
    parameters -- python dictionary containing your parameters "W1", "b1", ..., "WL", "bL":
                    W1 -- we