神经网络初始化参数

对神经网络初始化参数有助于帮助我们到的模型更好的收敛

比如我定义了如下一个模型：

net = nn.Sequential(nn.Conv2d(in_channels=1,out_channels=6,kernel_size=5,padding=2),
                    nn.Sigmoid(), nn.AvgPool2d(kernel_size=2,stride=2), nn.Conv2d(in_channels=6,out_channels=16,kernel_size=5),
                    nn.Sigmoid(), nn.AvgPool2d(2,2), nn.Flatten(), nn.Linear(16*5*5,120), nn.Sigmoid(),
                    nn.Linear(120,84), nn.Sigmoid(), nn.Linear(84,10))

对其中线性层和卷积层参数进行初始化：

print(type(net[7]))#net[7]是线性层
print(net[7].weight.data)#初始化前的参数
def init_weights(m):
    if type(m) in [nn.Linear, nn.Conv2d]:
        nn.init.xavier_uniform_(m.weight)
net.apply(init_weights)
print(net[7].weight.data)#初始化后的参数

输出结果：

<class 'torch.nn.modules.linear.Linear'>
tensor([[ 0.0077,  0.0050,  0.0297,  ..., -0.0043,  0.0328, -0.0255],
        [ 0.0390,  0.0207, -0.0395,  ..., -0.0479, -0.0427, -0.0108],
        [-0.0219, -0.0300,  0.0488,  ..., -0.0401,  0.0403,  0.0201],
        ...,
        [ 0.0101, -0.0425, -0.0184,  ...,  0.0345,  0.0317,  0.0052],
        [-0.0370,  0.0334,  0.0397,  ...,  0.0209,  0.0399, -0.0006],
        [-0.0335, -0.0332, -0.0322,  ..., -0.0132, -0.0330,  0.0154]])
tensor([[ 0.0980,  0.0989,  0.0994,  ...,  0.0469,  0.0446,  0.0048],
        [-0.0460,  0.1001,  0.0564,  ...,  0.0341,  0.0710,  0.0141],
        [ 0.0658, -0.0911,  0.0426,  ..., -0.0566, -0.0984, -0.0264],
        ...,
        [-0.0376,  0.0051, -0.0124,  ..., -0.0891, -0.0234, -0.0367],
        [ 0.0027,  0.0146, -0.0614,  ..., -0.0178,  0.0805,  0.0962],
        [ 0.0413, -0.0993, -0.0275,  ...,  0.0110,  0.0561, -0.0965]])