对神经网络初始化参数有助于帮助我们到的模型更好的收敛
比如我定义了如下一个模型:
net = nn.Sequential(nn.Conv2d(in_channels=1,out_channels=6,kernel_size=5,padding=2),
nn.Sigmoid(), nn.AvgPool2d(kernel_size=2,stride=2), nn.Conv2d(in_channels=6,out_channels=16,kernel_size=5),
nn.Sigmoid(), nn.AvgPool2d(2,2), nn.Flatten(), nn.Linear(16*5*5,120), nn.Sigmoid(),
nn.Linear(120,84), nn.Sigmoid(), nn.Linear(84,10))
对其中线性层和卷积层参数进行初始化:
print(type(net[7]))#net[7]是线性层
print(net[7].weight.data)#初始化前的参数
def init_weights(m):
if type(m) in [nn.Linear, nn.Conv2d]:
nn.init.xavier_uniform_(m.weight)
net.apply(init_weights)
print(net[7].weight.data)#初始化后的参数
输出结果:
<class 'torch.nn.modules.linear.Linear'>
tensor([[ 0.0077, 0.0050, 0.0297, ..., -0.0043, 0.0328, -0.0255],
[ 0.0390, 0.0207, -0.0395, ..., -0.0479, -0.0427, -0.0108],
[-0.0219, -0.0300, 0.0488, ..., -0.0401, 0.0403, 0.0201],
...,
[ 0.0101, -0.0425, -0.0184, ..., 0.0345, 0.0317, 0.0052],
[-0.0370, 0.0334, 0.0397, ..., 0.0209, 0.0399, -0.0006],
[-0.0335, -0.0332, -0.0322, ..., -0.0132, -0.0330, 0.0154]])
tensor([[ 0.0980, 0.0989, 0.0994, ..., 0.0469, 0.0446, 0.0048],
[-0.0460, 0.1001, 0.0564, ..., 0.0341, 0.0710, 0.0141],
[ 0.0658, -0.0911, 0.0426, ..., -0.0566, -0.0984, -0.0264],
...,
[-0.0376, 0.0051, -0.0124, ..., -0.0891, -0.0234, -0.0367],
[ 0.0027, 0.0146, -0.0614, ..., -0.0178, 0.0805, 0.0962],
[ 0.0413, -0.0993, -0.0275, ..., 0.0110, 0.0561, -0.0965]])