记住在PyTorch中构建模型后初始化权重

我尝试过不初始化__init__函数中的权重，会发生什么。结果是网络无法收敛。因此，有必要在构造函数中初始化您的模型。下面是一个例子。

这里我构建一个简单的模型.

from torch import nn

class simpleModel(nn.Module):
    def __init__(self):
        super(simpleModel,self).__init__()
        self.conv1 = nn.Conv2d(3,32,3,1,0)
        self.conv2 = nn.Conv2d(32,64,3,2,1)
        layers = [
            nn.AvgPool2d(7,7,0),
            nn.Linear(64,10)
        ]
        self.layers = nn.Sequential(*layers)

    def forward(self, batch):
        out = self.conv1(batch)
        out = self.conv2(batch)
        out = self.layers(out)
        return out

model = simpleModel()
print(model)

'''
simpleModel(
  (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1))
  (conv2): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
  (layers): Sequential(
    (0): AvgPool2d(kernel_size=7, stride=7, padding=0)
    (1): Linear(in_features=64, out_features=10, bias=True)
  )
)
'''

note that I don't initialize weight. now I do it .

      #init
        for m in self.modules():
            if isinstance(m,nn.Conv2d):
                m.weight.data.normal_(0,0.02)
                m.bias.data.fill_(0)
            if isinstance(m,nn.Linear):
                m.weight.data.normal_(0,0.02)
                m.bias.data.fill_(0)

add this paragraph befor forward function

another lime light(notes) is that all parameter must be produced in construction function.