pytorch 初始化权重

一般的网络初始化方法：

from torch.nn import functional as F, init

   def init_params(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                init.kaiming_normal_(m.weight, mode='fan_out')
                if m.bias is not None:
                    init.constant_(m.bias, 0)
            elif isinstance(m, nn.BatchNorm2d):
                init.constant_(m.weight, 1)
                init.constant_(m.bias, 0)
            elif isinstance(m, nn.Linear):
                init.normal_(m.weight, std=0.001)
                if m.bias is not None:
                    init.constant_(m.bias, 0)

yolov5 head用了这个初始化，loss直接为nan了。

初始化方法
常数初始化

w = torch.empty(3, 5)
nn.init.constant_(w, 0.3)

均匀分布

torch.nn.init.uniform_(tensor, a=0, b=1)
# a是分布的下界&#x