本文主要借助代码讲解Xavier和kaiming是如何借助_calculate_fan_in_and_fan_out
函数来计算当前网络层的fan_in(输入神经元个数)和fan_out(输出神经元个数的),先针对Linear和Conv2d两种。
m_c = nn.Conv2d(16, 33, 3, stride=2)
m_l = nn.Linear(1, 10)
m_c.weight.size()
m_l.weight.size()
out:
torch.Size([33, 16, 3, 3])
torch.Size([10, 1])
注意看Linear weight的维度为2,而Conv2d的维度为4.
首先判断tensor的维度,如果是二维,则是Linear,
if dimensions == 2: # Linear
fan_in = tensor.size(1)
fan_out = tensor.size(0)
此时: f a n _ i n = i n _ c h a n n e l s fan\_in = in\_channels fan_in=in_channels
f a n _ o u t = o u t _ c h a n n e l s fan\_out = out\_channels fan_out=o