torch.std计算标准差

通过查看pytorch手册，下式为pytorch中计算标准差的公式，其中δN为贝塞尔校正系数 

 

 pytorch默认使用贝塞尔校正系数计算标准，即torch.std(...,unbiased=True,...)

import torch
import torch.nn as nn
tensor = torch.FloatTensor([[1,2,4,1],
                            [6,3,2,4],
                            [2,4,6,1]])
layer_norm = nn.LayerNorm((3,4),elementwise_affine=False)
layer_norm(tensor)

eps = 1E-5
eps = torch.tensor(eps)
(tensor-tensor.mean().expand_as(tensor))/(tensor.std(unbiased=False).expand_as(tensor)+eps)