Pytorch学习笔记(三):nn.BatchNorm2d()函数详解

相关文章

Pytorch学习笔记(一):torch.cat()模块的详解
Pytorch学习笔记(二):nn.Conv2d()函数详解
Pytorch学习笔记(三):nn.BatchNorm2d()函数详解
Pytorch学习笔记(四):nn.MaxPool2d()函数详解
Pytorch学习笔记(五):nn.AdaptiveAvgPool2d()函数详解
Pytorch学习笔记(六):view()和nn.Linear()函数详解
Pytorch学习笔记(七):F.softmax()和F.log_softmax函数详解

1.函数语法格式和作用

作用:卷积层之后总会添加BatchNorm2d进行数据的归一化处理，这使得数据在进行Relu之前不会因为数据过大而导致网络性能的不稳定

torch.nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

2.参数解释

• num_features：一般输入参数为batch_size×num_features×height×width，即为其中特征的数量

• eps：分母中添加的一个值，目的是为了计算的稳定性，默认为：1e-5

• momentum：一个用于运行过程中均值和方差的一个估计参数（我的理解是一个稳定系数，类似于SGD中的momentum的系数）

• affine：当设为true时，会给定可以学习的系数矩阵gamma和beta

3.具体代码

# encoding:utf-8
import torch
import torch.nn as nn

# num_features - num_features from an expected input of size:batch_size*num_features*height*width
# eps:default:1e-5 (公式中为数值稳定性加到分母上的值)
# momentum:动量参数，用于running_mean and running_var计算的值，default：0.1
m = nn.BatchNorm2d(3)  # affine参数设为True表示weight和bias将被使用
m1 = nn.BatchNorm2d(3, affine=False)  # affine参数设为True表示weight和bias将被使用
input = torch.randn(2, 3, 2, 3)
output = m(input)
output1 = m1(input)
print('"""affine=True"""')
print(input)
print(m.weight)
print(m.bias)
print(output)
print(output.size())
print('"""affine=False"""')
print(output1)
print(output1.size())

结果如下

"""affine=True"""
tensor([[[[ 0.5408,  0.2707, -0.4395],
          [ 0.7942, -1.3403,  0.9146]],

         [[ 0.0082,  0.3639, -0.1986],
          [ 1.6522, -0.3494, -0.8619]],

         [[ 0.1021,  0.2455,  0.9168],
          [-0.2652,  0.0869, -1.3121]]],


        [[[-0.5038, -1.0989,  1.3820],
          [ 1.5612, -0.0384, -1.5507]],

         [[-0.4546,  2.5124, -1.1012],
          [ 1.0045, -0.7018,  1.3485]],

         [[-2.7837, -0.6371, -0.7099],
          [-0.0732,  1.1424,  0.6456]]]])
Parameter containing:
tensor([1., 1., 1.], requires_grad=True)
Parameter containing:
tensor([0., 0., 0.], requires_grad=True)
tensor([[[[ 0.4995,  0.2295, -0.4802],
          [ 0.7527, -1.3803,  0.8730]],

         [[-0.2414,  0.0885, -0.4332],
          [ 1.2832, -0.5730, -1.0483]],

         [[ 0.3156,  0.4560,  1.1133],
          [-0.0441,  0.3006, -1.0692]]],


        [[[-0.5444, -1.1390,  1.3400],
          [ 1.5191, -0.0794, -1.5906]],

         [[-0.6706,  2.0809, -1.2702],
          [ 0.6825, -0.8999,  1.0016]],

         [[-2.5102, -0.4082, -0.4795],
          [ 0.1439,  1.3342,  0.8478]]]], grad_fn=<NativeBatchNormBackward>)
torch.Size([2, 3, 2, 3])
"""affine=False"""
tensor([[[[ 0.4995,  0.2295, -0.4802],
          [ 0.7527, -1.3803,  0.8730]],

         [[-0.2414,  0.0885, -0.4332],
          [ 1.2832, -0.5730, -1.0483]],

         [[ 0.3156,  0.4560,  1.1133],
          [-0.0441,  0.3006, -1.0692]]],


        [[[-0.5444, -1.1390,  1.3400],
          [ 1.5191, -0.0794, -1.5906]],

         [[-0.6706,  2.0809, -1.2702],
          [ 0.6825, -0.8999,  1.0016]],

         [[-2.5102, -0.4082, -0.4795],
          [ 0.1439,  1.3342,  0.8478]]]])
torch.Size([2, 3, 2, 3])

Process finished with exit code 0