resnet PyTorch官方实现
resnet 通过构造和一般网络单向传播的多层网络不同,resblock中添加了将输入直接”加“到输出的捷径。这样每层网络就只需要学习残差,这样就让特别深的神经网络成为了可能。
“We hypothesize that it is easier to optimize the residual mapping than to optimize the original, unreferenced mapping” —— authors
本文是笔者对于resnet的PyTorch官方实现的阅读笔记,可供参考。
准备工作
定义卷积核大小为3x3的二维卷积封装(通过封装可以屏蔽掉不需要的参数,比如说bias)
def conv3x3(in_planes, out_planes, stride=1, groups=1, dilation=1):
"""3x3 convolution with padding"""
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,
padding=dilation, groups=groups, bias=False, dilation=dilation)
定义卷积核大小为1的封装
def conv1x1(in_planes, out_planes, stride=1):
"""1x1 convolution"""
return nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=stride, bias=False)
基本块
在forward中可以看到将identity也就是这一个block自己的输入,加到了网络的输出上。而一个block是由两个卷积层构成的,都跟着batch Normalization,但是只有第一个在bn之后跟了relu非线性激活,第二个并没有。(第二个也加了,输出的大小就一定比indentity大,因为常规的relu是非负的)
残差的引入去掉了主体部分,从而突出了微小的变化
class BasicBlock(nn.Module):
expansion = 1
__constants__ = ['downsample']
def __init__(self, inplanes, planes, stride=1, downsample=None, groups=1,
base_width=64, dilation=1, norm_layer=None):
super(BasicBlock, self).__init__()
if norm_layer is None:
norm_layer = nn.BatchNorm2d
if groups != 1 or base_width != 64:
raise ValueError('BasicBlock only supports groups=1 and base_width=64')
if dilation > 1:
raise NotImplementedError("Dilation > 1 not supported in BasicBlock")
# Both self.conv1 and self.downsample layers downsample the input when stride != 1
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = norm_layer(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3