1. 内容来源
2. Pytorch实现
2.1 Inception块
本系列中LeNet、AlexNet、VGG、NiN中交替选用了1*1/3*3/5*5卷积核以及各种池化层。为了减少使用大核卷积层带来参数量大的问题,以及提高网络多样性,如上图所示,GoogLeNet使用Inception块,采用多支路的方式集成不同模块。
Inception块结构特点:
- 支路一,使用1*1卷积层,提取通道信息
- 支路二,使用1*1卷积层降低通道数,接3*3卷积层提取空间信息并提高通道数
- 支路三,使用1*1卷积层降低通道数,接5*5卷积层提取空间信息并提高通道数
- 支路四,使用3*3最大池化提取空间信息,接1*1卷积层降低通道数
- 最后将四个支路的所有特征图在通道数维度进行拼接
从代码的角度,每一个Inception块需要(输入通道数,各个支路模块输出通道数)
代码实现:
import torch
from torch import nn
from torch.nn import functional as F
from d2l import torch as d2l
class Inception(nn.Module):
def __init__(self, in_channels, c1, c2, c3, c4, **kwargs):
super(Inception, self).__init__(**kwargs)
self.p1_1 = nn.Conv2d(in_channels, c1, kernel_size=1)
self.p2_1 = nn.Conv2d(in_channels, c2[0], kernel_size=1)
self.p2_2 = nn.Conv2d(c2[0], c2[1], kernel_size=3, padding=1)
self.p3_1 = nn.Conv2d(in_channels, c3[0], kernel_size=1)
self.p3_2 = nn.Conv2d(c3[0], c3[1], kernel_size=5, padding=2)
self.p4_1 = nn.MaxPool2d(kernel_size=3, stride=1, padding=1)
self.p4_2 = nn.Conv2d(in_channels, c4, kernel_size=1)
def forward(self, x):
p1 = F.relu(self.p1_1(x))
p2 = F.relu(self.p2_2(F.relu(self.p2_1(x))))
p3 = F.relu(self.p3_2(F.relu(self.p3_1(x))))
p4 = F.relu(self.p4_2(self.p4_1(x)))
return torch.cat((p1, p2, p3, p4), dim=1)
其中,卷积层和池化层的步长、填充都是为了让不同支路(下文简称P)的特征图大小保证一致,从而可以在通道数进行拼接
2.2 GoogLeNet构建
如上图所示为GoogLeNet网络架构,每一个Stage表示将输入特征图长宽减半。
GoogLeNet网络详细结构:
Stage1
- 卷积层1(输入通道1,输出通道64,7*7核,2步长,1填充)
- ReLU
- 最大池化层1(3*3核,2步长,1填充)
Stage2
- 卷积层1(输入通道64,输出通道64,1*1核,0步长,0填充)
- ReLU
- 卷积层2(输入通道64,输出通道192,3*3核,0步长,1填充)
- 最大池化层1(3*3核,2步长,1填充)
Stage3
- Inception块1(输入通道192,P1 64通道,P2(96-128)通道,P3(16-32),P4 32通道)
- Inception块2(输入通道256,P1 128通道,P2(128-192)通道,P3(32-96),P4 64通道)
- 最大池化层1(3*3核,2步长,1填充)
Stage4
- Inception块1(输入通道480,P1 192通道,P2(96-208)通道,P3(16-48),P4 64通道)
- Inception块2(输入通道512,P1 160通道,P2(112-224)通道,P3(24-64),P4 64通道)
- Inception块3(输入通道512,P1 128通道,P2(128-256)通道,P3(24-64),P4 64通道)
- Inception块4(输入通道512,P1 112通道,P2(144-288)通道,P3(24-64),P4 64通道)
- Inception块5(输入通道528,P1 256通道,P2(160-320)通道,P3(24-128),P4 128通道)
- 最大池化层1(3*3核,2步长,1填充)
Stage5
- Inception块1(输入通道832,P1 256通道,P2(160-320)通道,P3(32-128),P4 128通道)
- Inception块2(输入通道832,P1 384通道,P2(192-384)通道,P3(48-128),P4 128通道)
- 全局平均池化层1(输出维度(1,1))
- 展开
- 线性层(1024->10)
代码实现:
# Model
b1 = nn.Sequential(
nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3), nn.ReLU(),
nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
b2 = nn.Sequential(
nn.Conv2d(64, 64, kernel_size=1), nn.ReLU(),
nn.Conv2d(64, 192, kernel_size=3, padding=1),
nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
b3 = nn.Sequential(
Inception(192, 64, (96, 128), (16, 32), 32),
Inception(256, 128, (128, 192), (32, 96), 64),
nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
b4 = nn.Sequential(
Inception(480, 192, (96, 208), (16, 48), 64),
Inception(512, 160, (112, 224), (24, 64), 64),
Inception(512, 128, (128, 256), (24, 64), 64),
Inception(512, 112, (144, 288), (32, 64), 64),
Inception(528, 256, (160, 320), (32, 128), 128),
nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
)
b5 = nn.Sequential(
Inception(832, 256, (160, 320), (32, 128), 128),
Inception(832, 384, (192, 384), (48, 128), 128),
nn.AdaptiveAvgPool2d((1, 1)),
nn.Flatten()
)
net = nn.Sequential(b1, b2, b3, b4, b5, nn.Linear(1024, 10))
CHECK:
Sequential output shape: torch.Size([1, 64, 24, 24])
Sequential output shape: torch.Size([1, 192, 12, 12])
Sequential output shape: torch.Size([1, 480, 6, 6])
Sequential output shape: torch.Size([1, 832, 3, 3])
Sequential output shape: torch.Size([1, 1024])
Linear output shape: torch.Size([1, 10])
2.3 模型训练
# Training
lr, num_epochs, batch_size = 0.1, 10, 128
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=96)
d2l.train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())
结果:
loss 0.248, train acc 0.906, test acc 0.891
1676.0 examples/sec on cuda:0