Xception: Deep Learning with Depthwise Separable Convolutions
Xception:使用深度可分离卷积的深度学习
Xception:extreme inception,分解到极致的Inception;
发表时间:[Submitted on 7 Oct 2016 (v1), last revised 4 Apr 2017 (this version, v3)];
发表期刊/会议:Computer Vision and Pattern Recognition;
论文地址:https://arxiv.org/abs/1610.02357;
深度可分离卷积相关内容:
🔗https://blog.csdn.net/COINVK/article/details/128264121;
Inception发展演变:
- GoogLeNet/Inception V1:2014年9月 《Going deeper with convolutions》;
- BN-Inception 2015年2月 《Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift》;
- Inception V2/V3 2015年12月《Rethinking the Inception Architecture for Computer Vision》;
- Inception V4、Inception-ResNet 2016年2月 《Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning》;
- Xception 2016年10月 《Xception: Deep Learning with Depthwise Separable Convolutions》;
0 摘要
Inception结构是介于传统卷积和深度可分离卷积(深度可分离卷积=depthwise深度卷积 + pointwise1×1逐点卷积)的一种中间形态;
深度可分离卷积相关内容:
🔗https://blog.csdn.net/COINVK/article/details/128264121;
受此启发,深度可分离卷积可以被看作有很多分支结构的Inception模块(就是说…如果Inception里的分支有很多的话(分解到极致),可以等同于深度可分离卷积);
基于此发现,本文本文提出一个新的Inception架构:Xception,将原Inception替换成深度可分离卷积;
实验证明,Xception在ImageNet分类数据集上性能优于Inception-V3;
由于Xception架构具有与InceptionV3相同数量的参数,因此性能的提高不是由于容量的增加,而是由于模型参数的更有效使用;
1 简介
2012——AlexNet;
https://blog.csdn.net/COINVK/article/details/128708430;
2013——ZFNet;
https://blog.csdn.net/COINVK/article/details/128713071;
2014——VGG;
https://blog.csdn.net/COINVK/article/details/128756488;
2014——GoogLeNet(Inception V1);
https://blog.csdn.net/COINVK/article/details/128995633;
2015——BN-Inception;
https://blog.csdn.net/COINVK/article/details/129033209;
2015——Inception V2/V3;
https://blog.csdn.net/COINVK/article/details/129061046;
2016——Inception-V4,Inception-ResNet;
https://blog.csdn.net/COINVK/article/details/129122247;
下图展示了一个经典的Inception模块:
1.1 Inception 假设
Inception模块背后的想法是通过将其明确解耦,从而使该过程更加简单和高效;
1.2 常规卷积和可分离卷积之间的形态
在图3的基础上,分解到极致,如图4所示,这样处理的Inception(Xception)就变成了和深度可分离卷积等价的一个结构;
Xception和深度可分离卷积的区别:
- 操作顺序不同
- 深度可分离卷积先进行depwise深度卷积再进行pointwise逐点卷积;
- Xception模块先进行pointwise逐点卷积,再进行depwise深度卷积;
- 第一次操作后激活函数不同(实验结果见图10)
- Xception使用非线性激活函数ReLu;
- 深度可分离卷积不使用非线性激活函数;
3 Xception架构
本文提出Xception网络架构,主要基于深度可分离卷积(参考图4);
Xception架构如图5所示,一共有36层,主要研究多类别图像分类任务;
36个层被划分为14个模块,除了第一个和最后一个模块之外,所有模块周围都有线性残差连接;
pytorch代码实现,Xception框架:
class Xception(nn.Module):
def __init__(self, num_classes=1000):
super(Xception, self).__init__()
self.num_classes = num_classes # 总分类数
# Entry flow
self.conv1 = nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3, stride=2, padding=0, bias=False)
self.bn1 = nn.BatchNorm2d(32)
self.relu = nn.ReLU(inplace=True)
self.conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, stride=1, padding=0, bias=False)
self.bn2 = nn.BatchNorm2d(64)
self.block1 = Block(64, 128, 2, 2, start_with_relu=False, grow_first=True)
self.block2 = Block(128, 256, 2, 2, start_with_relu=True, grow_first=True)
self.block3 = Block(256, 728, 2, 2, start_with_relu=True, grow_first=True)
# Middle flow
self.block4 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
self.block5 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
self.block6 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
self.block7 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
self.block8 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
self.block9 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
self.block10 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
self.block11 = Block(728, 728, 3, 1, start_with_relu=True, grow_first=True)
# Exit flow
self.block12 = Block(728, 1024, 2, 2, start_with_relu=True, grow_first=False)
self.conv3 = SeparableConv2d(1024, 1536, 3, 1, 1)
self.bn3 = nn.BatchNorm2d(1536)
self.conv4 = SeparableConv2d(1536, 2048, 3, 1, 1)
self.bn4 = nn.BatchNorm2d(2048)
self.fc = nn.Linear(2048, num_classes)
def forward(self, x):
# Entry flow
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.conv2(x)
x = self.bn2(x)
x = self.relu(x)
x = self.block1(x)
x = self.block2(x)
x = self.block3(x)
# Middle flow
# 重复8次
x = self.block4(x)
x = self.block5(x)
x = self.block6(x)
x = self.block7(x)
x = self.block8(x)
x = self.block9(x)
x = self.block10(x)
x = self.block11(x)
# Exit flow
x = self.block12(x)
x = self.conv3(x)
x = self.bn3(x)
x = self.relu(x)
x = self.conv4(x)
x = self.bn4(x)
x = self.relu(x)
# GAP
x = F.adaptive_avg_pool2d(x, (1, 1))
x = x.view(x.size(0), -1)
x = self.fc(x)
return x
pytorch实现深度可分离卷积:
# 深度可分离卷积 = dw + pw
class SeparableConv2d(nn.Module):
def __init__(self, in_channels, out_channels, kernel_size, stride, padding, dilation=1, bias=False):
super(SeparableConv2d, self).__init__()
# 逐通道卷积:groups=in_channels=out_channels
self.conv1 = nn.Conv2d(in_channels, in_channels, kernel_size, stride, padding, dilation, groups=in_channels,
bias=bias)
# 逐点卷积:普通1x1卷积
self.pointwise = nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=1, padding=0, dilation=1, groups=1,
bias=bias)
def forward(self, x):
x = self.conv1(x)
x = self.pointwise(x)
return x
pytorch实现Xception中的残差块:
# Xception中的一个残差block
class Block(nn.Module):
def __init__(self, in_filters, out_filters, reps, strides=1, start_with_relu=True, grow_first=True):
#:parm reps:块重复次数
super(Block, self).__init__()
# 通道数不相等的情况
if out_filters != in_filters or strides != 1:
self.skip = nn.Conv2d(in_filters, out_filters, kernel_size=1, stride=strides, bias=False)
self.skipbn = nn.BatchNorm2d(out_filters)
else:
self.skip = None
self.relu = nn.ReLU(inplace=True)
rep = []
filters = in_filters
if grow_first:
rep.append(self.relu)
# 这里的卷积不改变特征图尺寸
rep.append(SeparableConv2d(in_filters, out_filters, kernel_size=3, stride=1, padding=1, bias=False))
rep.append(nn.BatchNorm2d(out_filters))
filters = out_filters
for i in range(reps - 1):
rep.append(self.relu)
# 这里的卷积不改变特征图尺寸
rep.append(SeparableConv2d(filters, filters, kernel_size=3, stride=1, padding=1, bias=False))
rep.append(nn.BatchNorm2d(filters))
if not grow_first:
rep.append(self.relu)
# 这里的卷积不改变特征图尺寸
rep.append(SeparableConv2d(in_filters, out_filters, kernel_size=3, stride=1, padding=1, bias=False))
rep.append(nn.BatchNorm2d(out_filters))
if not start_with_relu:
rep = rep[1:]
else:
rep[0] = nn.ReLU(inplace=False)
if strides != 1:
rep.append(nn.MaxPool2d(kernel_size=3, stride=strides, padding=1))
self.rep = nn.Sequential(*rep)
def forward(self, inp):
x = self.rep(inp)
if self.skip is not None:
skip = self.skip(inp)
skip = self.skipbn(skip)
else:
skip = inp
# 残差连接
x += skip
return x
4 实验
Inception V3与Xception参数量相似(网络容量接近),将两个模型进行比较;
对两个图像分类任务进行了比较:
- ImageNet数据集1000类单分类任务;
- 大规模JFT数据集上的17000类多标签分类任务;
4.1 JFT数据集介绍
JFT是一个用于大规模图像分类数据集的Google内部数据集,含超过3.5亿张高分辨率图像,其中标注了17000个类别的标签。
为了评估在JFT上训练的模型的性能,使用辅助数据集FastEval14k。FastEval14k是一个包含14000张图像的数据集,其中包含来自约6000个类的密集注释(平均每张图像36.5个标签)。
4.2 优化配置
- On Image Net:
- Optimizer: SGD;
- Momentmu:0.9;
- Initial learning rate: 0.045;
- Learning rate decay: decay of rate 0.94 every 2 epochs;
- On JFT:
- Optimizer: RMSprop ;
- Momentum: 0.9;
- Initial learning rate: 0.001;
- Learning rate decay: decay of rate 0.9 every 3,000,000 samples;
4.3 正则化配置(防止过拟合)
- Weight decay权重衰减;
- l2正则化;
- Inception-V3:weight decay rate: 4e-5(最优的,调参之后的);
- Xception:1e-5(次优的,没有经过调参的);
- Dropout;
- 两个模型dropout rate 0.5;
- 对于JFT实验,由于数据集的大小太大,在任何合理的时间内都不可能过度拟合,因此没有dropout。
- Auxiliary loss tower辅助分类头;
- 两个模型都没有用辅助分类器;
4.4 模型训练基础
TensorFlow;
60 NVIDIA K80 GPUs ;
ImageNet实验每次大约花费3天,而JFT实验每次花费一个多月(没有完全收敛 完全收敛需要三个月);
4.5 实验结果
4.5.1 分类性能
4.5.2 Size and speed
表3显示,两个模型参数量差不多,但Xception(28)比Inception-V3(31)训练慢一些;
4.6. 残差连接的作用
下图显示,加了残差连接的模型,收敛更快,精度更高;
4.7 激活函数的作用
在1×1卷积之后,depthwise卷积之前不同激活函数的效果;
红色:不用激活函数;
绿色:使用ELU激活函数;
蓝色:使用ReLU激活函数;
图10显示,没有任何非线性会导致更快的收敛和更好的最终性能;