Mobile NetV2代码及总结

V1地址：https://blog.csdn.net/weixin_44543648/article/details/124417135

MobileNet V2 创新性的提出了具有线性bottleneck 的Inverted 残差块。

The Inverted residual:

• Residual：标准的残差块先用1X1卷积降低通道数，再用3X3卷积获取特征，最后用1X1卷积恢复通道（reduce-transfer-expand）。
• Inverted residual：为了降低计算量，引入深度可分离卷积，为了获取更多的信息，引入bottleneck结构，并为了保证数据的不丢失，在前面的1x1卷积降低通道数的位置，改为扩展通道数。即expand-transfer-reduce。
  结构如下图
  
  不同步长的Inverted residual：
  

Linear bottleneck

由于ReLU会使小于0的特征等于0，即不进行反馈，会造成一定的特征丢失（在缩减特征图数量的时候），因此Bottleneck在最后一个输出通道位置不加入ReLU非线性激活函数。

宽度因子和分辨率因子

宽度因子a人为设置来减少每层网络的输出通道，可以减少axa的参数量，分辨率因子p主要用来减少输入图像大小，可以减少pxp的计算量，两个一起使用，可以减少axaxpxp的计算量。

宽度因子代码：保证通道缩小后也是divisor的倍数，保持想通过倍数有利于提高网络性能。

def _make_divisible(ch,divisor,min_ch = None):
    if not min_ch:
        min_ch = divisor

    new_ch = max(min_ch,int(ch+divisor/2)//divisor*divisor)#将通道数控制在divisor的倍数。
    if new_ch<0.9*ch:
        new_ch += divisor
    return new_ch

代码：

import torch
import torch.nn as nn

def _make_divisible(ch,divisor,min_ch = None):
    if not min_ch:
        min_ch = divisor

    new_ch = max(min_ch,int(ch+divisor/2)//divisor*divisor)#将通道数控制在divisor的倍数。
    if new_ch<0.9*ch:
        new_ch += divisor
    return new_ch

class Inverted_residual_bottleneck(nn.Module):
    def __init__(self,mul_ratio,inchannels,outchannels,stride):
        super(Inverted_residual_bottleneck, self).__init__()
        self.shortcut = True if stride==1 and inchannels==outchannels else False

        layer = []
        if mul_ratio != 1:
            layer.extend([nn.Conv2d(inchannels, inchannels * mul_ratio, 1, 1),
            nn.BatchNorm2d(inchannels * mul_ratio),
            nn.ReLU6()])

        layer.extend([
            nn.Conv2d(inchannels*mul_ratio,inchannels*mul_ratio,3,stride,1,groups = inchannels*mul_ratio),
            nn.BatchNorm2d(inchannels * mul_ratio),
            nn.ReLU6(),
            nn.Conv2d(inchannels * mul_ratio, outchannels, 1, 1),
            nn.BatchNorm2d(outchannels)])

        self.conv = nn.Sequential(*layer)

    def forward(self,x):
        if self.shortcut:
            return x+self.conv(x)
        else:
            return self.conv(x)




class Mobilenet_V2(nn.Module):
    def __init__(self,inchannel,numclasses,alpha,round_nearest = 8):
        super(Mobilenet_V2, self).__init__()
        input_channel = _make_divisible(32*alpha, round_nearest)
        last_channel = _make_divisible(1280*alpha,round_nearest)

        self.conv1 = nn.Conv2d(inchannel,input_channel,3,2,1)
        setting = [
            #t,c,n,s
            [1,16,1,1],
            [6,24,2,2],
            [6,32,3,2],
            [6,64,4,2],
            [6,96,3,1],
            [6,160,3,2],
            [6,320,1,1]
        ]
        self.block = Inverted_residual_bottleneck
        self.stages = nn.ModuleList([])
        for t,c,n,s in setting:
            cout = _make_divisible(c*alpha,round_nearest)
            self.stages.append(self._make_stage(t,input_channel,cout,n,s))
            input_channel = cout
        self.conv2 = nn.Sequential(nn.Conv2d(input_channel,last_channel,1,1),
                                   nn.BatchNorm2d(last_channel),
                                   nn.ReLU())
        self.pool = nn.MaxPool2d(7)
        self.fc = nn.Sequential(
            nn.Dropout(0.2),
            nn.Conv2d(last_channel,numclasses,1)
        )
        for m in self.modules():
            if isinstance(m,nn.Conv2d):
                nn.init.normal_(m.weight,0,0.01)
                if m.bias is not None:
                    nn.init.zeros_(m.bias)
            elif isinstance(m,nn.BatchNorm2d):
                nn.init.ones_(m.weight)
                nn.init.zeros_(m.bias)



    def _make_stage(self,mult,inchannel,ouchannel,repeat,stride):
        strides = [stride]+[1]*repeat
        layers = []
        for i in range(repeat):
            layers.append(self.block(mult,inchannel,ouchannel,strides[i]))
            inchannel = ouchannel
        return nn.Sequential(*layers)

    def forward(self,x):
        x = self.conv1(x)
        for stage in self.stages:
            x = stage(x)
        x = self.conv2(x)
        x = self.pool(x)

        x = self.fc(x)
        x = x.view(x.size(0), -1)
        return x

if __name__ == '__main__':
    input = torch.empty(1,3,224,224)
    m = Mobilenet_V2(3,10,0.5)
    out = m(input)
    print(out)