[深度之眼轻量化网络专题]1.MobileNets

最新推荐文章于 2024-08-13 08:55:23 发布

发的馒头

最新推荐文章于 2024-08-13 08:55:23 发布

阅读量326

点赞数

分类专栏：轻量化网络文章标签：深度学习卷积神经网络计算机视觉

本文链接：https://blog.csdn.net/xie01100110/article/details/107009025

版权

轻量化网络专栏收录该内容

1 篇文章 0 订阅

订阅专栏

1. 推导：MobileNets中的深度游可分离卷积与标准卷积的算力消耗之比？（input feature: FxFxC, kernel size: KxKxCxM, padding=same）

普通卷积：
- 参数量：KxKxCxM
- 运算量：由于是same padding，所以输出图尺寸是FxFxM
  
  FxFxMxKxKxC
深度可分离卷积：
- 参数量：(KxKxCx1) + (1x1xCxM) = (KxKxC) + (CxM)
- 运算量：(FxFxCxKxKx1) + (FxFxMx1x1xC) = (FxFxCxKxK) + (FxFxMxC)
所以两者参数量之比是：
$\frac{(K*K*C) + (C*M)}{F*F*C*M}=\frac{1}{M}+\frac{1}{F^2}$
两者运算量之比是：
$\frac{(F*F*C*K*K) + (M*K*K*C)}{F*F*M*K*K*C}=\frac{1}{M}+\frac{1}{F^2}$

2. 代码实践：利用pytorch复现MobileNet 28层网络结构，模型参数设置同论文一致。

首先定义深度卷积和点卷积

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-PyUJ1N5J-1593353541499)(…/…/image/image-20200628104315398.png)]

# define depthwise convlution using group conv
class dw_conv(nn.Module):
    def __init__(self, in_dim, stride):
        super(dw_conv, self).__init__()
        self.conv = nn.Conv2d(in_channels=in_dim, out_channels=in_dim, kernel_size=3,
                             stride=stride, groups=in_dim, bias=False)
        self.bn = nn.BatchNorm2d(in_dim)
        self.relu = nn.ReLU()
        
    def forward(self, x):
        out = self.conv(x)
        out = self.bn(out)
        out = self.relu(out)
        return out

dw = dw_conv(16, 1)
print(dw)

# define point convolution
class point_conv(nn.Module):
    def __init__(self, in_dim, out_dim):
        super(point_conv, self).__init__()
        self.conv = nn.Conv2d(in_channels=in_dim, out_channels=out_dim, kernel_size=1,
                             stride=1, bias=False)
        self.bn = nn.BatchNorm2d(out_dim)
        self.relu = nn.ReLU()
        
    def forward(self, x):
        out = self.conv(x)
        out = self.bn(out)
        out = self.relu(out)
        return out

point = point_conv(16, 32)
print(point)

然后根据网络模型搭建
在这里插入图片描述

# define mobilenetv1
class MobileNetV1(nn.Module):
    def __init__(self, num_classes):
        super(MobileNetV1, self).__init__()
        self.num_classes = num_classes
        self.net = nn.Sequential(
            nn.Conv2d(3, 32, 3, 2),
            nn.ReLU(),
            dw_conv(32, 1),
            point_conv(32, 64),
            dw_conv(64, 2),
            point_conv(64, 128),
            dw_conv(128, 1),
            point_conv(128, 128),
            dw_conv(128, 2),
            point_conv(128, 256),
            dw_conv(256, 1),
            point_conv(256, 256),
            dw_conv(256, 2),
            point_conv(256, 512),
            dw_conv(512, 1),
            point_conv(512, 512),
            dw_conv(512, 1),
            point_conv(512, 512),
            dw_conv(512, 1),
            point_conv(512, 512),
            dw_conv(512, 1),
            point_conv(512, 512),
            dw_conv(512, 1),
            point_conv(512, 512),
            dw_conv(512, 2),
            point_conv(512, 1024),
            dw_conv(1024, 2),
            point_conv(1024, 1024),
            nn.AvgPool2d(7, 1)
            )
            self.fc1 = nn.Linear(1024, self.num_classes)
            
    def forward(self, x):
        out = self.net(x)
        out = out.view(-1, 1024)
        out = self.fc1(out)
        return out
    
    def init_weights(self):
        print('=> init weights from normal distribution')
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                # nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
                nn.init.normal_(m.weight, std=0.001)
                # nn.init.constant_(m.bias, 0)
            elif isinstance(m, nn.BatchNorm2d):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)