MobileNet v1神经网络剖析

已于 2022-12-12 16:46:32 修改
阅读量687 收藏 3
点赞数 1
分类专栏： 神经网络 文章标签： 神经网络 深度学习
于 2022-12-08 11:07:41 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/benben044/article/details/128232072
版权

本文参考:

MobileNet网络_-断言-的博客-CSDN博客_mobile-ne

Conv2d中的groups参数（分组卷积）怎么理解？ 【分组卷积可以减少参数量、且不容易过拟合（类似正则化）】_马鹏森的博客-CSDN博客_conv groups

Pytorch MobileNetV1 学习_DevinDong123的博客-CSDN博客

1 为什么使用MobileNet网络

传统卷积神经网络，内存需求大、运算量大导致无法在移动设备以及嵌入式设备上运行。

MobileNet网络提供了轻量级CNN网络。相比于传统卷积神经网络，在准确率小幅降低的前提下大大减少模型参数与运算量。

MobileNet的核心是： 使用深度可分离卷积去替代普通的卷积核,对输入进行特征提取,在性能不下滑太多的前提下,大大减少参数量和计算量。

2 普通卷积、DW + PW的比较

（1）传统卷积

传统卷积把特征提取和特征组合一次完成并输出，其特点是：

  • 卷积核channel = 输入特征矩阵channel
  • 输出特征矩阵channel = 卷积核个数

（2） DW + PW卷积

DW卷积其特点为：

  • 卷积核channel = 1
  • 输入特征矩阵channel = 卷积核个数 = 输出特征矩阵channel

PW卷积其特点为:

  • 卷积核channel = 输入特征矩阵channel
  • 卷积核大小 = 1
  • 卷积核个数 = 输出特征矩阵channel

（3）参数量对比

假设某一网络卷积层，其卷积核大小为3*3，输入通道为16，输出通道为32。

常规CNN的参数量为：(3*3*16+1)*32=4640个。

深度可分离卷积参数量计算：

先用16个大小为3*3的卷积核（3*3*1）作用于16个通道的输入图像，得到了16个特征图。在做融合操作之前，接着用32个大小为1*1的卷积核（1*1*16）遍历上述得到16个特征图。则参数量为：(3*3*1+1)*16+(1*1*16+1)*32=706个。

在进行depthwise卷积时只使用了一种维度为in_channels的卷积核进行特征提取（没有进行特征组合）；在进行pointwise卷积时只使用了output_channels种维度为in_channles*1*1的卷积核进行特征组合。

 3 MobileNet网络的模型结构

 4 DW网络构造

DW的卷积改变了卷积核的输入通道数，本来卷积核的输入通道数=输入特征矩阵的通道数，而DW卷积输入通道数则是为1。

此时用到nn.Conv2d中的groups参数，表示对输入feature map进行分组，然后每组分别卷积，每组的卷积核输入通道数 = 输入特征矩阵通道数 / groups。 

如果groups=输入特征矩阵的通道数，则刚好形成了如下的卷积效果：

5 MobileNet pytorch实现及剪枝

import torch
import torch.nn as nn
from nni.compression.pytorch.pruning import L1NormPruner
from nni.compression.pytorch.speedup import ModelSpeedup


class BasicConv2dBlock(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, downsample=True, **kwargs):
        super(BasicConv2dBlock, self).__init__()
        stride = 2 if downsample else 1
        self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride=stride, **kwargs)
        self.bn = nn.BatchNorm2d(out_channels)
        self.relu = nn.ReLU(inplace=True)

    def forward(self, x):
        x = self.conv(x)
        x = self.bn(x)
        x = self.relu(x)
        return x

class DepthSeperabelConv2dBlock(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, **kwargs):
        super(DepthSeperabelConv2dBlock, self).__init__()

        # 深度卷积
        self.depth_wise = nn.Sequential(
            nn.Conv2d(in_channels, in_channels, kernel_size, groups=in_channels, **kwargs),
            nn.BatchNorm2d(in_channels),
            nn.ReLU(inplace=True)
        )

        # 逐点卷积
        self.point_wise = nn.Sequential(
            nn.Conv2d(in_channels, out_channels, 1),
            nn.BatchNorm2d(out_channels),
            nn.ReLU(inplace=True)
        )

    def forward(self, x):
        x = self.depth_wise(x)
        x = self.point_wise(x)
        return x

class MobileNet(nn.Module):
    def __init__(self, class_num=100):
        super(MobileNet, self).__init__()

        self.stem = nn.Sequential(
            BasicConv2dBlock(3, 32, kernel_size=3, padding=1, bias=False),
            DepthSeperabelConv2dBlock(32, 64, kernel_size=3, padding=1, bias=False)
        )

        self.conv1 = nn.Sequential(
            DepthSeperabelConv2dBlock(64, 128, kernel_size=3, stride=2, padding=1, bias=False),
            DepthSeperabelConv2dBlock(128, 128, kernel_size=3, padding=1, bias=False)
        )

        self.conv2 = nn.Sequential(
            DepthSeperabelConv2dBlock(128, 256, kernel_size=3, stride=2, padding=1, bias=False),
            DepthSeperabelConv2dBlock(256, 256, kernel_size=3, padding=1, bias=False)
        )

        self.conv3 = nn.Sequential(
            DepthSeperabelConv2dBlock(256, 512, kernel_size=3, stride=2, padding=1, bias=False),
            DepthSeperabelConv2dBlock(512, 512, kernel_size=3, padding=1, bias=False),
            DepthSeperabelConv2dBlock(512, 512, kernel_size=3, padding=1, bias=False),
            DepthSeperabelConv2dBlock(512, 512, kernel_size=3, padding=1, bias=False),
            DepthSeperabelConv2dBlock(512, 512, kernel_size=3, padding=1, bias=False),
            DepthSeperabelConv2dBlock(512, 512, kernel_size=3, padding=1, bias=False)
        )

        self.conv4 = nn.Sequential(
            DepthSeperabelConv2dBlock(512, 1024, kernel_size=3, stride=2, padding=1, bias=False),
            DepthSeperabelConv2dBlock(1024, 1024, kernel_size=3, padding=1, bias=False)
        )

        self.fc = nn.Linear(1024, class_num)
        self.avg = nn.AdaptiveAvgPool2d(1)

    def forward(self, x):
        x = self.stem(x)

        x = self.conv1(x)
        x = self.conv2(x)
        x = self.conv3(x)
        x = self.conv4(x)

        x = self.avg(x)
        x = x.view(x.size(0), -1)
        x = self.fc(x)
        return x

def main():
    config_list = [{
        'sparsity_per_layer':0.5,
        'op_types': ['Conv2d']
    }]


    model = MobileNet(10)
    print('-----------raw model------------')
    print(model)

    pruner = L1NormPruner(model, config_list)
    _, masks = pruner.compress()
    for name, mask in masks.items():
        print(name, ' sparsity: ', '{:.2f}'.format(mask['weight'].sum() / mask['weight'].numel()))
    pruner._unwrap_model()
    ModelSpeedup(model, torch.rand(1, 3, 512, 512), masks).speedup_model()

    print('------------after speedup------------')
    print(model)


if __name__ == '__main__':
    main()

 以上代码不仅实现了mobilenet，还通过nni对模型进行了剪枝。

nni剪枝需要15G以上的内存，所以电脑内存没个20G跑不起来。

整体日志如下：

-----------raw model------------

MobileNet(

  (stem): Sequential(

    (0): BasicConv2dBlock(

      (conv): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)

      (bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

      (relu): ReLU(inplace=True)

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)

        (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(32, 64, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv1): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=64, bias=False)

        (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=128, bias=False)

        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv2): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=128, bias=False)

        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256, bias=False)

        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv3): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=256, bias=False)

        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=512, bias=False)

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (2): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=512, bias=False)

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (3): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=512, bias=False)

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (4): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=512, bias=False)

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (5): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=512, bias=False)

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv4): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=512, bias=False)

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=1024, bias=False)

        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (fc): Linear(in_features=1024, out_features=10, bias=True)

  (avg): AdaptiveAvgPool2d(output_size=1)

)

stem.0.conv  sparsity:  0.50

stem.1.depth_wise.0  sparsity:  0.50

stem.1.point_wise.0  sparsity:  0.50

conv1.0.depth_wise.0  sparsity:  0.50

conv1.0.point_wise.0  sparsity:  0.50

conv1.1.depth_wise.0  sparsity:  0.50

conv1.1.point_wise.0  sparsity:  0.50

conv2.0.depth_wise.0  sparsity:  0.50

conv2.0.point_wise.0  sparsity:  0.50

conv2.1.depth_wise.0  sparsity:  0.50

conv2.1.point_wise.0  sparsity:  0.50

conv3.0.depth_wise.0  sparsity:  0.50

conv3.0.point_wise.0  sparsity:  0.50

conv3.1.depth_wise.0  sparsity:  0.50

conv3.1.point_wise.0  sparsity:  0.50

conv3.2.depth_wise.0  sparsity:  0.50

conv3.2.point_wise.0  sparsity:  0.50

conv3.3.depth_wise.0  sparsity:  0.50

conv3.3.point_wise.0  sparsity:  0.50

conv3.4.depth_wise.0  sparsity:  0.50

conv3.4.point_wise.0  sparsity:  0.50

conv3.5.depth_wise.0  sparsity:  0.50

conv3.5.point_wise.0  sparsity:  0.50

conv4.0.depth_wise.0  sparsity:  0.50

conv4.0.point_wise.0  sparsity:  0.50

conv4.1.depth_wise.0  sparsity:  0.50

conv4.1.point_wise.0  sparsity:  0.50

[2022-12-06 18:37:49] INFO (nni.compression.pytorch.speedup.compressor/MainThread) start to speed up the model

[2022-12-06 18:37:53] INFO (FixMaskConflict/MainThread) {'stem.0.conv': 1, 'stem.1.depth_wise.0': 1, 'stem.1.point_wise.0': 1, 'conv1.0.depth_wise.0': 1, 'conv1.0.point_wise.0': 1, 'conv1.1.depth_wise.0': 1, 'conv1.1.point_wise.0': 1, 'conv2.0.depth_wise.0': 1, 'conv2.0.point_wise.0': 1, 'conv2.1.depth_wise.0': 1, 'conv2.1.point_wise.0': 1, 'conv3.0.depth_wise.0': 1, 'conv3.0.point_wise.0': 1, 'conv3.1.depth_wise.0': 1, 'conv3.1.point_wise.0': 1, 'conv3.2.depth_wise.0': 1, 'conv3.2.point_wise.0': 1, 'conv3.3.depth_wise.0': 1, 'conv3.3.point_wise.0': 1, 'conv3.4.depth_wise.0': 1, 'conv3.4.point_wise.0': 1, 'conv3.5.depth_wise.0': 1, 'conv3.5.point_wise.0': 1, 'conv4.0.depth_wise.0': 1, 'conv4.0.point_wise.0': 1, 'conv4.1.depth_wise.0': 1, 'conv4.1.point_wise.0': 1}

[2022-12-06 18:37:53] INFO (FixMaskConflict/MainThread) dim0 sparsity: 0.500000

[2022-12-06 18:37:53] INFO (FixMaskConflict/MainThread) dim1 sparsity: 0.000000

[2022-12-06 18:37:53] INFO (FixMaskConflict/MainThread) Dectected conv prune dim" 0

[2022-12-06 18:37:53] INFO (nni.compression.pytorch.speedup.compressor/MainThread) infer module masks...

[2022-12-06 18:37:53] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.0.conv

[2022-12-06 18:37:54] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.0.bn

[2022-12-06 18:37:55] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.0.relu

[2022-12-06 18:37:55] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.1.depth_wise.0

[2022-12-06 18:37:57] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.1.depth_wise.1

[2022-12-06 18:37:59] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.1.depth_wise.2

[2022-12-06 18:38:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.1.point_wise.0

[2022-12-06 18:38:02] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.1.point_wise.1

[2022-12-06 18:38:05] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for stem.1.point_wise.2

[2022-12-06 18:38:07] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.0.depth_wise.0

[2022-12-06 18:38:08] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.0.depth_wise.1

[2022-12-06 18:38:09] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.0.depth_wise.2

[2022-12-06 18:38:09] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.0.point_wise.0

[2022-12-06 18:38:11] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.0.point_wise.1

[2022-12-06 18:38:12] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.0.point_wise.2

[2022-12-06 18:38:13] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.1.depth_wise.0

[2022-12-06 18:38:14] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.1.depth_wise.1

[2022-12-06 18:38:16] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.1.depth_wise.2

[2022-12-06 18:38:17] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.1.point_wise.0

[2022-12-06 18:38:18] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.1.point_wise.1

[2022-12-06 18:38:20] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv1.1.point_wise.2

[2022-12-06 18:38:20] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.0.depth_wise.0

[2022-12-06 18:38:21] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.0.depth_wise.1

[2022-12-06 18:38:21] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.0.depth_wise.2

[2022-12-06 18:38:21] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.0.point_wise.0

[2022-12-06 18:38:22] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.0.point_wise.1

[2022-12-06 18:38:23] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.0.point_wise.2

[2022-12-06 18:38:23] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.1.depth_wise.0

[2022-12-06 18:38:24] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.1.depth_wise.1

[2022-12-06 18:38:25] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.1.depth_wise.2

[2022-12-06 18:38:25] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.1.point_wise.0

[2022-12-06 18:38:26] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.1.point_wise.1

[2022-12-06 18:38:26] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv2.1.point_wise.2

[2022-12-06 18:38:27] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.0.depth_wise.0

[2022-12-06 18:38:27] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.0.depth_wise.1

[2022-12-06 18:38:27] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.0.depth_wise.2

[2022-12-06 18:38:27] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.0.point_wise.0

[2022-12-06 18:38:27] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.0.point_wise.1

[2022-12-06 18:38:28] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.0.point_wise.2

[2022-12-06 18:38:28] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.1.depth_wise.0

[2022-12-06 18:38:28] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.1.depth_wise.1

[2022-12-06 18:38:29] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.1.depth_wise.2

[2022-12-06 18:38:29] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.1.point_wise.0

[2022-12-06 18:38:29] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.1.point_wise.1

[2022-12-06 18:38:30] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.1.point_wise.2

[2022-12-06 18:38:30] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.2.depth_wise.0

[2022-12-06 18:38:30] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.2.depth_wise.1

[2022-12-06 18:38:30] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.2.depth_wise.2

[2022-12-06 18:38:31] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.2.point_wise.0

[2022-12-06 18:38:31] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.2.point_wise.1

[2022-12-06 18:38:31] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.2.point_wise.2

[2022-12-06 18:38:31] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.3.depth_wise.0

[2022-12-06 18:38:32] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.3.depth_wise.1

[2022-12-06 18:38:32] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.3.depth_wise.2

[2022-12-06 18:38:32] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.3.point_wise.0

[2022-12-06 18:38:33] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.3.point_wise.1

[2022-12-06 18:38:33] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.3.point_wise.2

[2022-12-06 18:38:33] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.4.depth_wise.0

[2022-12-06 18:38:34] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.4.depth_wise.1

[2022-12-06 18:38:34] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.4.depth_wise.2

[2022-12-06 18:38:34] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.4.point_wise.0

[2022-12-06 18:38:34] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.4.point_wise.1

[2022-12-06 18:38:35] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.4.point_wise.2

[2022-12-06 18:38:35] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.5.depth_wise.0

[2022-12-06 18:38:35] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.5.depth_wise.1

[2022-12-06 18:38:36] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.5.depth_wise.2

[2022-12-06 18:38:36] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.5.point_wise.0

[2022-12-06 18:38:36] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.5.point_wise.1

[2022-12-06 18:38:36] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv3.5.point_wise.2

[2022-12-06 18:38:36] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.0.depth_wise.0

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.0.depth_wise.1

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.0.depth_wise.2

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.0.point_wise.0

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.0.point_wise.1

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.0.point_wise.2

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.1.depth_wise.0

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.1.depth_wise.1

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.1.depth_wise.2

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.1.point_wise.0

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.1.point_wise.1

[2022-12-06 18:38:37] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for conv4.1.point_wise.2

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for avg

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for .aten::size.83

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for .aten::Int.84

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for .aten::view.85

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.jit_translate/MainThread) View Module output size: [8, -1]

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update mask for fc

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the fc

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the .aten::view.85

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the .aten::Int.84

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the .aten::size.83

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the avg

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.1.point_wise.2

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.1.point_wise.1

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.1.point_wise.0

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.1.depth_wise.2

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.1.depth_wise.1

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.1.depth_wise.0

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.0.point_wise.2

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.0.point_wise.1

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.0.point_wise.0

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.0.depth_wise.2

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.0.depth_wise.1

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv4.0.depth_wise.0

[2022-12-06 18:38:38] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.5.point_wise.2

[2022-12-06 18:38:39] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.5.point_wise.1

[2022-12-06 18:38:39] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.5.point_wise.0

[2022-12-06 18:38:39] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.5.depth_wise.2

[2022-12-06 18:38:39] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.5.depth_wise.1

[2022-12-06 18:38:39] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.5.depth_wise.0

[2022-12-06 18:38:39] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.4.point_wise.2

[2022-12-06 18:38:39] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.4.point_wise.1

[2022-12-06 18:38:40] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.4.point_wise.0

[2022-12-06 18:38:40] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.4.depth_wise.2

[2022-12-06 18:38:40] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.4.depth_wise.1

[2022-12-06 18:38:40] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.4.depth_wise.0

[2022-12-06 18:38:40] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.3.point_wise.2

[2022-12-06 18:38:40] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.3.point_wise.1

[2022-12-06 18:38:40] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.3.point_wise.0

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.3.depth_wise.2

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.3.depth_wise.1

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.3.depth_wise.0

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.2.point_wise.2

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.2.point_wise.1

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.2.point_wise.0

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.2.depth_wise.2

[2022-12-06 18:38:41] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.2.depth_wise.1

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.2.depth_wise.0

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.1.point_wise.2

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.1.point_wise.1

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.1.point_wise.0

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.1.depth_wise.2

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.1.depth_wise.1

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.1.depth_wise.0

[2022-12-06 18:38:42] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.0.point_wise.2

[2022-12-06 18:38:43] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.0.point_wise.1

[2022-12-06 18:38:43] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.0.point_wise.0

[2022-12-06 18:38:43] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.0.depth_wise.2

[2022-12-06 18:38:43] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.0.depth_wise.1

[2022-12-06 18:38:43] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv3.0.depth_wise.0

[2022-12-06 18:38:43] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.1.point_wise.2

[2022-12-06 18:38:43] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.1.point_wise.1

[2022-12-06 18:38:44] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.1.point_wise.0

[2022-12-06 18:38:44] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.1.depth_wise.2

[2022-12-06 18:38:44] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.1.depth_wise.1

[2022-12-06 18:38:45] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.1.depth_wise.0

[2022-12-06 18:38:45] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.0.point_wise.2

[2022-12-06 18:38:45] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.0.point_wise.1

[2022-12-06 18:38:46] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.0.point_wise.0

[2022-12-06 18:38:46] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.0.depth_wise.2

[2022-12-06 18:38:46] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.0.depth_wise.1

[2022-12-06 18:38:46] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv2.0.depth_wise.0

[2022-12-06 18:38:47] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.1.point_wise.2

[2022-12-06 18:38:47] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.1.point_wise.1

[2022-12-06 18:38:48] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.1.point_wise.0

[2022-12-06 18:38:49] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.1.depth_wise.2

[2022-12-06 18:38:49] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.1.depth_wise.1

[2022-12-06 18:38:50] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.1.depth_wise.0

[2022-12-06 18:38:50] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.0.point_wise.2

[2022-12-06 18:38:51] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.0.point_wise.1

[2022-12-06 18:38:52] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.0.point_wise.0

[2022-12-06 18:38:52] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.0.depth_wise.2

[2022-12-06 18:38:53] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.0.depth_wise.1

[2022-12-06 18:38:53] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the conv1.0.depth_wise.0

[2022-12-06 18:38:54] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.1.point_wise.2

[2022-12-06 18:38:55] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.1.point_wise.1

[2022-12-06 18:38:56] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.1.point_wise.0

[2022-12-06 18:38:57] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.1.depth_wise.2

[2022-12-06 18:38:58] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.1.depth_wise.1

[2022-12-06 18:38:58] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.1.depth_wise.0

[2022-12-06 18:38:59] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.0.relu

[2022-12-06 18:38:59] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.0.bn

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Update the indirect sparsity for the stem.0.conv

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) resolve the mask conflict

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace compressed modules...

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.0.conv, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.0.bn, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 6

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.0.relu, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.1.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.1.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 6

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.1.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.1.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.1.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 14

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: stem.1.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.0.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.0.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 14

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.0.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.0.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.0.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 36

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.0.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.1.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.1.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 36

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.1.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.1.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.1.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 30

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv1.1.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.0.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.0.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 30

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.0.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.0.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.0.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 62

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.0.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.1.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.1.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 62

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.1.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.1.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.1.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 63

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv2.1.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.0.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.0.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 63

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.0.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.0.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.0.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 121

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.0.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.1.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.1.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 121

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.1.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.1.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.1.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 128

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.1.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:00] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.2.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.2.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 128

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.2.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.2.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.2.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 125

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.2.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.3.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.3.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 125

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.3.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.3.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.3.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 135

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.3.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.4.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.4.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 135

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.4.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.4.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.4.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 117

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.4.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.5.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.5.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 117

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.5.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.5.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.5.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 125

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv3.5.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.0.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.0.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 125

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.0.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.0.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.0.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 265

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.0.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.1.depth_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.1.depth_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 265

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.1.depth_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.1.point_wise.0, op_type: Conv2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.1.point_wise.1, op_type: BatchNorm2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace batchnorm2d with num_features: 512

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: conv4.1.point_wise.2, op_type: ReLU)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: avg, op_type: AdaptiveAvgPool2d)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Warning: cannot replace (name: .aten::size.83, op_type: aten::size) which is func type

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Warning: cannot replace (name: .aten::Int.84, op_type: aten::Int) which is func type

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) Warning: cannot replace (name: .aten::view.85, op_type: aten::view) which is func type

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) replace module (name: fc, op_type: Linear)

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compress_modules/MainThread) replace linear with new in_features: 512, out_features: 10

[2022-12-06 18:39:01] INFO (nni.compression.pytorch.speedup.compressor/MainThread) speedup done

------------after speedup------------

MobileNet(

  (stem): Sequential(

    (0): BasicConv2dBlock(

      (conv): Conv2d(3, 6, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)

      (bn): BatchNorm2d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

      (relu): ReLU(inplace=True)

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(6, 6, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=6, bias=False)

        (1): BatchNorm2d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(6, 14, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(14, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv1): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(14, 14, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=14, bias=False)

        (1): BatchNorm2d(14, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(14, 36, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(36, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(36, 36, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=36, bias=False)

        (1): BatchNorm2d(36, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(36, 30, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(30, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv2): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(30, 30, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=30, bias=False)

        (1): BatchNorm2d(30, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(30, 62, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(62, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(62, 62, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=62, bias=False)

        (1): BatchNorm2d(62, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(62, 63, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(63, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv3): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(63, 63, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=63, bias=False)

        (1): BatchNorm2d(63, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(63, 121, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(121, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(121, 121, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=121, bias=False)

        (1): BatchNorm2d(121, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(121, 128, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (2): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=128, bias=False)

        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(128, 125, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(125, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (3): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(125, 125, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=125, bias=False)

        (1): BatchNorm2d(125, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(125, 135, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(135, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (4): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(135, 135, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=135, bias=False)

        (1): BatchNorm2d(135, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(135, 117, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(117, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (5): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(117, 117, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=117, bias=False)

        (1): BatchNorm2d(117, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(117, 125, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(125, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (conv4): Sequential(

    (0): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(125, 125, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=125, bias=False)

        (1): BatchNorm2d(125, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(125, 265, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(265, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

    (1): DepthSeperabelConv2dBlock(

      (depth_wise): Sequential(

        (0): Conv2d(265, 265, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=265, bias=False)

        (1): BatchNorm2d(265, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

      (point_wise): Sequential(

        (0): Conv2d(265, 512, kernel_size=(1, 1), stride=(1, 1))

        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

        (2): ReLU(inplace=True)

      )

    )

  )

  (fc): Linear(in_features=512, out_features=10, bias=True)

  (avg): AdaptiveAvgPool2d(output_size=1)

)

数据猴赛雷
关注
  • 1
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
NNI --模型剪枝
qq_42178122的博客
12-28 1071
剪枝是压缩神经网络模型的常用技术。剪枝方法探索模型权重（参数）中的冗余，并尝试删除/修剪冗余和非关键权重。冗余元素从模型中修剪，其值归零，我们确保它们不参与反向传播过程。修剪深度学习模型有三种常见做法，NNI 通过在关键修剪阶段工作来支持上述所有修剪实践。
深度学习网络模型 MobileNet系列MobileNet V1、MobileNet V2、MobileNet V3网络详解以及pytorch代码复现
guoqingru0311的博客
11-01 1万+
深度学习网络模型 MobileNet系列MobileNet V1、MobileNet V2、MobileNet V3网络详解以及pytorch代码复现 1、DW卷积与普通卷积计算量对比 DW与PW计算量 普通卷积计算量 计算量对比 2、MobileNet V1 MobileNet V1网络结构 MobileNet V1网络结构代码 3、MobileNet V2 倒残差结构模块 倒残差模块代码 MobileNet V2详细网络结构 MobileNet V2网络结构代码 4、MobileNet V3
MobileNetV1详细原理（含torch源码）
Code_and516的博客
04-17 2187
MobilenetV1（含torch源码）—— cifar10
mobileNetV1网络解析，以及实现（pytorch）
最新发布
m0_60607895的博客
05-15 241
所以深度可分离卷积参数量是标准卷积的 D K × D K × M + M × N D K × D K × M × N = 1 N + 1 D K 2 \frac{D_{K} \times D_{K} \times M+M \times N}{D_{K} \times D_{K} \times M \times N}=\frac{1}{N}+\frac{1}{D_{K}^{2}} DK​×DK​×M×NDK​×DK​×M+M×N​=N1​+DK2​1​。深度可分离卷积有深度卷积+逐点卷积。
【CNN】MobileNet——卷积神经网络中轻量级的经典
lingchen1906的博客
11-22 2182
作者实验发现 V2 网络最后一部分结构可以优化，如Figure5所示，原始的结构用 1x1 的卷积来调整 feature 的维度，从而提高预测的精度，但是这一部分也会造成一定的延时，为了减少延时，作者把 average pooling 提前，这样的话，这样就提前把 feature 的 size 减下来了（pooling 之后 feature size 从 7x7 降到了 1x1）。这样一来延时减小了，但是试验证明精度却几乎没有降低。
MobileNets(V1)简介及两个初步的代码实验
chenxy_bwave的专栏
04-25 3381
MobileNet在2017年由谷歌2017年提出[1]，主要是为了解决卷积神经网络在移动端或者嵌入式平台中落地应用的运算负荷的问题。一方面，各路大神不断地在修炼越来越大的网络，另一方面，移动端或者嵌入式则要求轻量级的网络，同时又要求性能上没有严重的损失，正所谓既要马儿跑的好，又要马儿不吃草。本文概要性地价绍MobileNet的基本要素，然后做了两个代码实验： (1) 基于预训练的MobileNet模型的特征提取训练的猫狗分类器；(2) 仅利用MobileNet模型结构在CIFAR10数据集
图像识别：基于mobilenet轻量级神经网络识别
03-20
《图像识别：基于Mobilenet轻量级神经网络的实现》 图像识别是计算机视觉领域的一个核心任务，它涉及到从图像中自动识别出物体、场景、文字等信息。随着深度学习技术的发展，尤其是神经网络的广泛应用，图像识别的...
基于神经网络的交通标志检测方法.pdf
09-25
具体来说，论文对比分析了SSD系列（如SSD_mobilenet_v1、SSD_mobilenet_v2、SSD_inception_v2）和Faster R-CNN_inception_v2的性能。实验结果显示，Faster R-CNN_inception_v2在平均精度（mAP）上表现出色，意味着它...
mobilenet-v1-ssd-mp-0_675.7z
11-29
**MobileNetV1**: MobileNetV1是由Google在2017年提出的一种轻量级卷积神经网络（CNN），专为移动设备设计。它通过深度可分离卷积（Depthwise Separable Convolution）大大减少了计算量和参数数量，使得模型能够在...
jetson inference ssd 预训练模型mobilenet-v1-ssd-mp-0-675.pth
05-15
**MobileNet-V1**: MobileNet是Google提出的一种轻量级卷积神经网络架构，主要用于移动设备和嵌入式平台上的高效计算。它的核心思想是使用深度可分离卷积（Depthwise Separable Convolution），这大大减少了计算量和...
基于卷积神经网络的抽油机故障诊断.docx
12-15
人工神经网络（ANN）如BP神经网络、RBF、小波神经网络、极限学习机和自组织神经网络等已被应用于抽油机故障诊断，但它们存在内部映射结构复杂、特征提取不充分的问题。 卷积神经网络（CNN）的出现，尤其是随着深度...
基于Slimming的MobileNetV3半自动剪枝
benben044的专栏
01-04 1812
针对MobileNetv3，介绍了NNI工具存在的不足之处，同时详细介绍了基于Slimming理论的剪枝方法，以及剪枝过程中碰到的问题定位方法。
NNI模型剪枝压缩使用的较详细记录
CaiDou的博客
01-31 3910
使用微软nni工具，基于pytorch进行模型压缩记录总结，最后的结论一很久以前实验的，可能会有点误差，或者NNI工具更新了。但是nni模型剪枝压缩的整体流程是这样的，在此简单记录一下。 一、模型的剪枝与训练 NNI可实现多种剪枝算法的自动剪枝，训练。比如：SlimPruner，L1FilterPruner，L2FilterPruner，FPGMPruner，LevelPruner，AGP_Pruner等多种剪枝算法。 1.在自己的数据集上正常训练网络结构模型。 2.在训练结束后，调用NNI的模型剪
【深度学习经典网络架构—７】：MobileNet系列（V1、V2、V3）
王乐予的博客
03-23 4120
😺一、简述 自从AlexNet赢得ImageNet挑战赛：ILSVRC 2012以来，卷积神经网络已成为计算机视觉中的普遍应用。总体趋势是制作更深、更复杂的网络，以实现更高的准确度。然而，这些提高准确性的进步并不一定会使网络在规模和速度方面更有效率。在机器人、汽车自动驾驶和增强现实等许多现实世界的应用中，识别任务需要在计算受限的平台上及时执行。 谷歌提出MobileNet的目的就是**构造一个可以在移动端和嵌入式端应用深度学习的网络，使得在CPU上也能达到理想的速度要求。** MobileNet V1
MobileNetV3基于NNI剪枝操作
benben044的专栏
12-15 1285
原因，返回的数据不符合torch.jit.trace的要求，而示例model返回的是一个dict，它不是tensors | lists | tuples of tensors | dictionary of tensors中的一种。需要把MobileNetV3的代码拆分成两部分，一部分是核心代码，另一部分是根据业务进行view之后的相关代码。本文的剪枝操作针对CenterNet算法的BackBone，即MobileNetV3算法。根据核心代码的输出，只调整in_channel的值，不做剪枝处理。
模型剪枝-使用NNI框架进行推理测试
qq_37212703的博客
09-03 1222
使用NNI库进行模型剪枝推理速度对比。
MobileNet v1 结构详解
蓝羽飞鸟的博客
10-24 381
mobileNet v1结构详解
轻量级神经网络:MobileNetV1论文学习及理解
Jackkii的博客
11-26 1317
为了在某些真实的应用场景如移动或者嵌入式设备有效使用深度学习神经网络模型，使得在保证模型性能前提下，减小模型网络大小，提高运行速度，提出了可应用于算力有限平台的轻量级神经网络。
详细且通俗讲解轻量级神经网络——MobileNets【V1、V2、V3】
热门推荐
秃头小苏的博客
02-20 1万+
文章目录轻量级神经网络——MobileNetsMobileNetV1深度可分离卷积1、**深度卷积**✨✨✨2、**逐点卷积**✨✨✨参数量和计算量1、**标准卷积**2、**深度可分离卷积**✨✨✨MobileNetV1的网络结构及效果MobileNetV2Linear Bottlenecks✨✨✨Inverted Residuals✨✨✨MobileNetV2的网络结构及效果MobileNetV3新增SE模块✨✨✨重新设计耗时层结构重新设计激活函数✨✨✨MobileNetV3的网络结构及结果 轻量级神经
MobileNet v1 和 MobileNet v2
05-11
MobileNet v1和MobileNet v2都是Google开发的轻量级深度神经网络模型，用于在移动设备等资源受限环境下进行图像分类和目标检测任务。它们的主要区别在于： 1. 网络结构：MobileNet v2相对于MobileNet v1增加了一些新的结构，如倒残差和线性瓶颈等，使得网络性能有所提升。 2. 参数量：MobileNet v2相对于MobileNet v1的参数量有所增加，但是仍然比其他深度神经网络模型如VGG和ResNet要小很多，因此在移动设备等资源受限的环境下更具有优势。 3. 准确率：MobileNet v2相对于MobileNet v1在ImageNet数据集上的准确率有所提升，但是相对提升幅度并不是很大。 总的来说，MobileNet v2相对于MobileNet v1在性能上有所提升，但是也需要更多的计算资源和存储空间。选择哪个版本取决于具体应用场景和需求。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值