深度学习论文: An Energy and GPU-Computation Efficient Backbone Network for Object Detection及其PyTorch

最新推荐文章于 2023-02-11 13:35:51 发布

mingo_敏

最新推荐文章于 2023-02-11 13:35:51 发布

阅读量1.3k

点赞数 3

分类专栏： Object Detection Deep Learning Paper Reading 文章标签：深度学习 pytorch 机器学习

本文链接：https://blog.csdn.net/shanglianlm/article/details/106482678

版权

Deep Learning 同时被 3 个专栏收录

269 篇文章 39 订阅

订阅专栏

Paper Reading

230 篇文章 24 订阅

订阅专栏

Object Detection

66 篇文章 9 订阅

订阅专栏

An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection
PDF:https://arxiv.org/pdf/1904.09730.pdf
PyTorch: https://github.com/shanglianlm0525/PyTorch-Networks

1 概述

DenseNet用更少的参数与Flops而性能比ResNet更好，主要是因为concat比add能保留更多的信息。但是，实际上DenseNet却比ResNet要慢且消耗更多资源。

在这里插入图片描述
本文提出了One-short Aggregation，其实就是改变了shortcut的连接方式，从原来的前层feature与后面每一层都连接，变成了前层feature只与最后一层的连接。大大减少了连接数量。作者声称同样的结构，性能差不多，但是比DenseNet快2倍，energy consumption也减少了1.6~4.1倍。

2 Factors of Efficient Network Design(高效网络设计要素)

2-1 Memory Access Cost (MAC)

对于CNN，能耗在内存访问而不是计算上。
影响MAC的主要是是内存占用（intermediate activation memory footprint），它主要受卷积核和feature map大小的影响。

每个卷积层的MAC计算如下，
在这里插入图片描述
其中， h, w分别为高和宽，k为核尺寸, $c_{i}$ , $c_{o}$ 分别为输入和输出通道数。由上式有

其中 $B = k^{2}*h*w*c_{i}* c_{o}$
明显可以看出当输入和输出通道数相等时，MAC开销最小。

2-2 GPU-Computation Efficiency

GPU特性是擅长parallel computation，tensor越大，GPU使用效率越高。
把大的卷积操作拆分成碎片的小操作将不利于GPU计算。
设计layer数量少的网络是更好的选择。
1x1卷积来减少计算量，不过这不利于GPU计算。

3 Network

3-1 One-Shot Aggregation

提出OSA（one-shot-aggregation）模块。在OSA module中，每一层产生两种连接，一种是通过conv和下一层连接，产生receptive field 更大的feature map，另一种是和最后的输出层相连，以聚合足够好的特征。
通过使用OSA module，5层43channels的DenseNet-40的MAC可以被减少30%（3.7M -> 2.5M）

PyTorch代码：

class OSA_module(nn.Module):
    def __init__(self, in_channels,mid_channels, out_channels, block_nums=5):
        super(OSA_module, self).__init__()

        self._layers = nn.ModuleList()
        self._layers.append(Conv3x3BNReLU(in_channels=in_channels, out_channels=mid_channels, stride=1))
        for idx in range(block_nums-1):
            self._layers.append(Conv3x3BNReLU(in_channels=mid_channels, out_channels=mid_channels, stride=1))

        self.conv1x1 = Conv1x1BNReLU(in_channels+mid_channels*block_nums,out_channels)

    def forward(self, x):
        outputs = []
        outputs.append(x)
        for _layer in self._layers:
            x = _layer(x)
            outputs.append(x)
        out = torch.cat(outputs, dim=1)
        out = self.conv1x1(out)
        return out

3-2 VoVNet

提出的各种VoVNet结构
在这里插入图片描述
PyTorch代码：

# !/usr/bin/env python
# -- coding: utf-8 --
# @Time : 2020/6/1 14:40
# @Author : liumin
# @File : VoVNet.py

import torch
import torch.nn as nn
import torchvision

__all__ = ['VoVNet', 'vovnet27_slim', 'vovnet39', 'vovnet57']

from PIL.Image import Image


def Conv3x3BNReLU(in_channels,out_channels,stride,groups=1):
    return nn.Sequential(
            nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, stride=stride, padding=1,groups=groups, bias=False),
            nn.BatchNorm2d(out_channels),
            nn.ReLU6(inplace=True)
        )


def Conv3x3BN(in_channels,out_channels,stride,groups):
    return nn.Sequential(
            nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, stride=stride, padding=1,groups=groups, bias=False),
            nn.BatchNorm2d(out_channels)
        )


def Conv1x1BNReLU(in_channels,out_channels):
    return nn.Sequential(
            nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=1, bias=False),
            nn.BatchNorm2d(out_channels),
            nn.ReLU6(inplace=True)
        )


def Conv1x1BN(in_channels,out_channels):
    return nn.Sequential(
            nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=1, bias=False),
            nn.BatchNorm2d(out_channels)
        )

class OSA_module(nn.Module):
    def __init__(self, in_channels,mid_channels, out_channels, block_nums=5):
        super(OSA_module, self).__init__()

        self._layers = nn.ModuleList()
        self._layers.append(Conv3x3BNReLU(in_channels=in_channels, out_channels=mid_channels, stride=1))
        for idx in range(block_nums-1):
            self._layers.append(Conv3x3BNReLU(in_channels=mid_channels, out_channels=mid_channels, stride=1))

        self.conv1x1 = Conv1x1BNReLU(in_channels+mid_channels*block_nums,out_channels)

    def forward(self, x):
        outputs = []
        outputs.append(x)
        for _layer in self._layers:
            x = _layer(x)
            outputs.append(x)
        out = torch.cat(outputs, dim=1)
        out = self.conv1x1(out)
        return out


class VoVNet(nn.Module):
    def __init__(self, planes, layers, num_classes=2):
        super(VoVNet, self).__init__()

        self.groups = 1
        self.stage1 = nn.Sequential(
            Conv3x3BNReLU(in_channels=3, out_channels=64, stride=2, groups=self.groups),
            Conv3x3BNReLU(in_channels=64, out_channels=64, stride=1, groups=self.groups),
            Conv3x3BNReLU(in_channels=64, out_channels=128, stride=1, groups=self.groups),
        )

        self.stage2 = self._make_layer(planes[0][0],planes[0][1],planes[0][2],layers[0])

        self.stage3 = self._make_layer(planes[1][0],planes[1][1],planes[1][2],layers[1])

        self.stage4 = self._make_layer(planes[2][0],planes[2][1],planes[2][2],layers[2])

        self.stage5 = self._make_layer(planes[3][0],planes[3][1],planes[3][2],layers[3])

        self.avgpool = nn.AdaptiveAvgPool2d(output_size=1)
        self.flatten = nn.Flatten()
        self.dropout = nn.Dropout(p=0.2)
        self.linear = nn.Linear(in_features=planes[3][2], out_features=num_classes)

    def _make_layer(self, in_channels, mid_channels,out_channels, block_num):
        layers = []
        layers.append(nn.MaxPool2d(kernel_size=3, stride=2, padding=1))
        for idx in range(block_num):
            layers.append(OSA_module(in_channels=in_channels, mid_channels=mid_channels, out_channels=out_channels))
            in_channels = out_channels
        return nn.Sequential(*layers)

    def init_params(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight)
                if m.bias is not None:
                    nn.init.constant_(m.bias, 0)
            elif isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.Linear):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)

    def forward(self, x):
        x = self.stage1(x)
        x = self.stage2(x)
        x = self.stage3(x)
        x = self.stage4(x)
        x = self.stage5(x)
        x = self.avgpool(x)
        x = self.flatten(x)
        x = self.dropout(x)
        out = self.linear(x)
        return out

def vovnet27_slim(**kwargs):
    planes = [[128, 64, 128],
              [128, 80, 256],
              [256, 96, 384],
              [384, 112, 512]]
    layers = [1, 1, 1, 1]
    model = VoVNet(planes, layers)
    return model

def vovnet39(**kwargs):
    planes = [[128, 128, 256],
              [256, 160, 512],
              [512, 192, 768],
              [768, 224, 1024]]
    layers = [1, 1, 2, 2]
    model = VoVNet(planes, layers)
    return model

def vovnet57(**kwargs):
    planes = [[128, 128, 256],
              [256, 160, 512],
              [512, 192, 768],
              [768, 224, 1024]]
    layers = [1, 1, 4, 3]
    model = VoVNet(planes, layers)
    return model


if __name__=='__main__':
    model = vovnet27_slim()
    print(model)

    input = torch.randn(1, 3, 64, 64)
    out = model(input)
    print(out.shape)

mingo_敏

关注

3
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
深度学习论文: An Energy and GPU-Computation Efficient Backbone Network for Object Detection及其PyTorch

An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object DetectionPDF:https://arxiv.org/pdf/1904.09730.pdfPyTorch: https://github.com/shanglianlm0525/PyTorch-Networks1 概述DenseNet用更少的参数与Flops而性能比ResNet更好，主要是因为concat比add能保留更多的信息。但是，实
复制链接

扫一扫