yolov7使用MobileNetV3主干网络加速

aiwase

已于 2024-02-18 12:44:37 修改

阅读量1.3k

点赞数 30

分类专栏：计算机视觉学习记录文章标签：深度学习 YOLO 计算机视觉神经网络

于 2024-02-17 10:58:27 首次发布

本文链接：https://blog.csdn.net/qq_74147688/article/details/136136206

版权

计算机视觉学习记录专栏收录该内容

1 篇文章 0 订阅

订阅专栏

yolov7使用MobileNetV3主干网络加速

本文为博主自行摸索，如有错误欢迎指出。

创建配置文件

在cfg/training文件夹下，复制yolov7.yaml，在当前文件夹下创建其副本，我命名为yolov7-MobileNetV3.yaml，这里可以随意命名。

修改配置文件

YOLOV7的网络结构分为两个部分。骨干网络（Backbone）和检测头（Detection Head）

以下是yolov7-MobileNetV3.yaml的内容。

注意要修改nc，nc是你的目标有几类。

在修改网络的时候，要注意输入输出保持一致

骨干网络MobileNetV3参数

在MobileNetV3中，[ 16, 16, 3, 1, 0, 0 ]从前往后为

oup, hidden_dim, kernel_size, stride, use_se, use_hs

即MobileNet_Block层的参数去掉inp

这里六个参数分别指输出通道数，扩张通道数，卷积核大小，步长，是否使用se，是否使用hs

hs为h_swish，是一种在sigmoid基础山修改得到的激活函数

猜测没有填入inp参数的原因是[ -1, 1, MobileNet_Block, [ 16, 16, 3, 1, 0, 0 ] ]第一个参数-1表示从上一层获取输入

检测头yolov7参数

可以参考yolov7.yaml文件详解_yolo的yaml参数含义-CSDN博客

[from, number, module, args]

from表示输入层的位置，args第一个参数为输入通道数，这里要保证输入层输出的通道数与当前层的输入一致。

代码

backbone主干网络为网络上的提供的MobileNetV3代码，p在这里认为是池化层的意思

# parameters
nc: 2  # number of classes
depth_multiple: 1.0  # model depth multiple
width_multiple: 1.0  # layer channel multiple

# anchors
anchors:
  - [12,16, 19,36, 40,28]  # P3/8
  - [36,75, 76,55, 72,146]  # P4/16
  - [142,110, 192,243, 459,401]  # P5/32

# yolov7 backbone
backbone:
  # [from, number, module, args] c2, k=1, s=1, p=None, g=1, act=True
  [ [ -1, 1, conv_bn_hswish, [ 16, 2 ] ],
    [ -1, 1, MobileNet_Block, [ 16,  16, 3, 1, 0, 0 ] ],
    [ -1, 1, MobileNet_Block, [ 24,  64, 3, 2, 0, 0 ] ],  # p2
    [ -1, 1, MobileNet_Block, [ 24,  72, 3, 1, 0, 0 ] ],
    [ -1, 1, MobileNet_Block, [ 40,  72, 5, 2, 1, 0 ] ], # p3
    [ -1, 1, MobileNet_Block, [ 40,  120, 5, 1, 1, 0 ] ],
    [ -1, 1, MobileNet_Block, [ 40, 120, 5, 1, 1, 0 ] ],# 6
    [ -1, 1, MobileNet_Block, [ 80, 240, 3, 2, 0, 1 ] ], # p4
    [ -1, 1, MobileNet_Block, [ 80, 200, 3, 1, 0, 1 ] ],
    [ -1, 1, MobileNet_Block, [ 80, 184, 3, 1, 0, 1 ] ],
    [ -1, 1, MobileNet_Block, [ 80, 184, 3, 1, 0, 1 ] ],
    [ -1, 1, MobileNet_Block, [ 112, 480, 3, 1, 1, 1 ] ],
    [ -1, 1, MobileNet_Block, [ 112, 672, 3, 1, 1, 1 ] ],# 12
    [ -1, 1, MobileNet_Block, [ 160,  672, 5, 2, 1, 1 ] ],#p5
    [ -1, 1, MobileNet_Block, [ 160,  960, 5, 1, 1, 1 ] ],
    [ -1, 1, MobileNet_Block, [ 160,  960, 5, 1, 1, 1 ] ],#15
  ]
# yolov7 head
head:
  [[-1, 1, SPPCSPC, [512]], # 16

   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [12, 1, Conv, [256, 1, 1]], # route backbone P4
   [[-1, -2], 1, Concat, [1]],

   [-1, 1, Conv, [256, 1, 1]],
   [-2, 1, Conv, [256, 1, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [[-1, -2, -3, -4, -5, -6], 1, Concat, [1]],
   [-1, 1, Conv, [256, 1, 1]], # 28

   [-1, 1, Conv, [128, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [6, 1, Conv, [128, 1, 1]], # route backbone P3
   [[-1, -2], 1, Concat, [1]],

   [-1, 1, Conv, [128, 1, 1]],
   [-2, 1, Conv, [128, 1, 1]],
   [-1, 1, Conv, [64, 3, 1]],
   [-1, 1, Conv, [64, 3, 1]],
   [-1, 1, Conv, [64, 3, 1]],
   [-1, 1, Conv, [64, 3, 1]],
   [[-1, -2, -3, -4, -5, -6], 1, Concat, [1]],
   [-1, 1, Conv, [128, 1, 1]], # 40

   [-1, 1, MP, []],
   [-1, 1, Conv, [128, 1, 1]],
   [-3, 1, Conv, [128, 1, 1]],
   [-1, 1, Conv, [128, 3, 2]],
   [[-1, -3, 28], 1, Concat, [1]],

   [-1, 1, Conv, [256, 1, 1]],
   [-2, 1, Conv, [256, 1, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [-1, 1, Conv, [128, 3, 1]],
   [[-1, -2, -3, -4, -5, -6], 1, Concat, [1]],
   [-1, 1, Conv, [256, 1, 1]], # 53

   [-1, 1, MP, []],
   [-1, 1, Conv, [256, 1, 1]],
   [-3, 1, Conv, [256, 1, 1]],
   [-1, 1, Conv, [256, 3, 2]],
   [[-1, -3, 16], 1, Concat, [1]],

   [-1, 1, Conv, [512, 1, 1]],
   [-2, 1, Conv, [512, 1, 1]],
   [-1, 1, Conv, [256, 3, 1]],
   [-1, 1, Conv, [256, 3, 1]],
   [-1, 1, Conv, [256, 3, 1]],
   [-1, 1, Conv, [256, 3, 1]],
   [[-1, -2, -3, -4, -5, -6], 1, Concat, [1]],
   [-1, 1, Conv, [512, 1, 1]], # 66

   [40, 1, RepConv, [256, 3, 1]],
   [53, 1, RepConv, [512, 3, 1]],
   [66, 1, RepConv, [1024, 3, 1]],

   [[67,68,69], 1, IDetect, [nc, anchors]],   # Detect(P3, P4, P5)
  ]

修改model软件包下的common.py

这里是为了修改网络层的定义

此处来源于目标检测算法——YOLOv5/YOLOv7改进之结合轻量化网络MobileNetV3（降参提速）_将去噪与yolo目标检测连结-CSDN博客

#——————MobileNetV3——————
 
class h_sigmoid(nn.Module):
    def __init__(self, inplace=True):
        super(h_sigmoid, self).__init__()
        self.relu = nn.ReLU6(inplace=inplace)
 
    def forward(self, x):
        return self.relu(x + 3) / 6
 
 
class h_swish(nn.Module):
    def __init__(self, inplace=True):
        super(h_swish, self).__init__()
        self.sigmoid = h_sigmoid(inplace=inplace)
 
    def forward(self, x):
        return x * self.sigmoid(x)
 
 
class SELayer(nn.Module):
    def __init__(self, channel, reduction=4):
        super(SELayer, self).__init__()
        # Squeeze操作
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        # Excitation操作(FC+ReLU+FC+Sigmoid)
        self.fc = nn.Sequential(
            nn.Linear(channel, channel // reduction),
            nn.ReLU(inplace=True),
            nn.Linear(channel // reduction, channel),
            h_sigmoid()
        )
 
    def forward(self, x):
        b, c, _, _ = x.size()
        y = self.avg_pool(x)
        y = y.view(b, c)
        y = self.fc(y).view(b, c, 1, 1)  # 学习到的每一channel的权重
        return x * y
 
 
class conv_bn_hswish(nn.Module):
    """
    This equals to
    def conv_3x3_bn(inp, oup, stride):
        return nn.Sequential(
            nn.Conv2d(inp, oup, 3, stride, 1, bias=False),
            nn.BatchNorm2d(oup),
            h_swish()
        )
    """
 
    def __init__(self, c1, c2, stride):
        super(conv_bn_hswish, self).__init__()
        self.conv = nn.Conv2d(c1, c2, 3, stride, 1, bias=False)
        self.bn = nn.BatchNorm2d(c2)
        self.act = h_swish()
 
    def forward(self, x):
        return self.act(self.bn(self.conv(x)))
 
    def fuseforward(self, x):
        return self.act(self.conv(x))
 
 
class MobileNet_Block(nn.Module):
    def __init__(self, inp, oup, hidden_dim, kernel_size, stride, use_se, use_hs):
        super(MobileNet_Block, self).__init__()
        assert stride in [1, 2]
 
        self.identity = stride == 1 and inp == oup
 
        # 输入通道数=扩张通道数 则不进行通道扩张
        if inp == hidden_dim:
            self.conv = nn.Sequential(
                # dw
                nn.Conv2d(hidden_dim, hidden_dim, kernel_size, stride, (kernel_size - 1) // 2, groups=hidden_dim,
                          bias=False),
                nn.BatchNorm2d(hidden_dim),
                h_swish() if use_hs else nn.ReLU(inplace=True),
                # Squeeze-and-Excite
                SELayer(hidden_dim) if use_se else nn.Sequential(),
                # pw-linear
                nn.Conv2d(hidden_dim, oup, 1, 1, 0, bias=False),
                nn.BatchNorm2d(oup),
            )
        else:
            # 否则 先进行通道扩张
            self.conv = nn.Sequential(
                # pw
                nn.Conv2d(inp, hidden_dim, 1, 1, 0, bias=False),
                nn.BatchNorm2d(hidden_dim),
                h_swish() if use_hs else nn.ReLU(inplace=True),
                # dw
                nn.Conv2d(hidden_dim, hidden_dim, kernel_size, stride, (kernel_size - 1) // 2, groups=hidden_dim,
                          bias=False),
                nn.BatchNorm2d(hidden_dim),
                # Squeeze-and-Excite
                SELayer(hidden_dim) if use_se else nn.Sequential(),
                h_swish() if use_hs else nn.ReLU(inplace=True),
                # pw-linear
                nn.Conv2d(hidden_dim, oup, 1, 1, 0, bias=False),
                nn.BatchNorm2d(oup),
            )
 
    def forward(self, x):
        y = self.conv(x)
        if self.identity:
            return x + y
        else:
            return y

修改model软件包下的yolo.py

找到parse_model函数，加入h_sigmoid， h_swish，SELayer，conv_bn_hswish， MobileNet_Block等5个模块即可。（我的在758行，找不到可以从下边辅助几个词搜索一下）

if m in [nn.Conv2d, Conv, RobustConv, RobustConv2, DWConv, GhostConv, RepConv, RepConv_OREPA, DownC,
         SPP, SPPF, SPPCSPC, GhostSPPCSPC, MixConv2d, Focus, Stem, GhostStem, CrossConv,
         Bottleneck, BottleneckCSPA, BottleneckCSPB, BottleneckCSPC,
         RepBottleneck, RepBottleneckCSPA, RepBottleneckCSPB, RepBottleneckCSPC,
         Res, ResCSPA, ResCSPB, ResCSPC,
         RepRes, RepResCSPA, RepResCSPB, RepResCSPC,
         ResX, ResXCSPA, ResXCSPB, ResXCSPC,
         RepResX, RepResXCSPA, RepResXCSPB, RepResXCSPC,
         Ghost, GhostCSPA, GhostCSPB, GhostCSPC,
         SwinTransformerBlock, STCSPA, STCSPB, STCSPC,
         SwinTransformer2Block, ST2CSPA, ST2CSPB, ST2CSPC, h_sigmoid, h_swish, SELayer, conv_bn_hswish,
         MobileNet_Block]:

训练

这里给出我的训练指令，修改–cfg cfg/training/yolov7-MobileNetV3.yaml即可，可以参考

python train.py --workers 2 --device 0 --batch-size 8 --img-size 416 --data data/Detection.v2_data.yaml --cfg cfg/training/yolov7-MobileNetV3.yaml --name yolov7-MobileNetV3-rgb-epochs100 --hyp data/hyp.scratch.p5.yaml --epochs=100