秋招面试专栏推荐 :深度学习算法工程师面试问题总结【百面算法工程师】——点击即可跳转
💡💡💡本专栏所有程序均经过测试,可成功执行💡💡💡
专栏目录 :《YOLOv8改进有效涨点》专栏介绍 & 专栏目录 | 目前已有100+篇内容,内含各种Head检测头、损失函数Loss、Backbone、Neck、NMS等创新点改进——点击即可跳转
视觉变换器(ViT)近年来在计算机视觉领域受到了越来越多的关注。然而,ViT的核心组件——自注意力,缺乏显式的空间先验,并且具有二次计算复杂度,这限制了ViT的应用范围。为了缓解这些问题,研究人员从自然语言处理领域的最新研究成果——RetNet中获得灵感,提出了RMT,本文将介绍C2f与RetBlcok的融合改进。文章在介绍主要的原理后,将手把手教学如何进行模块的代码添加和修改,并将修改后的完整代码放在文章的最后,方便大家一键运行,小白也可轻松上手实践。以帮助您更好地学习深度学习目标检测YOLO系列的挑战。
目录
1. 原理
论文地址:RMT: Retentive Networks Meet Vision Transformers——点击即可跳转
官方代码:官方代码仓库——点击即可跳转
RetNet(Retentive Network)的主要原理是引入一个基于距离的衰减机制,用于序列建模。这一机制最初应用于自然语言处理(NLP)任务,提供了显式的时序先验。RetNet通过一种递归的方式来处理序列,并在模型中引入了时间衰减矩阵,使模型能够感知序列中不同元素的相对距离。
在计算机视觉领域中,RetNet的这一衰减机制被扩展为空间衰减,用于建模图像中的空间关系。这种空间衰减机制通过曼哈顿距离定义,允许目标token感知全局信息,同时对不同距离的token赋予不同的注意力权重。该模型的关键是通过自注意力机制中的空间衰减矩阵,显式引入空间先验,从而增强模型对空间信息的捕获能力。此外,RetNet通过一种分解形式来实现全局信息建模,同时保持空间衰减矩阵的完整性,从而降低了计算复杂度。
这种改进后的模型(例如文中提到的RMT,Retentive Networks Meet Vision Transformers)结合了线性复杂度和显式的空间先验,展现了在图像分类、目标检测、实例分割等多个视觉任务上的卓越性能。
2. 将C2f_RetBlock添加到yolov8网络中
2.1 C2f_RetBlock代码实现
关键步骤一: 将下面代码粘贴到在/ultralytics/ultralytics/nn/modules/block.py中,并在该文件的__all__中添加“C2f_RetBlock”
from typing import Tuple
from timm.models.layers import DropPath
class DWConv2d(nn.Module):
def __init__(self, dim, kernel_size, stride, padding):
super().__init__()
self.conv = nn.Conv2d(dim, dim, kernel_size, stride, padding, groups=dim)
def forward(self, x: torch.Tensor):
'''
x: (b h w c)
'''
x = x.permute(0, 3, 1, 2) #(b c h w)
x = self.conv(x) #(b c h w)
x = x.permute(0, 2, 3, 1) #(b h w c)
return x
class MaSAd(nn.Module):
def __init__(self, embed_dim, num_heads, value_factor=1):
super().__init__()
self.factor = value_factor
self.embed_dim = embed_dim
self.num_heads = num_heads
self.head_dim = self.embed_dim * self.factor // num_heads
self.key_dim = self.embed_dim // num_heads
self.scaling = self.key_dim ** -0.5
self.q_proj = nn.Linear(embed_dim, embed_dim, bias=True)
self.k_proj = nn.Linear(embed_dim, embed_dim, bias=True)
self.v_proj = nn.Linear(embed_dim, embed_dim * self.factor, bias=True)
self.lepe = DWConv2d(embed_dim, 5, 1, 2)
self.out_proj = nn.Linear(embed_dim*self.factor, embed_dim, bias=True)
self.reset_parameters()
def reset_parameters(self):
nn.init.xavier_normal_(self.q_proj.weight, gain=2 ** -2.5)
nn.init.xavier_normal_(self.k_proj.weight, gain=2 ** -2.5)
nn.init.xavier_normal_(self.v_proj.weight, gain=2 ** -2.5)
nn.init.xavier_normal_(self.out_proj.weight)
nn.init.constant_(self.out_proj.bias, 0.0)
def forward(self, x: torch.Tensor, rel_pos, chunkwise_recurrent=False, incremental_state=None):
'''
x: (b h w c)
mask_h: (n h h)
mask_w: (n w w)
'''
bsz, h, w, _ = x.size()
mask_h, mask_w = rel_pos
q = self.q_proj(x)
k = self.k_proj(x)
v = self.v_proj(x)
lepe = self.lepe(v)
k *= self.scaling
qr = q.view(bsz, h, w, self.num_heads, self.key_dim).permute(0, 3, 1, 2, 4) #(b n h w d1)
kr = k.view(bsz, h, w, self.num_heads, self.key_dim).permute(0, 3, 1, 2, 4) #(b n h w d1)
'''
qr: (b n h w d1)
kr: (b n h w d1)
v: (b h w n*d2)
'''
qr_w = qr.transpose(1, 2) #(b h n w d1)
kr_w = kr.transpose(1, 2) #(b h n w d1)
v = v.reshape(bsz, h, w, self.num_heads, -1).permute(0, 1, 3, 2, 4) #(b h n w d2)
qk_mat_w = qr_w @ kr_w.transpose(-1, -2) #(b h n w w)
qk_mat_w = qk_mat_w + mask_w #(b h n w w)
qk_mat_w = torch.softmax(qk_mat_w, -1) #(b h n w w)
v = torch.matmul(qk_mat_w, v) #(b h n w d2)
qr_h = qr.permute(0, 3, 1, 2, 4) #(b w n h d1)
kr_h = kr.permute(0, 3, 1, 2, 4) #(b w n h d1)
v = v.permute(0, 3, 2, 1, 4) #(b w n h d2)
qk_mat_h = qr_h @ kr_h.transpose(-1, -2) #(b w n h h)
qk_mat_h = qk_mat_h + mask_h #(b w n h h)
qk_mat_h = torch.softmax(qk_mat_h, -1) #(b w n h h)
output = torch.matmul(qk_mat_h, v) #(b w n h d2)
output = output.permute(0, 3, 1, 2, 4).flatten(-2, -1) #(b h w n*d2)
output = output + lepe
output = self.out_proj(output)
return output
class FeedForwardNetwork(nn.Module):
def __init__(
self,
embed_dim,
ffn_dim,
activation_fn=F.gelu,
dropout=0.0,
activation_dropout=0.0,
layernorm_eps=1e-6,
subln=False,
subconv=False
):
super().__init__()
self.embed_dim = embed_dim
self.activation_fn = activation_fn
self.activation_dropout_module = torch.nn.Dropout(activation_dropout)
self.dropout_module = torch.nn.Dropout(dropout)
self.fc1 = nn.Linear(self.embed_dim, ffn_dim)
self.fc2 = nn.Linear(ffn_dim, self.embed_dim)
self.ffn_layernorm = nn.LayerNorm(ffn_dim, eps=layernorm_eps) if subln else None
self.dwconv = DWConv2d(ffn_dim, 3, 1, 1) if subconv else None
def reset_parameters(self):
self.fc1.reset_parameters()
self.fc2.reset_parameters()
if self.ffn_layernorm is not None:
self.ffn_layernorm.reset_parameters()
def forward(self, x: torch.Tensor):
'''
x: (b h w c)
'''
x = self.fc1(x)
x = self.activation_fn(x)
x = self.activation_dropout_module(x)
if self.dwconv is not None:
residual = x
x = self.dwconv(x)
x = x + residual
if self.ffn_layernorm is not None:
x = self.ffn_layernorm(x)
x = self.fc2(x)
x = self.dropout_module(x)
return x
class MaSA(nn.Module):
def __init__(self, embed_dim, num_heads, value_factor=1):
super().__init__()
self.factor = value_factor
self.embed_dim = embed_dim
self.num_heads = num_heads
self.head_dim = self.embed_dim * self.factor // num_heads
self.key_dim = self.embed_dim // num_heads
self.scaling = self.key_dim ** -0.5
self.q_proj = nn.Linear(embed_dim, embed_dim, bias=True)
self.k_proj = nn.Linear(embed_dim, embed_dim, bias=True)
self.v_proj = nn.Linear(embed_dim, embed_dim * self.factor, bias=True)
self.lepe = DWConv2d(embed_dim, 5, 1, 2)
self.out_proj = nn.Linear(embed_dim*self.factor, embed_dim, bias=True)
self.reset_parameters()
def reset_parameters(self):
nn.init.xavier_normal_(self.q_proj.weight, gain=2 ** -2.5)
nn.init.xavier_normal_(self.k_proj.weight, gain=2 ** -2.5)
nn.init.xavier_normal_(self.v_proj.weight, gain=2 ** -2.5)
nn.init.xavier_normal_(self.out_proj.weight)
nn.init.constant_(self.out_proj.bias, 0.0)
def forward(self, x: torch.Tensor, rel_pos, chunkwise_recurrent=False, incremental_state=None):
'''
x: (b h w c)
rel_pos: mask: (n l l)
'''
bsz, h, w, _ = x.size()
mask = rel_pos
assert h*w == mask.size(1)
q = self.q_proj(x)
k = self.k_proj(x)
v = self.v_proj(x)
lepe = self.lepe(v)
k *= self.scaling
qr = q.view(bsz, h, w, self.num_heads, -1).permute(0, 3, 1, 2, 4) #(b n h w d1)
kr = k.view(bsz, h, w, self.num_heads, -1).permute(0, 3, 1, 2, 4) #(b n h w d1)
qr = qr.flatten(2, 3) #(b n l d1)
kr = kr.flatten(2, 3) #(b n l d1)
vr = v.reshape(bsz, h, w, self.num_heads, -1).permute(0, 3, 1, 2, 4) #(b n h w d2)
vr = vr.flatten(2, 3) #(b n l d2)
qk_mat = qr @ kr.transpose(-1, -2) #(b n l l)
qk_mat = qk_mat + mask #(b n l l)
qk_mat = torch.softmax(qk_mat, -1) #(b n l l)
output = torch.matmul(qk_mat, vr) #(b n l d2)
output = output.transpose(1, 2).reshape(bsz, h, w, -1) #(b h w n*d2)
output = output + lepe
output = self.out_proj(output)
return output
class RetBlock(nn.Module):
def __init__(self, retention: str, embed_dim: int, num_heads: int, ffn_dim: int, drop_path=0., layerscale=False, layer_init_values=1e-5):
super().__init__()
self.layerscale = layerscale
self.embed_dim = embed_dim
self.retention_layer_norm = nn.LayerNorm(self.embed_dim, eps=1e-6)
assert retention in ['chunk', 'whole']
if retention == 'chunk':
self.retention = MaSAd(embed_dim, num_heads)
else:
self.retention = MaSA(embed_dim, num_heads)
self.drop_path = DropPath(drop_path)
self.final_layer_norm = nn.LayerNorm(self.embed_dim, eps=1e-6)
self.ffn = FeedForwardNetwork(embed_dim, ffn_dim)
self.pos = DWConv2d(embed_dim, 3, 1, 1)
if layerscale:
self.gamma_1 = nn.Parameter(layer_init_values * torch.ones(1, 1, 1, embed_dim),requires_grad=True)
self.gamma_2 = nn.Parameter(layer_init_values * torch.ones(1, 1, 1, embed_dim),requires_grad=True)
def forward(
self,
x: torch.Tensor,
incremental_state=None,
chunkwise_recurrent=False,
retention_rel_pos=None
):
x = x + self.pos(x)
if self.layerscale:
x = x + self.drop_path(self.gamma_1 * self.retention(self.retention_layer_norm(x), retention_rel_pos, chunkwise_recurrent, incremental_state))
x = x + self.drop_path(self.gamma_2 * self.ffn(self.final_layer_norm(x)))
else:
x = x + self.drop_path(self.retention(self.retention_layer_norm(x), retention_rel_pos, chunkwise_recurrent, incremental_state))
x = x + self.drop_path(self.ffn(self.final_layer_norm(x)))
return x
class PatchMerging(nn.Module):
r""" Patch Merging Layer.
Args:
input_resolution (tuple[int]): Resolution of input feature.
dim (int): Number of input channels.
norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm
"""
def __init__(self, dim, out_dim, norm_layer=nn.LayerNorm):
super().__init__()
self.dim = dim
self.reduction = nn.Conv2d(dim, out_dim, 3, 2, 1)
self.norm = nn.BatchNorm2d(out_dim)
def forward(self, x):
'''
x: B H W C
'''
x = x.permute(0, 3, 1, 2).contiguous() #(b c h w)
x = self.reduction(x) #(b oc oh ow)
x = self.norm(x)
x = x.permute(0, 2, 3, 1) #(b oh ow oc)
return x
class RelPos2d(nn.Module):
def __init__(self, embed_dim, num_heads, initial_value, heads_range):
'''
recurrent_chunk_size: (clh clw)
num_chunks: (nch ncw)
clh * clw == cl
nch * ncw == nc
default: clh==clw, clh != clw is not implemented
'''
super().__init__()
angle = 1.0 / (10000 ** torch.linspace(0, 1, embed_dim // num_heads // 2))
angle = angle.unsqueeze(-1).repeat(1, 2).flatten()
self.initial_value = initial_value
self.heads_range = heads_range
self.num_heads = num_heads
decay = torch.log(1 - 2 ** (-initial_value - heads_range * torch.arange(num_heads, dtype=torch.float) / num_heads))
self.register_buffer('angle', angle)
self.register_buffer('decay', decay)
def generate_2d_decay(self, H: int, W: int):
'''
generate 2d decay mask, the result is (HW)*(HW)
'''
index_h = torch.arange(H).to(self.decay)
index_w = torch.arange(W).to(self.decay)
grid = torch.meshgrid([index_h, index_w])
grid = torch.stack(grid, dim=-1).reshape(H*W, 2) #(H*W 2)
mask = grid[:, None, :] - grid[None, :, :] #(H*W H*W 2)
mask = (mask.abs()).sum(dim=-1)
mask = mask * self.decay[:, None, None] #(n H*W H*W)
return mask
def generate_1d_decay(self, l: int):
'''
generate 1d decay mask, the result is l*l
'''
index = torch.arange(l).to(self.decay)
mask = index[:, None] - index[None, :] #(l l)
mask = mask.abs() #(l l)
mask = mask * self.decay[:, None, None] #(n l l)
return mask
def forward(self, slen: Tuple[int], activate_recurrent=False, chunkwise_recurrent=False):
'''
slen: (h, w)
h * w == l
recurrent is not implemented
'''
if activate_recurrent:
retention_rel_pos = self.decay.exp()
elif chunkwise_recurrent:
mask_h = self.generate_1d_decay(slen[0])
mask_w = self.generate_1d_decay(slen[1])
retention_rel_pos = (mask_h, mask_w)
else:
mask = self.generate_2d_decay(slen[0], slen[1]) #(n l l)
retention_rel_pos = mask
return retention_rel_pos
class C3_RetBlock(C3):
def __init__(self, c1, c2, n=1, retention='chunk', num_heads=8, shortcut=False, g=1, e=0.5):
super().__init__(c1, c2, n, shortcut, g, e)
c_ = int(c2 * e) # hidden channels
self.retention = retention
self.Relpos = RelPos2d(c_, num_heads, 2, 4)
self.m = nn.Sequential(*(RetBlock(retention, c_, num_heads, c_) for _ in range(n)))
def forward(self, x):
"""Forward pass through the CSP bottleneck with 2 convolutions."""
b, c, h, w = x.size()
rel_pos = self.Relpos((h, w), chunkwise_recurrent=self.retention == 'chunk')
cv1 = self.cv1(x)
for idx, layer in enumerate(self.m):
if idx == 0:
cv1 = layer(cv1.permute(0, 2, 3, 1), None, self.retention == 'chunk', rel_pos)
else:
cv1 = layer(cv1, None, self.retention == 'chunk', rel_pos)
cv2 = self.cv2(x)
return self.cv3(torch.cat((cv1.permute(0, 3, 1, 2), cv2), 1))
class C2f_RetBlock(C2f):
def __init__(self, c1, c2, n=1, retention='chunk', num_heads=8, shortcut=False, g=1, e=0.5):
super().__init__(c1, c2, n, shortcut, g, e)
self.retention = retention
self.Relpos = RelPos2d(self.c, num_heads, 2, 4)
self.m = nn.ModuleList(RetBlock(retention, self.c, num_heads, self.c) for _ in range(n))
def forward(self, x):
"""Forward pass through C2f layer."""
b, c, h, w = x.size()
rel_pos = self.Relpos((h, w), chunkwise_recurrent=self.retention == 'chunk')
y = list(self.cv1(x).chunk(2, 1))
for layer in self.m:
y.append(layer(y[-1].permute(0, 2, 3, 1), None, self.retention == 'chunk', rel_pos).permute(0, 3, 1, 2))
return self.cv2(torch.cat(y, 1))
2.2 C2f_RetBlock的神经网络模块代码解析
C2f_RetBlock
继承自 C2f
类,并结合了 RetNet 的注意力机制(Retention Mechanism)。
代码进行详细解析:
-
__init__
初始化方法:-
c1
,c2
: 输入和输出通道的数量。 -
n
:RetBlock
的数量。 -
retention
: 控制 retention 机制的类型,默认为'chunk'
(分块)。 -
num_heads
: 多头自注意力机制的头数。 -
shortcut
,g
,e
: 控制残差连接、组卷积和扩展比等。 -
Relpos
: 实例化RelPos2d
,用于计算相对位置编码,它接受特征维度self.c
(通常为输出通道数c2
)、注意力头数num_heads
、以及额外的参数。 -
self.m
: 使用RetBlock
模块构建一个nn.ModuleList
,每个RetBlock
块使用 retention 机制与多头注意力。
-
-
forward
前向传播:-
b, c, h, w = x.size()
: 获取输入张量x
的批量大小、通道数、高度和宽度。 -
rel_pos = self.Relpos((h, w), chunkwise_recurrent=self.retention == 'chunk')
: 计算相对位置编码,基于输入的空间尺寸(h, w)
,并根据self.retention
是否为'chunk'
决定是否以分块方式进行计算。 -
y = list(self.cv1(x).chunk(2, 1))
: 对输入x
进行卷积操作后,沿通道维度将输出分块成两个张量,并将结果存储到列表y
中。 -
for layer in self.m
: 遍历每个RetBlock
,将y[-1]
(即y
列表中的最新张量)进行转置(通道移到最后),并传入RetBlock
。输出再次转置,恢复原通道维度,并追加到y
列表。 -
return self.cv2(torch.cat(y, 1))
: 将y
列表中的所有张量在通道维度上拼接,并通过卷积层cv2
输出最终结果。
-
总结: 这个模块结合了 RetNet 的保留机制(Retention Mechanism)和卷积操作,通过多层 RetBlock
来处理输入张量,同时利用相对位置编码增强空间信息感知。
2.3 更改init.py文件
关键步骤二:修改modules文件夹下的__init__.py文件,先导入函数
然后在下面的__all__中声明函数
2.4 添加yaml文件
关键步骤三:在/ultralytics/ultralytics/cfg/models/v8下面新建文件yolov8_C2f_RetBlock.yaml文件,粘贴下面的内容
- OD【目标检测】
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs
s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs
m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs
l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs
x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs
# YOLOv8.0n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, Conv, [64, 3, 2]] # 0-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 1-P2/4
- [-1, 3, C2f_RetBlock, [128, 'chunk', 8, True]]
- [-1, 1, Conv, [256, 3, 2]] # 3-P3/8
- [-1, 6, C2f_RetBlock, [256, 'chunk', 8, True]]
- [-1, 1, Conv, [512, 3, 2]] # 5-P4/16
- [-1, 6, C2f_RetBlock, [512, 'whole', 8, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 7-P5/32
- [-1, 3, C2f_RetBlock, [1024, 'whole', 8, True]]
- [-1, 1, SPPF, [1024, 5]] # 9
# YOLOv8.0n head
head:
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 6], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 12
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 4], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 15 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 12], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 18 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 9], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 21 (P5/32-large)
- [[15, 18, 21], 1, Detect, [nc]] # Detect(P3, P4, P5)
- Seg【语义分割】
# Ultralytics YOLO 🚀, AGPL-3.0 license
# YOLOv8 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect
# Parameters
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'
# [depth, width, max_channels]
n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200 parameters, 3157184 gradients, 8.9 GFLOPs
s: [0.33, 0.50, 1024] # YOLOv8s summary: 225 layers, 11166560 parameters, 11166544 gradients, 28.8 GFLOPs
m: [0.67, 0.75, 768] # YOLOv8m summary: 295 layers, 25902640 parameters, 25902624 gradients, 79.3 GFLOPs
l: [1.00, 1.00, 512] # YOLOv8l summary: 365 layers, 43691520 parameters, 43691504 gradients, 165.7 GFLOPs
x: [1.00, 1.25, 512] # YOLOv8x summary: 365 layers, 68229648 parameters, 68229632 gradients, 258.5 GFLOPs
# YOLOv8.0n backbone
backbone:
# [from, repeats, module, args]
- [-1, 1, Conv, [64, 3, 2]] # 0-P1/2
- [-1, 1, Conv, [128, 3, 2]] # 1-P2/4
- [-1, 3, C2f_RetBlock, [128, 'chunk', 8, True]]
- [-1, 1, Conv, [256, 3, 2]] # 3-P3/8
- [-1, 6, C2f_RetBlock, [256, 'chunk', 8, True]]
- [-1, 1, Conv, [512, 3, 2]] # 5-P4/16
- [-1, 6, C2f_RetBlock, [512, 'whole', 8, True]]
- [-1, 1, Conv, [1024, 3, 2]] # 7-P5/32
- [-1, 3, C2f_RetBlock, [1024, 'whole', 8, True]]
- [-1, 1, SPPF, [1024, 5]] # 9
# YOLOv8.0n head
head:
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 6], 1, Concat, [1]] # cat backbone P4
- [-1, 3, C2f, [512]] # 12
- [-1, 1, nn.Upsample, [None, 2, 'nearest']]
- [[-1, 4], 1, Concat, [1]] # cat backbone P3
- [-1, 3, C2f, [256]] # 15 (P3/8-small)
- [-1, 1, Conv, [256, 3, 2]]
- [[-1, 12], 1, Concat, [1]] # cat head P4
- [-1, 3, C2f, [512]] # 18 (P4/16-medium)
- [-1, 1, Conv, [512, 3, 2]]
- [[-1, 9], 1, Concat, [1]] # cat head P5
- [-1, 3, C2f, [1024]] # 21 (P5/32-large)
- [[15, 18, 21], 1, Segment, [nc, 32, 256]] # Segment(P3, P4, P5)
温馨提示:因为本文只是对yolov8基础上添加模块,如果要对yolov8n/l/m/x进行添加则只需要指定对应的depth_multiple 和 width_multiple。不明白的同学可以看这篇文章: yolov8yaml文件解读——点击即可跳转
# YOLOv8n
depth_multiple: 0.33 # model depth multiple
width_multiple: 0.25 # layer channel multiple
max_channels: 1024 # max_channels
# YOLOv8s
depth_multiple: 0.33 # model depth multiple
width_multiple: 0.50 # layer channel multiple
max_channels: 1024 # max_channels
# YOLOv8l
depth_multiple: 1.0 # model depth multiple
width_multiple: 1.0 # layer channel multiple
max_channels: 512 # max_channels
# YOLOv8m
depth_multiple: 0.67 # model depth multiple
width_multiple: 0.75 # layer channel multiple
max_channels: 768 # max_channels
# YOLOv8x
depth_multiple: 1.33 # model depth multiple
width_multiple: 1.25 # layer channel multiple
max_channels: 512 # max_channels
2.5 注册模块
关键步骤四:在task.py的parse_model函数中注册
2.6 执行程序
在train.py中,将model的参数路径设置为yolov8_C2f_RetBlock.yaml的路径
建议大家写绝对路径,确保一定能找到
from ultralytics import YOLO
import warnings
warnings.filterwarnings('ignore')
from pathlib import Path
if __name__ == '__main__':
# 加载模型
model = YOLO("ultralytics/cfg/v8/yolov8.yaml") # 你要选择的模型yaml文件地址
# Use the model
results = model.train(data=r"你的数据集的yaml文件地址",
epochs=100, batch=16, imgsz=640, workers=4, name=Path(model.cfg).stem) # 训练模型
🚀运行程序,如果出现下面的内容则说明添加成功🚀
from n params module arguments
0 -1 1 464 ultralytics.nn.modules.conv.Conv [3, 16, 3, 2]
1 -1 1 4672 ultralytics.nn.modules.conv.Conv [16, 32, 3, 2]
2 -1 1 4960 ultralytics.nn.modules.block.C2f_RetBlock [32, 32, 1, 'chunk', 8, True]
3 -1 1 18560 ultralytics.nn.modules.conv.Conv [32, 64, 3, 2]
4 -1 2 27776 ultralytics.nn.modules.block.C2f_RetBlock [64, 64, 2, 'chunk', 8, True]
5 -1 1 73984 ultralytics.nn.modules.conv.Conv [64, 128, 3, 2]
6 -1 2 104704 ultralytics.nn.modules.block.C2f_RetBlock [128, 128, 2, 'whole', 8, True]
7 -1 1 295424 ultralytics.nn.modules.conv.Conv [128, 256, 3, 2]
8 -1 1 269056 ultralytics.nn.modules.block.C2f_RetBlock [256, 256, 1, 'whole', 8, True]
9 -1 1 164608 ultralytics.nn.modules.block.SPPF [256, 256, 5]
10 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
11 [-1, 6] 1 0 ultralytics.nn.modules.conv.Concat [1]
12 -1 1 148224 ultralytics.nn.modules.block.C2f [384, 128, 1]
13 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
14 [-1, 4] 1 0 ultralytics.nn.modules.conv.Concat [1]
15 -1 1 37248 ultralytics.nn.modules.block.C2f [192, 64, 1]
16 -1 1 36992 ultralytics.nn.modules.conv.Conv [64, 64, 3, 2]
17 [-1, 12] 1 0 ultralytics.nn.modules.conv.Concat [1]
18 -1 1 123648 ultralytics.nn.modules.block.C2f [192, 128, 1]
19 -1 1 147712 ultralytics.nn.modules.conv.Conv [128, 128, 3, 2]
20 [-1, 9] 1 0 ultralytics.nn.modules.conv.Concat [1]
21 -1 1 493056 ultralytics.nn.modules.block.C2f [384, 256, 1]
22 [15, 18, 21] 1 897664 ultralytics.nn.modules.head.Detect [80, [64, 128, 256]]
YOLOv8_C2f_RetBlock summary: 295 layers, 2848752 parameters, 2848736 gradients, 8.0 GFLOPs
3. 完整代码分享
https://pan.baidu.com/s/1Kj1LSav1G_CMXCL04BMLqQ?pwd=fd9u
提取码: fd9u
4. GFLOPs
关于GFLOPs的计算方式可以查看:百面算法工程师 | 卷积基础知识——Convolution
未改进的YOLOv8nGFLOPs
改进后的GFLOPs
5. 进阶
可以与其他的注意力机制或者损失函数等结合,进一步提升检测效果
6. 总结
RetNet的核心原理在于引入了一个基于距离的衰减机制,用以建模序列中不同元素之间的相对关系。最初应用于自然语言处理(NLP),RetNet通过时间衰减矩阵为一维文本数据提供显式的时序先验,使得模型能够根据元素间的距离逐渐减少注意力权重,从而感知全局信息。在计算机视觉任务中,这一机制被扩展到二维空间中,使用曼哈顿距离构建空间衰减矩阵。这个矩阵允许模型根据图像中token之间的空间距离分配不同的注意力权重,显式引入空间先验以加强对全局信息的感知。同时,RetNet采用了自注意力的分解形式,使其在保持空间衰减的同时,通过分解水平和垂直方向的注意力,实现了线性计算复杂度。这种设计既降低了计算成本,又提升了模型的表现,使其在图像分类、目标检测和语义分割等任务中具有优异的性能。