ECANet 通道注意力模块
首先来看其模型结构:
代码实现:
import torch
from torch import nn
class eca_layer(nn.Module):
def __init__(self, channel, k_size=3):
super(eca_layer, self).__init__()
self.avg_pool = nn.AdaptiveAvgPool2d(1)
self.conv = nn.Conv1d(1, 1, kernel_size=k_size, padding=(k_size - 1) // 2, bias=False)
self.sigmoid = nn.Sigmoid()
def forward(self, x):
y = self.avg_pool(x)
y = self.conv(y.squeeze(-1).transpose(-1, -2)).transpose(-1, -2).unsqueeze(-1)
y = self.sigmoid(y)
return x * y.expand_as(x)
model = eca_layer(256)
data = torch.randn((2, 3, 224, 224))
feature = model(data)
print(model)
首先传入数据:torch.Size([2, 3, 224, 224])
随后通过平均池化y = self.avg_pool(x)
变为:torch.Size([2, 3, 1, 1])
这里的维度信息我们将其拆开查看:
squeeze方法缩减维度,unsqueeze方法增加维度,transpose方法维度转换
torch.Size([2, 1, 3])的维度编号从前往后为0,1,2,从后往前为-1,-2,-3
y = self.avg_pool(x)#torch.Size([2, 3, 1, 1])
y=y.squeeze(-1) #torch.Size([2, 3, 1])
y=y.transpose(-1, -2)#torch.Size([2, 1, 3])
y = self.conv(y)#torch.Size([2, 1, 3])
y=y.transpose(-1, -2)#torch.Size([2, 3, 1])
y=y.unsqueeze(-1)#torch.Size([2, 3, 1, 1])
y = self.sigmoid(y)#torch.Size([2, 3, 1, 1])
return x * y.expand_as(x)#torch.Size([2, 3, 224, 224])
最终通过卷积来获取的维度信息:torch.Size([2, 3, 1, 1])
此时数据为:
tensor([[[[-0.0002]],
[[ 0.0001]],
[[ 0.0010]]],
[[[ 0.0009]],
[[ 0.0028]],
[[-0.0019]]]])
随后进行sigmoid进行归一化处理:
tensor([[[[0.5000]],
[[0.5000]],
[[0.5002]]],
[[[0.5002]],
[[0.5007]],
[[0.4995]]]])
通过y.expand_as(x)
进行计算恢复到原来大小:torch.Size([2, 3, 224, 224])
channel在这里并没有用到。