每日Attention学习9——Efficient Channel Attention

模块出处

[CVPR 20] [link] [code] ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

---

模块名称

Efficient Channel Attention (ECA)

---

模块作用

通道注意力

---

模块结构

---

模块代码

import torch
import torch.nn as nn
import torch.nn.functional as F

class ECA(nn.Module):
    """Constructs a ECA module.

    Args:
        channel: Number of channels of the input feature map
        k_size: Adaptive selection of kernel size
    """
    def __init__(self, channel, k_size=3):
        super(ECA, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.conv = nn.Conv1d(1, 1, kernel_size=k_size, padding=(k_size - 1) // 2, bias=False) 
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        # feature descriptor on the global spatial information
        y = self.avg_pool(x)
        # Two different branches of ECA module
        y = self.conv(y.squeeze(-1).transpose(-1, -2)).transpose(-1, -2).unsqueeze(-1)
        # Multi-scale information fusion
        y = self.sigmoid(y)
        return x * y.expand_as(x)


if __name__ == '__main__':
    x = torch.randn([3, 256, 40, 40])
    rca = ECA(256)
    out = rca(x)
    print(out.shape)  # 3, 256, 40, 40

---

原文表述

通过剖析 SENet 中的通道注意模块，我们通过经验证明避免降维对于学习通道注意很重要，适当的跨通道交互可以保持性能，同时显着降低模型复杂性。因此，我们提出了一种无需降维的局部跨通道交互策略，可以通过 1D 卷积有效实现。此外，我们开发了一种自适应选择 1D 卷积核大小的方法，确定局部跨通道交互的覆盖范围。