场景分割的双注意网络
IEEE Conference on Computer Vision and Pattern Recognition 2019
Scene Segmentation With Dual Relation-Aware Attention Network
IEEE Transactions on Neural Networks and Learning Systems 2020
Cross-Level Gating Decoder
高低特征交叉融合门控(赋权值)解码器模块。数据流如下,H和L分别代表高低图像特征,这里的关键,不是简单像U-Net一样将高低特征在通道层进行堆叠,而是选择将高级图像特征作为门控对低级特征进行赋权值,进而有效选取有用的低级特征与高级特征进行堆叠。
许多研究表明,低级视觉信息提高了对边界和细节的准确预测,但其局部感受野也可能导致对其他区域的错误分类。与直接融合低层和高层特征的论文不同,我们采用跨层门控机制来提供指导,选择性地增强空间细节并抑制其他区域,从而获得更好的交叉特征融合。
因此,提出了一种简单而有效的带有跨级门的解码器结构,如图所示。融合模块的输出执行2倍的双线性上采样以生成高级特征H。同时,在ResNet block1的输出上使用32kernels(Conv+BN+ReLU)的3×3卷积层来获得低级特征L。为了有效地融合低级特征L和特征H,我们将它们输入用于细化低级特征的跨级门。具体操作:将这两个特征连接起来,并将它们输入一个1 × 1的卷积层,其中一个内核后跟一个sigmoid层,以获得空间门控图;然后,我们根据门控图对低级特征L重新加权,两个特征之间的边界或细节不同,可以得到空间门控图来强调边界和细节的区域;接着,根据门控图对低级特征L进行重新加权,使边界区域和细节区域更具响应性 在获得细化的低级特征M后,我们将其和H连接在一起,然后是两个3×3卷积层,具有256 kernels (卷积+BN+ReLU);最后,采用卷积层进行像素级分类,并对高分辨率输出进行4倍的双线性上采样。
class CLGD(Module):
"""
Cross-level Gating Decoder
"""
def __init__(self, in_channels, out_channels, norm_layer):
super(CLGD, self).__init__()
inter_channels= 32
self.conv_low = nn.Sequential(nn.Conv2d(in_channels, inter_channels, 3, padding=1, bias=False),
norm_layer(inter_channels),
nn.ReLU()) #skipconv
self.conv_cat = nn.Sequential(nn.Conv2d(in_channels+inter_channels, in_channels, 3, padding=1, bias=False),
norm_layer(in_channels),
nn.ReLU()) # fusion1
self.conv_att = nn.Sequential(nn.Conv2d(in_channels+inter_channels, 1, 1),
nn.Sigmoid()) # att
self.conv_out = nn.Sequential(nn.Conv2d(in_channels, out_channels, 3, padding=1, bias=False),
norm_layer(out_channels),
nn.ReLU()) # fusion2
self._up_kwargs = up_kwargs
self.gamma = nn.Parameter(torch.ones(1))
def forward(self, x,y):
"""
inputs :
x : low level feature(N,C,H,W) y:high level feature(N,C,H,W)
returns :
out : cross-level gating decoder feature
"""
low_lvl_feat = self.conv_low(x)
high_lvl_feat = upsample(y, low_lvl_feat.size()[2:], **self._up_kwargs)
feat_cat = torch.cat([low_lvl_feat,high_lvl_feat],1)
low_lvl_feat_refine = self.gamma*self.conv_att(feat_cat)*low_lvl_feat
low_high_feat = torch.cat([low_lvl_feat_refine,high_lvl_feat],1)
low_high_feat = self.conv_cat(low_high_feat)
low_high_feat = self.conv_out(low_high_feat)
return low_high_feat