TinyCD
题目:TINYCD: A (Not So) Deep Learning Model For Change Detection
论文地址:https://arxiv.org/abs/2207.13159https://arxiv.org/abs/2207.13159
源码:https://github.com/AndreaCodegoni/Tiny_model_4_CDhttps://github.com/AndreaCodegoni/Tiny_model_4_CD
摘要
翻译:
在本文中,我们提出了一个轻量级且有效的变化检测模型,称为TinyCD。由于工业需求,该模型被设计为比当前最先进的变更检测模型更快、更小。尽管我们的模型比所比较的变化检测模型小13到140倍,并且暴露了至少四分之一的计算复杂性,但我们的模型在LEVIR-CD数据集的F1分数和IoU上至少比目前最先进的模型高出1%,在WHU-CD数据集上超过8%。为了达到这些结果,TinyCD使用了Siamese U-Net架构,以全局时间和局部空间的方式开发低级特征。此外,该方法采用了一种新的策略,即在时空域中混合特征来合并从Siamese主干得到的嵌入,并与MLP块相结合,形成了一种新的空间语义注意机制——混合和注意掩码块(MAMB)。
创新点
- 轻量有效
- 主要利用图像的浅层特征做比较
- MAMB混合和注意掩码模块
模型图如下所示:
MAMB模块图如下所示:
模块及源码
模型采用encoder和decoder结构。主要有以下模块
- Backbone(efficientnet_b4)
- Mixing block
- PixelwiseLinear
- MixingMaskAttentionBlock
- UpMask
模块源码如下(基于PyTorch),只有上述模块代码,详情见GitHub源码。
from typing import List, Optional
from torch import Tensor, reshape, stack
from torch.nn import (Conv2d, InstanceNorm2d, Module, PReLU, Sequential, Upsample)
class PixelwiseLinear(Module):
def __init__(
self,
fin: List[int],
fout: List[int],
last_activation: Module = None,
) -> None:
assert len(fout) == len(fin)
super().__init__()
n = len(fin)
self._linears = Sequential(
*[
Sequential(
Conv2d(fin[i], fout[i], kernel_size=1, bias=True),
PReLU()
if i < n - 1 or last_activation is None
else last_activation,
)
for i in range(n)
]
)
def forward(self, x: Tensor) -> Tensor:
# Processing the tensor:
return self._linears(x)
class MixingBlock(Module):
def __init__(
self,
ch_in: int,
ch_out: int,
):
super().__init__()
self._convmix = Sequential(
Conv2d(ch_in, ch_out, 3, groups=ch_out, padding=1),
PReLU(),
InstanceNorm2d(ch_out),
)
def forward(self, x: Tensor, y: Tensor) -> Tensor:
# Packing the tensors and interleaving the channels:
mixed = stack((x, y), dim=2)
mixed = reshape(mixed, (x.shape[0], -1, x.shape[2], x.shape[3]))
# Mixing:
return self._convmix(mixed)
class MixingMaskAttentionBlock(Module):
"""use the grouped convolution to make a sort of attention"""
def __init__(
self,
ch_in: int,
ch_out: int,
fin: List[int],
fout: List[int],
generate_masked: bool = False,
):
super().__init__()
self._mixing = MixingBlock(ch_in, ch_out)
self._linear = PixelwiseLinear(fin, fout)
self._final_normalization = InstanceNorm2d(ch_out) if generate_masked else None
self._mixing_out = MixingBlock(ch_in, ch_out) if generate_masked else None
def forward(self, x: Tensor, y: Tensor) -> Tensor:
z_mix = self._mixing(x, y)
z = self._linear(z_mix)
z_mix_out = 0 if self._mixing_out is None else self._mixing_out(x, y)
return (
z
if self._final_normalization is None
else self._final_normalization(z_mix_out * z)
)
class UpMask(Module):
def __init__(
self,
scale_factor: float,
nin: int,
nout: int,
):
super().__init__()
self._upsample = Upsample(
scale_factor=scale_factor, mode="bilinear", align_corners=True
)
self._convolution = Sequential(
Conv2d(nin, nin, 3, 1, groups=nin, padding=1),
PReLU(),
InstanceNorm2d(nin),
Conv2d(nin, nout, kernel_size=1, stride=1),
PReLU(),
InstanceNorm2d(nout),
)
def forward(self, x: Tensor, y: Optional[Tensor] = None) -> Tensor:
x = self._upsample(x)
if y is not None:
x = x * y
return self._convolution(x)
实验结果
在LEVIR-CD和WHU-CD数据集上进行了实验,并跑出了最优结果。它的优点就在于很少参数的模型跑出了很好的结果!
总结
该文章用简单的模型完成了变化检测任务,可以说是一个非常优秀的卷积孪生网络,轻量化模型在未来也将会是一个趋势!
本文只是简单的介绍了一下TinyCD模型,更多细节见原文。