【文献阅读】ChangeNet——变化检测网络（A. Varghese等人，ECCV，2018）

一、背景

文章题目：《ChangeNet: A Deep Learning Architecture for Visual Change Detection》

这篇文章思路非常简单，觉得能中ECCV还是有点牵强啊。变化检测一般就是孪生网络+反卷积，能还原出变化的mask就行，考虑到不同尺度下的变化特征，引入多尺度特征层就可以了。即使不看这篇文章，一般人也能想到这个思路。感觉能中ECCV有点玄学，因为亮点不多。

文章下载地址：https://link.springer.com/content/pdf/10.1007%2F978-3-030-11012-3_10.pdf

文章引用格式：Ashley Varghese, Jayavardhana Gubbi, Akshaya Ramaswamy, and Balamuralidhar P. "ChangeNet: A Deep Learning Architecture for Visual Change Detection." European Conference on Computer Vision (ECCV), 2018.

项目地址：暂无

二、文章摘要

The increasing urban population in cities necessitates the need for the development of smart cities that can offer better services to its citizens. Drone technology plays a crucial role in the smart city environment and is already involved in a number of functions in smart cities such as traffic control and construction monitoring. A major challenge in fast growing cities is the encroachment of public spaces. A robotic solution using visual change detection can be used for such purposes. For the detection of encroachment, a drone can monitor outdoor urban areas over a period of time to infer the visual changes. Visual change detection is a higher level inference task that aims at accurately identifying variations between a reference image (historical) and a new test image depicting the current scenario. In case of images, the challenges are complex considering the variations caused by environmental conditions that are actually unchanged events. Human mind interprets the change by comparing the current status with historical data at intelligence level rather than using only visual information. In this paper, we present a deep architecture called ChangeNet for detecting changes between pairs of images and express the same semantically (label the change). A parallel deep convolutional neural network (CNN) architecture for localizing and identifying the changes between image pair has been proposed in this paper. The architecture is evaluated with VL-CMU-CD street view change detection, TSUNAMI and Google Street View (GSV) datasets that resemble drone captured images. The performance of the model for different lighting and seasonal conditions are experimented quantitatively and qualitatively. The result shows that ChangeNet outperforms the state of the art by achieving 98.3% pixel accuracy, 77.35% object based Intersection over Union (IoU) and 88.9% area under Receiver Operating Characteristics (RoC) curve.

首先作者提到了智慧城市，然后谈到城市的变化非常快。为了自动检测出这种变化，作者提出了ChangeNet，它是一个并行CNN结构来检测图像对之间的变化。实验基于三个数据集VL-CMU-CD，TSUNAMI，Google Street View。实验表明ChangeNet达到了98.3%的识别精度，77.35%的IoU和88.9%的RoC。

三、文章介绍

一般做目标变化检测的难点在于：光照，光强，对比度，分辨率，质量，尺度，位置等因素的微妙变化。传统方法就是利用图像分割，结合阈值，这种低级决策方法，遇到一些复杂情况就变得极其不稳定。这里作者给了一个例子：

该例子来自VL-CMU-CD数据集。实际上的变化之处仅仅为门口的垃圾，其他地方尽管稍有不同，但实际上是没有发生变化的，这种变化是一种高级信息。

因此，本文设计ChangNet，它使用ResNet来提取图像特征。然后结合不同层（尺度）的变化信息。最后使用相同的网络来检测变化的label。

1. 相关工作及动机

这里先不介绍了，因为做变化检测的其实挺多的。

2. 模型结构

作者的想法来源于孪生网络和全卷积网络FCN，网络结构如下所示：

基本上就是一个孪生网络的结构，上下两个CNN的权值共享，作者提出的这个网络和孪生网络一个最大的区别是，权重和反卷积是无关的（not tied），这可以使模型提升5%。

对于特征提取模块，作者使用的是ResNet50：

最后为了获得mask，需要进行上采样获得和输入相同的feature map，这里作者上采样使用了双线性内插来替代卷积（Upsampling is done with bilinear interpolation filter）。然后把两个并行网络的输出做连接。连接好的输出最后再连接一个softmax做分类就OK。

下面看一下作者给的一个例子吧：

四、小结

这篇文章很简单，实现起来也非常容易，自己就简单写了下核心伪代码：

# 以下代码基于tensorflow，简单写下思路，真正在写的时候，除了train，val，test，还要考虑placeholder，loss，optimizer等等，这些都要设置

### 导入相关库
from tensorflow.contrib import slim
from tensorflow.contrib.slim.nets import resnet_v2
from tensorflow.contrib.slim.python.slim.nets.resnet_utils import resnet_arg_scope

### 读取数据
before_info = ........
after_info = ........

### 用resnet提取两组数据的特征
with slim.arg_scope(resnet_arg_scope):
    net1, end_points1 = resnet_v2.resnet_v2_50(before_info)
    net2, end_points2 = resnet_v2.resnet_v2_50(after_info, reuse=True)

### 拿出不同层的特征
b1 = end_points1[.......]
b2 = end_points1[.......]

a1 = end_points2[.......]
a2 = end_points2[.......]

### 然后反卷积
w = tf.constant(1.0, shape=[.......])
b1_mask = tf.nn.conv2d_transpose(......)
b2_mask = tf.nn.conv2d_transpose(......)

a1_mask = tf.nn.conv2d_transpose(......)
a2_mask = tf.nn.conv2d_transpose(......)

### 然后连接,用tf.concat(....)

### 最后softmax就可以了