论文阅读笔记之——《Deep Stacked Hierarchical Multi-patch Network for Image Deblurring》

论文：https://arxiv.org/pdf/1904.03468v1.pdf

本文是做非均匀deblur的。The goal of non-uniform blind image deblurring is to remove the undesired blur caused by the camera motion and the scene dynamics

开篇作者抛出来两个问题：

1) Deconvolution/upsampling operations in the coarse-to-fine scheme result in expensive runtime;（时间长）

2) Simply increasing the model depth with finer-scale levels cannot improve the quality of deblurring.（简单的增加模型的深度是不可以提升performance的）

下面这张图是实时性与performance的图。其实在low-level vision问题上，当performance提到一定程度，大多数都会选择往参数量低或者运算量低的方向走

we present a deep hierarchical （分层的） multi-patch network inspired by Spatial Pyramid Matching （空间金字塔匹配） to deal with blurry images via a fine-to-coarse hierarchical representation.

Prior to the success of deep learning, conventional deblurring methods used to employ a variety of constraints or regularizations to approximate the motion blur filters, involving an expensive non-convex nonlinear optimization. Moreover, the commonly used assumption of spatially-uniform blur kernel is overly restrictive, resulting in a poor deblurring of complex blur patterns.

the “coarse-to-fine” scheme（“粗到细”方案）a sharp image is gradually restored at different resolutions in a pyramid.

we propose a simple yet effective multi-level CNN model called Deep Multi-Patch Hierarchical Network (DMPHN) which uses multi-patch hierarchy as input.

Feature aggregation （聚合） over multiple patches has been used in image classification

Contributions：

1、We propose an end-to-end CNN hierarchical model akin （相似） to Spatial Pyramid Matching (SPM) that performs deblurring in the fine-to-coarse grids thus exploiting multi-patch localized-to-coarse operations. 

2、We identify the limitation to stacking depth of current deep deblurring models and introduce novel stacking approaches which overcome this limitation.（打破了限制网络深度的因素）

Conventional image deblurring methods fail to remove non-uniform motion blur due to the use of spatially-invariant （空间不变） deblurring kernel.

Recurrent Neural Network (RNN) is a popular tool employed in deblurring due to its advantage in sequential information processing.

整体的结构如下图2所示

we use the (1-2-4-8) model for illustration purposes. Notation (1-2-4-8) indicates the numbers of image patches from the coarsest to the finniest level

Note that the numbers of input and output patches at each level are different as the main idea of our work is to make the lower level focus on local information (finer grid) to produce residual information for the coarser gird (obtained by concatenating convolutional features).

 

论文的网络结构跟其他结构对比

尽管本文用的结构比较简单，没有了skip and recurrent connections，但是却非常有效

每一个level都有encoder和decoder，如下图所示

这篇论文的思路很不错，可惜作者没有release源码