RAISR论文学习

论文题目《RAISR: Rapid and Accurate Image Super Resolution 》

作者Yaniv Romano, John Isidoro, and Peyman Milanfar, Fellow, IEEE

 

部分个人觉得关键的点：

二、全局滤波器：

训练的目的是使距离最小化： {yi} and the desired training HR images {xi}.

Y:初始化的放大图像  X:高清原图像

filter h:d*d

A:d*d*M*N的patches(d*d)

B:M*N composed of pixels from x corresponding to the center coordinates of patches

学习过程：

采样K patches or pixels来构建Ai与bi

 

问题转化为最小化：

其中 Q = AT A and V = AT b.

Q:d^2*d^2

 

避免将数组全部放入内存

 

Q，V可以通过累加求和计算

(for example sub matrices , 

此处还可以并行计算

 

的计算效率很高，因为Q是一个半正定矩阵，可使用快速共轭梯度法 

三、处理aliasing问题

所采用的简单的升采样方法:双线性插值

插值权重与位置有关；两个线性filter的卷积可以统一为一个filter

因此学习四种filter，对应四种可能的像素类型

filter就像带通滤波器，滤除高频（如aliasing）

应用过程：

注：若upscaling的参数为s（提升的倍数），则学习的滤波器个数是s^2

 

四、RAISR：基于哈希的learning与upscaling

使用高效的哈希方法：为了保持线性滤波器的低复杂度，将图像分桶

每个桶q对应滤波器hq：

A.计算hashtable(Angle, Strength, Coherence)

论文建议通过eigen analysis分析局部梯度特征,可以 产生 梯度的角度，局部梯度的strength与coherence信息

基本方法：由第k个像素周围的水平和垂直梯度构成：gx 与 gy,  表达为. 通过 eigen-decomposition，2×2 的矩阵：可以更有效的计算特征

此外，为了在每个像素中合并小的邻域梯度样本，我们采用对角加权矩阵Wk，使用可分离的归一化高斯核构造。the eigenvector φk1, corresponding to the largest eigenvalue of , can be used to derive the angle of the gradient θk, given by

注：根据对称性, a filter that corresponds to the angle θk is identical to the one corresponding to θk +180◦.

The square root of the largest eigenvalue  is analogousto the “strength” of the gradient.The square root of the smaller eigenvalue  can be considered as the “spread” of the local gradients, or rather how much they vary in direction. Both of these can be measured in units of intensity. The two eigenvalues can be combined into a unitless measure known as “coherence” . The coherence value μk ranges from 0 to 1, and formulated as

量化因素的意义：力度与偶合性低意味着 a lack of image structure, and usually corresponds to noise or compression artifacts;力度高偶合性低通常表示角落或多方向结构;偶合性高通常是 an edge, or series of stripes in the same direction, with the strength measuring the relative intensity of the stripes.

B. 根据patch对称性生成8个example

有些hash很常见: 垂直，水平结构与平面 (such as sky, and painted surfaces)

In order to make the patches that hash to uncommon hash values more effective, we leverage the patch symmetry and increase the learning power of each patch. More specifically, we can generate 8 different example patches;four 90◦ rotations, and four mirrored 90◦ rotations.

无需对实际图像进行变换: If the gradient angle dependent hash bucket boundaries are symmetric to flips in x, y and xy-swaps, the accumulation for transformed patches can be performed. in our case, having a number of angle buckets that is divisible by 4 satisfies this requirement .As such,we can accumulate the per bucket and per pixel-type matrices and  across all training samples, Then, the symmetry can be applied as one last accumulation of permuted matrices, which act as a set of symmetry augmented matrices.

C. 内置压缩伪影去除与图像锐化

D.通过blending有效地保留图像结构

the cheap interpolated image typically does an adequate job on areas of the image that contain low spatial frequencies(e.g. flat regions).On the other hand,the higher spatial frequencies that need to be reconstructed require the careful treatment of the estimated filters.