点击上方“机器学习与生成对抗网络”，关注"星标"
获取有趣、好玩的前沿干货！
这日，你伸着懒腰，打着呵欠，对着窗外，正感慨时光已逝，红了樱桃绿了芭蕉……忽然，桌面上的手机传来了一声微信的振动声，你极其不耐烦地走过去。
“老猪，我在超市看到了一个气质佳人！”
面对老铁这未见世面的无措，你弹指键飞：
“你还能见到啥佳人？再说，就你审美？？？”
“稍等！……”
“你要干嘛……”
很快，对面传来一幅图：
“你偷拍人家真的好吗。。再说脸呢？？……”
这时手机又亮起：
“我刚刚把无关的截了一下，再截个脸吧~
”
“？？？……”
“隔得有点远，可能拍的有点小，好像看不清……”
正文引言
摘自SRGAN:The highly challenging task of estimating a highresolution (HR) image from its low-resolution (LR) counterpart is referred to as super-resolution (SR).
图像超分辨率，简称超分SR，一般指放大分辨率，例如把256X256变到512X512的分辨率，这时的放大倍数scale为2。显然，这是一个无中生有、去补全像素的ill-posed问题，没有唯一解。图像超分，应用场景自然是广泛的。一般的方法是将低分辨率的图像LR作为方法的输入，进行处理得到高分辨率的HR图像。
但值得注意的是，在现实场景中，匹配成对的数据集是极其难以获取得到的。如今相当多的论文，都是自制这种LR-HR图像对去作为训练集。比如先将原图HR通过下采样得到LR，再进行LR到HR的映射学习。但真正应用到实际中，LR和HR之间的关系是不是我们自以为是的“下采样”的关系呢？这恐怕是未知、难以模拟的，人为的下采样或其他人工方法不过是一厢情愿罢。在医学图像SR上可能更需谨慎。
今天整理的是结合GAN生成对抗网络的图像超分。首先总结两篇极具代表意义的、大名鼎鼎的超分GAN即SRGAN和ESRGAN，并大概提一篇用网络去收集小分辨率的数据的论文，最后给出70多篇结合GAN做超分的论文！！！希望给有志这方面探索、了解的同学一个参考！
（70多篇论文已经下载打包好，获取方式进入公众号后台，回复【超分GAN】即可）
1. (2017-05-25) （SRGAN）Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
https://arxiv.xilesou.top/pdf/1609.04802.pdf
尽管使用更快、更深的卷积神经网络实现单图像超分辨率在准确性和速度上都有突破，仍然存在一个焦点问题在很大程度上未解决：当在较大的放大倍数上去获取超分辨率图像时，如何恢复更精细的纹理细节？以往的工作主要集中在均方差重建上，在结果评估时使用PSNR等，但通常缺乏高频细节，并且在视觉上难以令人满意。在本文提出SRGAN，第一个用于图像超分辨率（SR）的生成对抗网络（GAN），能够推断出4倍逼真的自然图像。为了实现这一目标，提出了一种感知损失函数，包括对抗损失和内容损失。使用基于感知相似性的内容损失摈弃了在像素空间进行相似性衡量。平均意见分数（MOS）表明了方法卓越的性能。
如下图所示，放大4倍的超分方法对比。第一个是双立方插值，第二个是基于均方差损失驱动的卷积神经网络，第三个是本文SRGAN，最后是参考原始图。
优化：
损失函数：
生成器损失（原文作者把整个生成器损失叫感知损失：内容损失+生成器对抗损失）：
内容损失：
生成器对抗损失：
作者做了蛮多一些消融探究的，此不述。
最后是实验结果其一。堪称大型SSIM和PSNR打脸现场。SRGAN在PSNR和SSIM上的表现不如SRResNet但在MOS、也就是人眼观察上吊打前者足矣。
2. (2018-09-17) ESRGAN Enhanced Super-Resolution Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1809.00219.pdf
SRGAN是具有开创性的工作。但细节之处仍然难令人满意，为此进一步研究了SRGAN的三个关键组成部分：网络架构，对抗损失和感知损失，并将其改善得到增强型SRGAN（ESRGAN）。特别地，引入了无BN批归一化的残差密集块Residual-in-Residual Dense Block（RRDB）作为基本的网络构建单元。而且，借用相对GAN 的思想让判别器进行预测相对真实性。最后，通过使用在激活之前的特征去进行感知损失计算，来达到在亮度一致性和纹理恢复方面提供更强的监督的目的。受益于这些改进， ESRGAN相比SRGAN，具有更好的视觉质量、更逼真的自然纹理并赢得PIRM2018-SR挑战赛的第一名。
网络结构上的改进：
由于BN在比如粗粒度任务分类等中具有积极效果，但对于类似于风格迁移这种单幅图像具有鲜明特点的任务中，不宜使用批量的统计量，否则容易弱化单图像固有的本身细节信息。于是作者尝试去掉BN，但这又容易导致网络训练的困难，于是采用Dense block这种更易提升网络性能的结构。
对抗方式的改进：
参考了相对GAN的设计思路。
对抗损失：
大致推导一下：
原始GAN：
感知Loss的改进：
使用relu激活之前的特征进行损失计算。这样的特征可以包含更丰富和细节的响应信息。
使用网络插值：
GAN过于“自由胡来”，有一些细节可能不太自然。而以往基于MSE优化的卷积网络偏向平滑模糊丢失细节。于是网络插值提出综合两者网络的方法：先训练一个常规的超分网络，在这个网络的基础上再fine-tuning得到GAN的生成器，然后把两个网络的参数加权相加：
如下图所示，通过调节α可以找到一个更偏好或平衡的的中间效果。
3. (2018-07-30) To learn image super-resolution use a GAN to learn how to do image degradation first
https://arxiv.xilesou.top/pdf/1807.11458.pdf
在前面提到，超分的训练里，通过简单的双线性下采样（少数情况下是先模糊后下采样）人工生成的低分辨率的图像，然后将它们进行超分处理。但在现实生活中，这种方法并不能产生很好的效果。
为此提出一个两阶段的过程，首先训练一个High-to-Low GAN来学习如何对高分辨率图像进行下采样，在训练过程中，只需要非配对的高分辨率和低分辨率图像。实现了这部分后，该网络的输出可以用来训练一个Low-to-High GAN来实现超分辨率重建，这次利用配对的低分辨率和高分辨率图像。我们的主要结果是，这个网络可以有效地提高真实世界低分辨率图像的质量。本文将这种方法应用于人脸超分辨率的问题，并验证其有效性，方法也可能适用于其他图像对象类别。
实验结果：
001 (2020-03-4) Turbulence Enrichment using Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/2003.01907.pdf
002 (2020-03-2) MRI Super-Resolution with GAN and 3D Multi-Level DenseNet Smaller Faster and Better
https://arxiv.xilesou.top/pdf/2003.01217.pdf
003 (2020-02-29) Joint Face Completion and Super-resolution using Multi-scale Feature Relation Learning
https://arxiv.xilesou.top/pdf/2003.00255.pdf
004 (2020-02-21) Generator From Edges Reconstruction of Facial Images
https://arxiv.xilesou.top/pdf/2002.06682.pdf
005 (2020-01-22) Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization
https://arxiv.xilesou.top/pdf/2001.08126.pdf
006 (2020-01-21) Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques
https://arxiv.xilesou.top/pdf/2001.07766.pdf
007 (2020-01-10) Segmentation and Generation of Magnetic Resonance Images by Deep Neural Networks
https://arxiv.xilesou.top/pdf/2001.05447.pdf
008 (2019-12-15) Image Processing Using Multi-Code GAN Prior
https://arxiv.xilesou.top/pdf/1912.07116.pdf
009 (2020-02-6) Quality analysis of DCGAN-generated mammography lesions
https://arxiv.xilesou.top/pdf/1911.12850.pdf
010 (2019-12-19) A deep learning framework for morphologic detail beyond the diffraction limit in infrared spectroscopic imaging
https://arxiv.xilesou.top/pdf/1911.04410.pdf
011 (2019-11-8) Joint Demosaicing and Super-Resolution (JDSR) Network Design and Perceptual Optimization
https://arxiv.xilesou.top/pdf/1911.03558.pdf
012 (2019-11-4) FCSR-GAN Joint Face Completion and Super-resolution via Multi-task Learning
https://arxiv.xilesou.top/pdf/1911.01045.pdf
013 (2019-10-9) Wavelet Domain Style Transfer for an Effective Perception-distortion Tradeoff in Single Image Super-Resolution
https://arxiv.xilesou.top/pdf/1910.04074.pdf
014 (2020-02-3) Optimal Transport CycleGAN and Penalized LS for Unsupervised Learning in Inverse Problems
https://arxiv.xilesou.top/pdf/1909.12116.pdf
015 (2019-08-26) RankSRGAN Generative Adversarial Networks with Ranker for Image Super-Resolution
https://arxiv.xilesou.top/pdf/1908.06382.pdf
016 (2019-07-24) Progressive Perception-Oriented Network for Single Image Super-Resolution
https://arxiv.xilesou.top/pdf/1907.10399.pdf
017 (2019-07-26) Boosting Resolution and Recovering Texture of micro-CT Images with Deep Learning
https://arxiv.xilesou.top/pdf/1907.07131.pdf
018 (2019-07-15) Enhanced generative adversarial network for 3D brain MRI super-resolution
https://arxiv.xilesou.top/pdf/1907.04835.pdf
019 (2019-07-5) MRI Super-Resolution with Ensemble Learning and Complementary Priors
https://arxiv.xilesou.top/pdf/1907.03063.pdf
020 (2019-11-25) Image-Adaptive GAN based Reconstruction
https://arxiv.xilesou.top/pdf/1906.05284.pdf
021 (2019-06-13) A Hybrid Approach Between Adversarial Generative Networks and Actor-Critic Policy Gradient for Low Rate High-Resolution Image Compression
https://arxiv.xilesou.top/pdf/1906.04681.pdf
022 (2019-06-4) A Multi-Pass GAN for Fluid Flow Super-Resolution
https://arxiv.xilesou.top/pdf/1906.01689.pdf
023 (2019-05-23) Generative Imaging and Image Processing via Generative Encoder
https://arxiv.xilesou.top/pdf/1905.13300.pdf
024 (2019-05-26) Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation
https://arxiv.xilesou.top/pdf/1905.10777.pdf
025 (2019-05-9) 3DFaceGAN Adversarial Nets for 3D Face Representation Generation and Translation
https://arxiv.xilesou.top/pdf/1905.00307.pdf
026 (2019-08-27) Super-Resolved Image Perceptual Quality Improvement via Multi-Feature Discriminators
https://arxiv.xilesou.top/pdf/1904.10654.pdf
027 (2019-03-28) SRDGAN learning the noise prior for Super Resolution with Dual Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1903.11821.pdf
028 (2019-03-21) Bandwidth Extension on Raw Audio via Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1903.09027.pdf
029 (2019-03-6) DepthwiseGANs Fast Training Generative Adversarial Networks for Realistic Image Synthesis
https://arxiv.xilesou.top/pdf/1903.02225.pdf
030 (2019-02-28) A Unified Neural Architecture for Instrumental Audio Tasks
https://arxiv.xilesou.top/pdf/1903.00142.pdf
031 (2019-02-28) Two-phase Hair Image Synthesis by Self-Enhancing Generative Model
https://arxiv.xilesou.top/pdf/1902.11203.pdf
032 (2019-10-23) GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems
https://arxiv.xilesou.top/pdf/1902.09698.pdf
033 (2019-02-17) Progressive Generative Adversarial Networks for Medical Image Super resolution
https://arxiv.xilesou.top/pdf/1902.02144.pdf
034 (2019-01-31) Compressing GANs using Knowledge Distillation
https://arxiv.xilesou.top/pdf/1902.00159.pdf
035 (2019-01-18) Generative Adversarial Classifier for Handwriting Characters Super-Resolution
https://arxiv.xilesou.top/pdf/1901.06199.pdf
036 (2019-01-10) How Can We Make GAN Perform Better in Single Medical Image Super-Resolution A Lesion Focused Multi-Scale Approach
https://arxiv.xilesou.top/pdf/1901.03419.pdf
037 (2019-01-9) Detecting Overfitting of Deep Generative Networks via Latent Recovery
https://arxiv.xilesou.top/pdf/1901.03396.pdf
038 (2018-12-29) Brain MRI super-resolution using 3D generative adversarial networks
https://arxiv.xilesou.top/pdf/1812.11440.pdf
039 (2019-01-13) Efficient Super Resolution For Large-Scale Images Using Attentional GAN
https://arxiv.xilesou.top/pdf/1812.04821.pdf
040 (2019-12-24) Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation
https://arxiv.xilesou.top/pdf/1811.09393.pdf
041 (2018-11-20) Adversarial Feedback Loop
https://arxiv.xilesou.top/pdf/1811.08126.pdf
042 (2018-11-1) Bi-GANs-ST for Perceptual Image Super-resolution
https://arxiv.xilesou.top/pdf/1811.00367.pdf
043 (2018-10-15) Lesion Focused Super-Resolution
https://arxiv.xilesou.top/pdf/1810.06693.pdf
044 (2018-10-15) Deep learning-based super-resolution in coherent imaging systems
https://arxiv.xilesou.top/pdf/1810.06611.pdf
045 (2018-10-10) Image Super-Resolution Using VDSR-ResNeXt and SRCGAN
https://arxiv.xilesou.top/pdf/1810.05731.pdf
046 (2019-01-28) Multi-Scale Recursive and Perception-Distortion Controllable Image Super-Resolution
https://arxiv.xilesou.top/pdf/1809.10711.pdf
047 (2018-09-2) Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1809.00437.pdf
048 (2018-09-17) ESRGAN Enhanced Super-Resolution Generative Adversarial Networks
https://arxiv.xilesou.top/pdf/1809.00219.pdf
049 (2018-09-6) CT Super-resolution GAN Constrained by the Identical Residual and Cycle Learning Ensemble(GAN-CIRCLE)
https://arxiv.xilesou.top/pdf/1808.04256.pdf
050 (2018-07-30) To learn image super-resolution use a GAN to learn how to do image degradation first
https://arxiv.xilesou.top/pdf/1807.11458.pdf
051 (2018-07-1) Performance Comparison of Convolutional AutoEncoders Generative Adversarial Networks and Super-Resolution for Image Compression
https://arxiv.xilesou.top/pdf/1807.00270.pdf
052 (2018-12-19) Generative Adversarial Networks and Perceptual Losses for Video Super-Resolution
https://arxiv.xilesou.top/pdf/1806.05764.pdf
053 (2018-08-22) cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM
https://arxiv.xilesou.top/pdf/1804.06244.pdf
054 (2018-04-10) A Fully Progressive Approach to Single-Image Super-Resolution
https://arxiv.xilesou.top/pdf/1804.02900.pdf
055 (2018-07-18) Maintaining Natural Image Statistics with the Contextual Loss
https://arxiv.xilesou.top/pdf/1803.04626.pdf
056 (2018-06-9) Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network
https://arxiv.xilesou.top/pdf/1803.01417.pdf
057 (2018-05-28) tempoGAN A Temporally Coherent Volumetric GAN for Super-resolution Fluid Flow
https://arxiv.xilesou.top/pdf/1801.09710.pdf
058 (2018-10-3) High-throughput high-resolution registration-free generated adversarial network microscopy
https://arxiv.xilesou.top/pdf/1801.07330.pdf
059 (2017-11-28) Super-Resolution for Overhead Imagery Using DenseNets and Adversarial Learning
https://arxiv.xilesou.top/pdf/1711.10312.pdf
060 (2019-10-3) The Perception-Distortion Tradeoff
https://arxiv.xilesou.top/pdf/1711.06077.pdf
061 (2017-11-7) Tensor-Generative Adversarial Network with Two-dimensional Sparse Coding Application to Real-time Indoor Localization
https://arxiv.xilesou.top/pdf/1711.02666.pdf
062 (2017-11-7) ZipNet-GAN Inferring Fine-grained Mobile Traffic Patterns via a Generative Adversarial Neural Network
https://arxiv.xilesou.top/pdf/1711.02413.pdf
063 (2017-10-19) Generative Adversarial Networks An Overview
https://arxiv.xilesou.top/pdf/1710.07035.pdf
064 (2018-05-21) Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution
https://arxiv.xilesou.top/pdf/1710.04783.pdf
065 (2018-11-28) Simultaneously Color-Depth Super-Resolution with Conditional Generative Adversarial Network
https://arxiv.xilesou.top/pdf/1708.09105.pdf
066 (2017-06-20) Perceptual Generative Adversarial Networks for Small Object Detection
https://arxiv.xilesou.top/pdf/1706.05274.pdf
067 (2017-05-7) A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA
https://arxiv.xilesou.top/pdf/1705.02583.pdf
068 (2017-05-5) Face Super-Resolution Through Wasserstein GANs
https://arxiv.xilesou.top/pdf/1705.02438.pdf
069 (2017-10-12) CVAE-GAN Fine-Grained Image Generation through Asymmetric Training
https://arxiv.xilesou.top/pdf/1703.10155.pdf
070 (2017-02-21) Amortised MAP Inference for Image Super-resolution
https://arxiv.xilesou.top/pdf/1610.04490.pdf
071 (2017-05-25) Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
https://arxiv.xilesou.top/pdf/1609.04802.pdf
公众号近期荐读：
GAN&CV交流群，无论小白还是大佬，诚挚邀您加入！
一起讨论交流！长按备注【进群】加入：
更多分享、长按关注本公众号：
2862
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
