论文:https://arxiv.org/pdf/2002.11616.pdf
代码:https://github.com/Mukosame/Zooming-Slow-Mo-CVPR-2020
任务简述:
时空超分辨率:将低分辨率低帧率的视频重建为高分辨率高帧率(慢动作slow motion?)视频。(一个联合帧插值和视频超分的任务。)
创新点:
- We propose a one-stage space-time super-resolution network that can address temporal interpolation and spatial SR simultaneously in a unified framework. 提出了一个一阶段的时空超分网络。在统一的框架里同时解决帧插值和视频超分,比起把两个任务的方法单纯串联起来做在性能,计算资源等方面都要好。
- We propose a frame feature temporal interpolation network leveraging local temporal contexts based on deformable sampling for intermediate LR frames. We devise a novel deformable ConvLSTM to explicitly enhance temporal alignment capacity and exploit global temporal contexts for handling large motions in videos. 针对中间LR帧,提出了一种基于可变形采样的利用局部时序上下文信息的帧特征时序插值网络。设计了一种新的可变形卷积加ConvLSTM模型来显式地增强时序对齐能力,并利用全局时序上下文信息来处理视频中的大运动。
- Our onestage method achieves state-of-the-art STVSR performance on both Vid4 and Vimeo . It is 3 times faster than the two-stage network: DAIN + EDVR while having a nearly 4xreduction in model size. 相比直接串联现存最好的帧插值方法和视频超分方法组合成的二阶段方法,该方法结果好,速度快。
方法
输入: