利用帧间相关性的视频语义分割

0. Points

• CRF models on video data can fuse the predicted label information across frames
• Joint Semantic Segmentation and 3D
  Reconstruction from Monocular Video. ECCV 2014. video
• Joint optical flow and temporally con- sistent semantic segmentation. page 4. “Optical flow with semantics”, ECCV 2016
• Optical flow with semantic segmentation and localized layers. CVPR 2016
• Video propagation networks. CVPR 2017

1. Semantic Video CNNs through Representation Warping. ICCV 2017

Github | video

1.1 Results(visualization)

1.2 Flow Path

• Transform Flow
  是一个借鉴了ResNet的小型4层CNN的网络，输入pair的init image以及difference，以及计算得到的光流 。在第三层的时候，又输入了原始的init image

没有利用关键帧

2. Deep Feature Flow for Video Recognition. CVPR 2017

Github | Video

关键帧5，10对应于两个数据集
光流的估计函数是用CNN处理。

3. Low-Latency Video Semantic Segmentation. CVPR 2018

自适应选择关键帧

4. Dynamic Video Segmentation Network. CVPR 2018

之后利用decision network的score，与threshold==93%做了自适应关键帧的选择。则cityspace上达到了19fps的速度。

SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks. 2017 ICRA

关键帧 10 on dataset NYUV2

这里的CNN是选择了VGG的一个简化版，只有两层deconvolution, for tradeoff between accuracy and speed. 为了使用depth，但是实际上第四个通道使用了rgb的average intensity 然后做了一个映射

MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Object. ISMAR 2018

Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation. IROS 2018