Highlight
- First paper to use trained CNN for optical flow estimation
- Introduce novel correlation layer
- Refine network by upsampling
Model
- FlowNetSimple: concatenate two consecutive images.
- FlowNetCorr: use correlation layer
- Correlation layer
- Calculated between two feature maps
- c(x1,x2)=∑o∈[−k,k]×[−k,k]<f1(x1+o),f2(x2+o)>
- See model picture for an illustration
- Refinement
- Concatenate the upsampled flow prediction and conv feature map
Experiment
- Datasets:
- Middlebury
- KITTI
- Sintel
- Flying Chairs (proposed, auto generated)
- Loss function: endpoint error – Euclidean distance between the predicted flow vector and GT.
- Conclusion
- FlowNet performs a little worse than other OF algorithm, but obviously faster.
- Network trained on Flying Chairs (auto generated) data has good generalization ability on natural scenes.