极简光流（optical flow） - 基于深度和相机位姿的光流

实在时间紧，还是仅上英文版的，诸位见谅。原文在我的个人主页上。

 

Simple optical flow based on depth and pose information

 

Recently, I was helping to implement a simple program that performs optical flow calculation between frames of video images with the help of known depth and pose information associated with the images.

 

Let us define the first frame as Frame 0, and the second to be Frame 1. And for my current task, the camera is under constant movement. For optical flow, it is usually represented by a color image with HSV color space. In this image, Channel H (Hue) is used to represent the flow direction of the individual pixel of the Frame 0. That is if the camera is moving, every pixel of Frame 0 seems to move with respect to Frame 1 and exhibiting a representative color regarding the flow direction. Channel V (Value or Lightness) tells the magnitude of the movement of a pixel. The larger the movement, the brighter it should be. We leave the S (Saturation) Channel to be its highest value and never touch it.

 

The thing is, once you know the camera pose and depth information of each frame, you know everything. No fancy algorithm is needed to perform optical flow calculation since you can calculate the movement of the individual pixel. The overall process looks very much like the following.