流程翻译于Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm
目标: u u u是图像 I I I中的一个点,我们要在图像 J J J中找到它对应的点 v v v
建立 I I I和 J J J的图像金字塔: { I L } L = 0 , … , L m and { J L } L = 0 , … , L m \left\{I^{L}\right\}_{L=0, \ldots, L_{m}} \text { and }\left\{J^{L}\right\}_{L=0, \ldots, L_{m}} {IL}L=0,…,Lm and {JL}L=0,…,Lm
初始化金字塔上的位移估计为0: g L m = [ g x L m g x L m ] T = [ 0 0 ] T \mathbf{g}^{L_{m}}=\left[g_{x}^{L_{m}} g_{x}^{L_{m}}\right]^{T}=\left[ \begin{array}{ll}{0} & {0}\end{array}\right]^{T} gLm=[gxLmgxLm]T=[00]T
使 L = L m L=L_{m} L=Lm,递减到 m = 0 m=0 m=0:
找到 u u u在图像 I L I^{L} IL中的位置: u L = [ p x p y ] T = u / 2 L \mathbf{u}^{L}=\left[p_{x} \quad p_{y}\right]^{T}=\mathbf{u} / 2^{L} uL=[pxpy]T=u/2L
计算 I L I^{L} IL对 x x x的导数: I x ( x , y ) = I L ( x + 1 , y ) − I L ( x − 1 , y ) 2 I_{x}(x, y) =\frac{I^{L}(x+1, y)-I^{L}(x-1, y)}{2} Ix(x,y)=2IL(x+1,y)−IL(x−1,y)
计算 I L I^{L} IL对 y y y的导数: I y ( x , y ) = I L ( x , y + 1 ) − I L ( x , y − 1 ) 2 I_{y}(x, y) = \frac{I^{L}(x, y+1)-I^{L}(x, y-1)}{2} Iy(x,y)=2IL(x,y+1)−IL(x,y−1)
计算空间导数矩阵: G = ∑ x = p x − ω x p x + ω x ∑ y = p y − ω y p y + ω y [ I x 2 ( x , y ) I x ( x , y ) I y ( x , y ) I x ( x , y ) I y ( x , y ) I y 2 ( x , y ) ] G=\sum_{x=p_{x}-\omega_{x}}^{p_{x}+\omega_{x}} \sum_{y=p_{y}-\omega_{y}}^{p_{y}+\omega_{y}} \left[ \begin{array}{cc}{I_{x}^{2}(x, y)} & {I_{x}(x, y) I_{y}(x, y)} \\ {I_{x}(x, y) I_{y}(x, y)} & {I_{y}^{2}(x, y)}\end{array}\right] G=∑x=px−ωxpx+ωx∑y=py−ωypy+ωy[Ix2(x,y)Ix(x,y)Iy(x,y)Ix(x,y)Iy(x,y)Iy2(x,y)]
初始化L-K迭代: ν ‾ 0 = [ 0 0 ] T \overline{\nu}^{0}=\left[ \begin{array}{ll}{0} & {0}\end{array}\right]^{T} ν0=[00]T
从 k = 1 k=1 k=1,累加到K(当 ∥ η ‾ k ∥ \left\|\overline{\eta}^{k}\right\| ∥∥ηk∥∥小于设定阈值的时候也会打断迭代)
图像相减: δ I k ( x , y ) = I L ( x , y ) − J L ( x + g x L + ν x k − 1 , y + g y L + ν y k − 1 ) \delta I_{k}(x, y)=I^{L}(x, y)-J^{L}\left(x+g_{x}^{L}+\nu_{x}^{k-1}, y+g_{y}^{L}+\nu_{y}^{k-1}\right) δIk(x,y)=IL(x,y)−JL(x+gxL+νxk−1,y+gyL+νyk−1)
计算图像误匹配向量: b ‾ k = ∑ x = p x − ω x p x + ω x ∑ y = p y − ω y p y + ω y [ δ I k ( x , y ) I x ( x , y ) δ I k ( x , y ) I y ( x , y ) ] \overline{b}_{k}=\sum_{x=p_{x}-\omega_{x}}^{p_{x}+\omega_{x}} \sum_{y=p_{y}-\omega_{y}}^{p_{y}+\omega_{y}} \left[ \begin{array}{c}{\delta I_{k}(x, y) I_{x}(x, y)} \\ {\delta I_{k}(x, y) I_{y}(x, y)}\end{array}\right] bk=∑x=px−ωxpx+ωx∑y=py−ωypy+ωy[δIk(x,y)Ix(x,y)δIk(x,y)Iy(x,y)]
计算光流(Lucas-Kanade): η ‾ k = G − 1 b ‾ k \overline{\eta}^{k}=G^{-1} \overline{b}_{k} ηk=G−1bk
累加迭代量: ν ‾ k = ν ‾ k − 1 + η ‾ k \overline{\nu}^{k}=\overline{\nu}^{k-1}+\overline{\eta}^{k} νk=νk−1+ηk
结束 k k k的循环
得到第 L L L层的光流估计: d L = ν ‾ K \mathbf{d}^{L}=\overline{\nu}^{K} dL=νK
计算出 L − 1 L-1 L−1层的初步估计: g L − 1 = [ g x L − 1 g y L − 1 ] T = 2 ( g L + d L ) \mathbf{g}^{L-1}=\left[g_{x}^{L-1} g_{y}^{L-1}\right]^{T}=2\left(\mathbf{g}^{\mathbf{L}}+\mathbf{d}^{L}\right) gL−1=[gxL−1gyL−1]T=2(gL+dL)
结束 L L L循环
得到最终的光流估计: d = g 0 + d 0 \mathbf{d}=\mathbf{g}^{0}+\mathbf{d}^{0} d=g0+d0
定位图像 J J J中的位置: v = u + d \mathbf{v}=\mathbf{u}+\mathbf{d} v=u+d
结果:图像 J J J中与 u u u相对应的点就是 v v v