1. 第三讲(上)_基于优化的IMU与视觉信息融合(上)
第三讲(上):
(1)最小二乘问题的求解推导
(2)其中有LM算法的相关推导以及鲁棒核函数的推导
1.1. 最小二乘与非线性优化
1.1.1. 最小二乘
1.1.2. 非线性最小二乘
1.1.2.1. LM算法增量 Δ X \Delta X ΔX表达式推导
求解如下:
(
J
T
J
+
μ
I
)
Δ
X
=
−
J
T
f
⟹
(
V
Λ
V
T
+
μ
I
)
Δ
X
=
−
J
T
f
⟹
(
V
(
Λ
+
μ
I
)
V
T
)
Δ
X
=
−
J
T
f
⟹
(
Λ
+
μ
I
)
V
T
Δ
X
=
−
V
T
J
T
f
⟹
(
Λ
+
μ
I
)
V
T
Δ
X
=
−
V
T
F
′
(
x
)
T
⟹
V
T
Δ
X
=
−
(
Λ
+
μ
I
)
−
1
V
T
F
′
(
x
)
T
\begin{aligned} &(\boldsymbol{J}^T\boldsymbol{J}+\mu\boldsymbol{I})\boldsymbol{\Delta X}=-\boldsymbol{J}^T\boldsymbol{f} \\ &\Longrightarrow (\boldsymbol{V \Lambda V^T}+\mu \boldsymbol{I})\boldsymbol{\Delta X}=-\boldsymbol{J}^T\boldsymbol{f} \\ &\Longrightarrow(\boldsymbol{V} (\boldsymbol{\Lambda}+\mu \boldsymbol{I})\boldsymbol{V}^T) \boldsymbol{\Delta X}=-\boldsymbol{J}^T\boldsymbol{f} \\ &\Longrightarrow(\boldsymbol{\Lambda}+\mu \boldsymbol{I})\boldsymbol{V}^T\boldsymbol{\Delta X}=-\boldsymbol{V}^T\boldsymbol{J}^T\boldsymbol{f} \\ &\Longrightarrow(\boldsymbol{\Lambda}+\mu \boldsymbol{I})\boldsymbol{V}^T\boldsymbol{\Delta X}=-\boldsymbol{V}^T F^{'}(x)^T \\ &\Longrightarrow\boldsymbol{V}^T\boldsymbol{\Delta X}=-(\boldsymbol{\Lambda}+\mu \boldsymbol{I})^{-1}\boldsymbol{V}^T F^{'}(x)^T \\ \end{aligned}
(JTJ+μI)ΔX=−JTf⟹(VΛVT+μI)ΔX=−JTf⟹(V(Λ+μI)VT)ΔX=−JTf⟹(Λ+μI)VTΔX=−VTJTf⟹(Λ+μI)VTΔX=−VTF′(x)T⟹VTΔX=−(Λ+μI)−1VTF′(x)T
V T Δ X = − [ 1 λ 1 + μ 0 0 ⋯ 0 1 λ 2 + μ 0 ⋯ 0 ⋯ 1 λ j + μ ] V T F ′ ( x ) T \begin{aligned} \boldsymbol{V}^T\boldsymbol{\Delta X}=- \begin{bmatrix} \frac{1}{\lambda_1+\mu}&0&0& \cdots \\ 0 & \frac{1}{\lambda_2+\mu} & 0 &\cdots \\ 0 & \cdots & & \frac{1}{\lambda_j+\mu} \end{bmatrix} \boldsymbol{V}^T F^{'}(x)^T \end{aligned} VTΔX=−⎣⎢⎡λ1+μ1000λ2+μ1⋯00⋯⋯λj+μ1⎦⎥⎤VTF′(x)T
Δ X = − [ v 1 ⋯ v j ] [ 1 λ 1 + μ 0 0 ⋯ 0 1 λ 2 + μ 0 ⋯ 0 ⋯ 1 λ j + μ ] [ v 1 ⋮ v j ] F ′ ( x ) T \begin{aligned} \boldsymbol{\Delta X}=- \begin{bmatrix} \boldsymbol{v_1} & \cdots &\boldsymbol{v_j} \end{bmatrix} \begin{bmatrix} \frac{1}{\lambda_1+\mu}&0&0& \cdots \\ 0 & \frac{1}{\lambda_2+\mu} & 0 &\cdots \\ 0 & \cdots & & \frac{1}{\lambda_j+\mu} \end{bmatrix} \begin{bmatrix} \boldsymbol{v_1} \\ \vdots \\ \boldsymbol{v_j} \end{bmatrix} F^{'}(x)^T \end{aligned} ΔX=−[v1⋯vj]⎣⎢⎡λ1+μ1000λ2+μ1⋯00⋯⋯λj+μ1⎦⎥⎤⎣⎢⎡v1⋮vj⎦⎥⎤F′(x)T
∴ Δ X = − ∑ j = 1 n v j T F ′ T λ j + μ v j \begin{aligned} \therefore \boldsymbol{\Delta X}= -\sum_{j=1}^{n}\frac{\boldsymbol{v}_j^T {\boldsymbol{F'}}^T}{\lambda_j+\mu} \boldsymbol{v}_j \end{aligned} ∴ΔX=−j=1∑nλj+μvjTF′Tvj
从上图可以看到,由于 μ \mu μ的值在抖动,使得 Δ X \Delta X ΔX的步长在来回震荡变化,使得loss值也随着抖动,有一部分的计算是多余的.