递推最小二乘法RLS公式详细推导

递推最小二乘法RLS公式详细推导

整理递推最小二乘法推导过程自我整理。
递推最小二乘估计（RLS）作为一种估计方式是在最小二乘法（LS）的基础上发展来的。
最小二乘法可以解决的问题是不需要知道先验的概率知识以及观察值的自相关和观察值与估计值之间的互相关等先验条件。
如果各次观察值反映了待估参数的线性组合，即表示成:
$y=\left[\begin{array}{lll}{x}_1& \dots & x_k\end{array}\right]\left[\begin{array}{l}{\theta }_1\\ ⋮\\ {\theta }_k\end{array}\right]$
已知k组观测值y和k组系数x矩阵的分布，来求$\theta$的估计。用矩阵的形式来表达为：
$\left[\begin{array}{c}{y}_1\\ ⋮\\ y_k\end{array}\right]=\left[\begin{array}{c}{\varphi }_1^T\\ ⋮\\ {\varphi }_k^T\end{array}\right]$
${\Phi }_k=\left[\begin{array}{c}{\varphi }_1^T\\ ⋮\\ {\varphi }_k^T\end{array}\right]\in {\mathbb{R}}^{k_{Nn}}$
$\Theta =\left[\begin{array}{c}{\theta }_1\\ ⋮\\ {\theta }_n\end{array}\right]\in {\mathbb{R}}^{n\times 1}$
令${\Phi }_k=\left[\begin{array}{c}{\varphi }_1^T\\ ⋮\\ {\varphi }_k^T\end{array}\right]\in {\mathbb{R}}^{k\times n}$
则最小二乘法观测值$\Theta$的解为
${\hat{\Theta }}_k={\left({\Phi }_k^T{\Phi }_k\right)}^{-1}{\Phi }_k^T{Y}_k$
下面推导递推的形式：
令$P_k^{-1}={\Phi }_k^T{\Phi }_k$
现在讨论$\Theta$的解的项中的因子
则$P_k^{-1}={\Phi }_k^T{\Phi }_k=\left[\begin{array}{llll}{\varphi }_1& {\varphi }_2& \cdots & {\varphi }_k\end{array}\right]\left[\begin{array}{c}{\varphi }_1^T\\ {\varphi }_2^I\\ ⋮\\ {\varphi }_k^T\end{array}\right]={\sum }_{i=1}^n{\varphi }_i{\varphi }_i^T={\sum }_{i=1}^{n-1}{\varphi }_i{\varphi }_i^T+{\varphi }_k{\varphi }_k^T=P_{k-1}^{-1}+{\varphi }_k{\varphi }_k^T$
${\Phi }_k^T{\mathbf{y}}_k=\left[\begin{array}{llll}{\varphi }_1& {\varphi }_2& \cdots & {\varphi }_k\end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_k\end{array}\right]={\sum }_{i=1}^n{\varphi }_i{y}_i={\sum }_{i=1}^{n-1}{\varphi }_i{y}_i+{\varphi }_k{y}_k={\Phi }_{k-1}^T{\mathbf{y}}_{k-1}+{\varphi }_k{y}_k$
由以上两式得到：
$P_k^{-1}=P_{k-1}^{-1}+{\varphi }_k{\varphi }_k^T$ (1)
${\Phi }_k^r{\mathbf{y}}_k={\Phi }_{k-1}^T{\mathbf{y}}_{k-1}+{\varphi }_k{y}_k$ (2)
${\hat{\theta }}_k={\left({\Phi }^T{\Phi }_k\right)}^{-1}{\Phi }_k^T{y}_k=P_k{\Phi }_k^T{y}_k=P_k\left({\Phi }_{k-1}^T{y}_{k-1}+{\varphi }_k{y}_k\right)$ (3)
因为${\hat{\theta }}_{k-1}=P_{k-1}\left({\Phi }_{k-1}^T{y}_{k-1}\right)$
所以有$P_k^{-1}{\hat{\theta }}_k={\Phi }_k^{\mathrm{T}}{\mathbf{y}}_k$(4)
${\hat{\mathbf{\theta }}}_k\stackrel{3}{⟶}{P}_k\left({\Phi }_{k-1}^T{y}_{k-1}+{\varphi }_k{y}_k\right)\stackrel{4}{⟶}{P}_k\left(P_{k-1}^{-1}{\hat{\theta }}_{k-1}+{\varphi }_k{y}_k\right)$$\stackrel{1}{⟶}{P}_k\left(P_k^{-1}{\hat{\theta }}_{k-1}-{\varphi }_k{\varphi }_k^T{\hat{\theta }}_{k-1}+{\varphi }_k{y}_k\right)$$={\hat{\theta }}_{k-1}+P_k{\varphi }_k\left(y_k-{\varphi }_k^T{\hat{\theta }}_{k-1}\right)={\hat{\theta }}_{k-1}+K_k\epsilon$
至此可以得到关于估计量$\Theta$的递推公式：
${\epsilon }_k=y_k-{\varphi }_k^T{\hat{\theta }}_{k-1}$
${\hat{\theta }}_k={\hat{\theta }}_{k-1}+K_k{\epsilon }_k$
$K_k=P_k{\varphi }_k$
$P_k={\left(P_{k-1}^{-1}+{\varphi }_k{\varphi }_k^T\right)}^{-1}$
此时已经基本得到递推公式了，但最后一个矩阵还需要计算逆矩阵，计算量仍然很大，现在继续做简化
根据化简公式：
$[A+BCD{]}^{-1}=A^{-1}-A^{-1}B{\left[C^{-1}+D{A}^{-1}R\right]}^{-1}D{A}^{-1}$
对于$P_k={\left(P_{k-1}^{-1}+{\varphi }_k{\varphi }_k^T\right)}^{-1}$
$A=P_{k-1}^{-1}$、$B={\varphi }_k$、$C=1$、$D={\varphi }_k^T$代换
得到$P_k={\left(P_{k-1}^{-1}+{\varphi }_k{\varphi }_k^T\right)}^{-1}=A^{-1}-A^{-1}B{\left[C^{-1}+D{A}^{-1}B\right]}^{-1}D{A}^{-1}$$=P_{k-1}-P_{k-1}{\varphi }_k{\left[1+{\varphi }_k^T{P}_{k-1}{\varphi }_k\right]}^{-1}{\varphi }_k^T{P}_{k-1}$$=P_{k-1}-\frac{P_{k-1}{\varphi }_k{\varphi }_k^T{P}_{k-1}}{1+{\varphi }_k^T{P}_{k-1}{\varphi }_k}$

参考：
知乎-递推最小二乘推导(RLS)

递推最小二乘法RLS：推导