最小二乘法-线性拟合

1、概述

最小二乘法是曲线拟合的常用方法，使用该方法对匹配函数的选取非常重要。

2、理论

• 假设拟合的多项式为:

$$y=a_0+a_1\ast x+a_2\ast x^2+...+a_m\ast x^m$$
其中，m代表多项式的阶数。

• 离散点与该多项式的平方和$F(a_0,a_1,,a_m)$为，其中n代表采样点数：
  $$F(a_0,a_1,,a_m)=\sum _{i=0}^n[y_i-(a_0+a_1\ast x_i+a_2\ast x_i^2+...+a_m\ast x_i^m){]}^2$$
• 最小二乘法的思想是求平方和函数$F(a_0,a_1,,a_m)$的最小值，而对于二次方程求最小值的问题，常见的思路就是对方程求导，倒数为零的点，及为方程的极值点。
• 下面对$F(a_0,a_1,,a_m)$分别求$a_i$偏导数，得：
  $$\begin{gathered} -2\sum _{i=0}^n[y_i-(a_0+a_1\ast x_i+a_2\ast x_i^2+...+a_m\ast x_i^m)]=0 \\ -2\sum _{i=0}^n[y_i-(a_0+a_1\ast x_i+a_2\ast x_i^2+...+a_m\ast x_i^m)]x_i=0 \\ -2\sum _{i=0}^n[y_i-(a_0+a_1\ast x_i+a_2\ast x_i^2+...+a_m\ast x_i^m)]x_i^2=0 \\ \dots \\ -2\sum _{i=0}^n[y_i-(a_0+a_1\ast x_i+a_2\ast x_i^2+...+a_m\ast x_i^m)]x_i^m=0 \end{gathered}$$
• 整理得
  $$\begin{gathered} a_0\sum _{i=0}^n+a_1\sum _{i=0}^n{x}_i+a_2\sum _{i=0}^n{x}_i^2+...+a_m\sum _{i=0}^n{x}_i^m=\sum _{i=0}^n{y}_i \\ {a}_0\sum _{i=0}^n{x}_i+a_1\sum _{i=0}^n{x}_i^2+a_2\sum _{i=0}^n{x}_i^3+...+a_m\sum _{i=0}^n{x}_i^{m+1}=\sum _{i=0}^n{y}_i{x}_i \\ \dots \\ {a}_0\sum _{i=0}^n{x}_i^m+a_1\sum _{i=0}^n{x}_i^{m+1}+a_2\sum _{i=0}^n{x}_i^{m+2}+...+a_m\sum _{i=0}^n{x}_i^{2m}=\sum _{i=0}^n{y}_i{x}_i^m \end{gathered}$$
• 使用矩阵形式表示为：
  $$\begin{gathered} \left[\begin{array}{ccccc}{\sum }_{i=0}^n& {\sum }_{i=0}^n{x}_i& {\sum }_{i=0}^n{x}_i^2& \dots & {\sum }_{i=0}^n{x}_i^m\\ {\sum }_{i=0}^n{x}_i& {\sum }_{i=0}^n{x}_i^2& {\sum }_{i=0}^n{x}_i^3& \dots & {\sum }_{i=0}^n{x}_i^{m+1}\\ {\sum }_{i=0}^n{x}_i^2& {\sum }_{i=0}^n{x}_i^3& {\sum }_{i=0}^n{x}_i^4& \dots & {\sum }_{i=0}^n{x}_i^{m+2}\\ \dots & \dots & \dots & \dots & \dots \\ {\sum }_{i=0}^n{x}_i^m& {\sum }_{i=0}^n{x}_i^{m+1}& {\sum }_{i=0}^n{x}_i^{m+2}& \dots & {\sum }_{i=0}^n{x}_i^{2m}\end{array}\right]\left[\begin{array}{c}{a}_0\\ a_1\\ a_2\\ \dots \\ a_m\end{array}\right]= \\ \left[\begin{array}{c}{\sum }_{i=0}^n{y}_i\\ {\sum }_{i=0}^n{y}_i{x}_i\\ {\sum }_{i=0}^n{y}_i{x}_i^2\\ \dots \\ {\sum }_{i=0}^n{y}_i{x}_i^m\end{array}\right] \end{gathered}$$
• 下面就是求解一次线性方程，常用的方法是使用高斯消元法，也可以使用求矩阵的秩或者求解逆矩阵等方式求解。

3、高斯消元法求解线性方程

3.1、原则

• 两个方程互换解不变
• 一个方程乘以非零K,解不变
• 一个方程乘以非零K,加上另一个方程解不变

3.2、一阶线性函数

假设多项式得阶数m为2，则上述矩阵方程为：
$$\begin{gathered} \left[\begin{array}{ccc}{\sum }_{i=0}^n& {\sum }_{i=0}^n{x}_i& \\ {\sum }_{i=0}^n{x}_i& {\sum }_{i=0}^n{x}_i^2& \end{array}\right]\left[\begin{array}{c}{a}_0\\ a_1\end{array}\right]= \\ \left[\begin{array}{c}{\sum }_{i=0}^n{y}_i\\ {\sum }_{i=0}^n{y}_i{x}_i\end{array}\right] \end{gathered}$$
写成行列式的形式为：
$$\left|\begin{array}{ccc}{\sum }_{i=0}^n& {\sum }_{i=0}^n{x}_i& {\sum }_{i=0}^n{y}_i\\ {\sum }_{i=0}^n{x}_i& {\sum }_{i=0}^n{x}_i^2& {\sum }_{i=0}^n{y}_i{x}_i\end{array}\right|$$
第一行乘以系数${\sum }_{i=0}^n$，第二行乘以系数${\sum }_{i=0}^n{x}_i$

$$\left|\begin{array}{ccc}1& \frac{{\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n}& \frac{{\sum }_{i=0}^n{y}_i}{{\sum }_{i=0}^n}\\ 1& \frac{{\sum }_{i=0}^n{x}_i^2}{{\sum }_{i=0}^n{x}_i}& \frac{{\sum }_{i=0}^n{y}_i{x}_i}{{\sum }_{i=0}^n{x}_i}\end{array}\right|$$
第二行减去第一行得
$$\left|\begin{array}{ccc}1& \frac{{\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n}& \frac{{\sum }_{i=0}^n{y}_i}{{\sum }_{i=0}^n}\\ 0& \frac{{\sum }_{i=0}^n{x}_i^2}{{\sum }_{i=0}^n{x}_i}-\frac{{\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n}& \frac{{\sum }_{i=0}^n{y}_i{x}_i}{{\sum }_{i=0}^n{x}_i}-\frac{{\sum }_{i=0}^n{y}_i}{{\sum }_{i=0}^n}\end{array}\right|$$
通过上式可得$a_1$为
$$a_1=\frac{\frac{{\sum }_{i=0}^n{y}_i{x}_i}{{\sum }_{i=0}^n{x}_i}-\frac{{\sum }_{i=0}^n{y}_i}{{\sum }_{i=0}^n}}{\frac{{\sum }_{i=0}^n{x}_i^2}{{\sum }_{i=0}^n{x}_i}-\frac{{\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n}}=\frac{{\sum }_{i=0}^n{y}_i{x}_i\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{y}_i\ast {\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i}$$
同理将$a_1$回代入得：
$$\begin{gathered} a_0=\frac{{\sum }_{i=0}^n{y}_i}{{\sum }_{i=0}^n}-\frac{({\sum }_{i=0}^n{y}_i{x}_i\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{y}_i\ast {\sum }_{i=0}^n{x}_i)\ast {\sum }_{i=0}^n{x}_i}{({\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i)\ast {\sum }_{i=0}^n} \\ =\frac{({\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i)\ast {\sum }_{i=0}^n{y}_i-({\sum }_{i=0}^n{y}_i{x}_i\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{y}_i\ast {\sum }_{i=0}^n{x}_i)\ast {\sum }_{i=0}^n{x}_i}{({\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i)\ast {\sum }_{i=0}^n} \\ =\frac{{\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n{\sum }_{i=0}^n{y}_i-{\sum }_{i=0}^n{y}_i{x}_i\ast {\sum }_{i=0}^n{\sum }_{i=0}^n{x}_i}{({\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i)\ast {\sum }_{i=0}^n} \\ =\frac{{\sum }_{i=0}^n{x}_i^2{\sum }_{i=0}^n{y}_i-{\sum }_{i=0}^n{y}_i{x}_i{\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i} \end{gathered}$$
故最终可得
$$a_0=\frac{{\sum }_{i=0}^n{x}_i^2{\sum }_{i=0}^n{y}_i-{\sum }_{i=0}^n{y}_i{x}_i{\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i}$$
$$a_1=\frac{{\sum }_{i=0}^n{y}_i{x}_i\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{y}_i\ast {\sum }_{i=0}^n{x}_i}{{\sum }_{i=0}^n{x}_i^2\ast {\sum }_{i=0}^n-{\sum }_{i=0}^n{x}_i\ast {\sum }_{i=0}^n{x}_i}$$