1、概述
最小二乘法是曲线拟合的常用方法,使用该方法对匹配函数的选取非常重要。
2、理论
- 假设拟合的多项式为:
y
=
a
0
+
a
1
∗
x
+
a
2
∗
x
2
+
.
.
.
+
a
m
∗
x
m
y = a_0 + a_1*x +a_2*x^2 + ...+a_m*x^m
y=a0+a1∗x+a2∗x2+...+am∗xm
其中,m代表多项式的阶数。
- 离散点与该多项式的平方和
F
(
a
0
,
a
1
,
,
a
m
)
F(a_0,a_1,,a_m)
F(a0,a1,,am)为,其中n代表采样点数:
F ( a 0 , a 1 , , a m ) = ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] 2 F(a_0,a_1,,a_m) = \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)]^2 F(a0,a1,,am)=i=0∑n[yi−(a0+a1∗xi+a2∗xi2+...+am∗xim)]2 - 最小二乘法的思想是求平方和函数 F ( a 0 , a 1 , , a m ) F(a_0,a_1,,a_m) F(a0,a1,,am)的最小值,而对于二次方程求最小值的问题,常见的思路就是对方程求导,倒数为零的点,及为方程的极值点。
- 下面对
F
(
a
0
,
a
1
,
,
a
m
)
F(a_0,a_1,,a_m)
F(a0,a1,,am)分别求
a
i
a_i
ai偏导数,得:
− 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] = 0 − 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] x i = 0 − 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] x i 2 = 0 … − 2 ∑ i = 0 n [ y i − ( a 0 + a 1 ∗ x i + a 2 ∗ x i 2 + . . . + a m ∗ x i m ) ] x i m = 0 -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] = 0 \\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i= 0\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^2= 0\\ \ldots\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^m= 0 −2i=0∑n[yi−(a0+a1∗xi+a2∗xi2+...+am∗xim)]=0−2i=0∑n[yi−(a0+a1∗xi+a2∗xi2+...+am∗xim)]xi=0−2i=0∑n[yi−(a0+a1∗xi+a2∗xi2+...+am∗xim)]xi2=0…−2i=0∑n[yi−(a0+a1∗xi+a2∗xi2+...+am∗xim)]xim=0 - 整理得
a 0 ∑ i = 0 n + a 1 ∑ i = 0 n x i + a 2 ∑ i = 0 n x i 2 + . . . + a m ∑ i = 0 n x i m = ∑ i = 0 n y i a 0 ∑ i = 0 n x i + a 1 ∑ i = 0 n x i 2 + a 2 ∑ i = 0 n x i 3 + . . . + a m ∑ i = 0 n x i m + 1 = ∑ i = 0 n y i x i … a 0 ∑ i = 0 n x i m + a 1 ∑ i = 0 n x i m + 1 + a 2 ∑ i = 0 n x i m + 2 + . . . + a m ∑ i = 0 n x i 2 m = ∑ i = 0 n y i x i m a_0\sum_{i=0}^{n}+a_1\sum_{i=0}^{n} x_i+ a_2\sum_{i=0}^{n} x_i^2+...+a_m\sum_{i=0}^{n}x_i^m = \sum_{i=0}^{n}y_i\\ a_0\sum_{i=0}^{n} x_i+a_1\sum_{i=0}^{n} x_i^2+ a_2\sum_{i=0}^{n} x_i^3+...+a_m\sum_{i=0}^{n}x_i^{m+1} = \sum_{i=0}^{n}y_ix_i\\ \ldots\\ a_0\sum_{i=0}^{n} x_i^m+a_1\sum_{i=0}^{n} x_i^{m+1}+ a_2\sum_{i=0}^{n} x_i^{m+2}+...+a_m\sum_{i=0}^{n}x_i^{2m} = \sum_{i=0}^{n}y_ix_i^m a0i=0∑n+a1i=0∑nxi+a2i=0∑nxi2+...+ami=0∑nxim=i=0∑nyia0i=0∑nxi+a1i=0∑nxi2+a2i=0∑nxi3+...+ami=0∑nxim+1=i=0∑nyixi…a0i=0∑nxim+a1i=0∑nxim+1+a2i=0∑nxim+2+...+ami=0∑nxi2m=i=0∑nyixim - 使用矩阵形式表示为:
[ ∑ i = 0 n ∑ i = 0 n x i ∑ i = 0 n x i 2 … ∑ i = 0 n x i m ∑ i = 0 n x i ∑ i = 0 n x i 2 ∑ i = 0 n x i 3 … ∑ i = 0 n x i m + 1 ∑ i = 0 n x i 2 ∑ i = 0 n x i 3 ∑ i = 0 n x i 4 … ∑ i = 0 n x i m + 2 … … … … … ∑ i = 0 n x i m ∑ i = 0 n x i m + 1 ∑ i = 0 n x i m + 2 … ∑ i = 0 n x i 2 m ] [ a 0 a 1 a 2 … a m ] = [ ∑ i = 0 n y i ∑ i = 0 n y i x i ∑ i = 0 n y i x i 2 … ∑ i = 0 n y i x i m ] \left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\ldots&\sum_{i=0}^{n}x_i^m\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\ldots&\sum_{i=0}^{n}x_i^{m+1}\\ \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\sum_{i=0}^{n}x_i^4&\ldots&\sum_{i=0}^{n}x_i^{m+2}\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ \sum_{i=0}^{n}x_i^{m}&\sum_{i=0}^{n}x_i^{m+1}&\sum_{i=0}^{n}x_i^{m+2}&\ldots&\sum_{i=0}^{n}x_i^{2m} \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1\\ a_2\\ \ldots\\ a_m \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i\\ \sum_{i=0}^{n}y_ix_i^2\\ \ldots\\ \sum_{i=0}^{n}y_ix_i^m\\ \end{matrix} \right] ⎣⎢⎢⎢⎢⎡∑i=0n∑i=0nxi∑i=0nxi2…∑i=0nxim∑i=0nxi∑i=0nxi2∑i=0nxi3…∑i=0nxim+1∑i=0nxi2∑i=0nxi3∑i=0nxi4…∑i=0nxim+2……………∑i=0nxim∑i=0nxim+1∑i=0nxim+2…∑i=0nxi2m⎦⎥⎥⎥⎥⎤⎣⎢⎢⎢⎢⎡a0a1a2…am⎦⎥⎥⎥⎥⎤=⎣⎢⎢⎢⎢⎡∑i=0nyi∑i=0nyixi∑i=0nyixi2…∑i=0nyixim⎦⎥⎥⎥⎥⎤ - 下面就是求解一次线性方程,常用的方法是使用高斯消元法,也可以使用求矩阵的秩或者求解逆矩阵等方式求解。
3、高斯消元法求解线性方程
3.1、原则
- 两个方程互换解不变
- 一个方程乘以非零K,解不变
- 一个方程乘以非零K,加上另一个方程解不变
3.2、一阶线性函数
假设多项式得阶数m为2,则上述矩阵方程为:
[
∑
i
=
0
n
∑
i
=
0
n
x
i
∑
i
=
0
n
x
i
∑
i
=
0
n
x
i
2
]
[
a
0
a
1
]
=
[
∑
i
=
0
n
y
i
∑
i
=
0
n
y
i
x
i
]
\left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2& \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1 \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i \end{matrix} \right]
[∑i=0n∑i=0nxi∑i=0nxi∑i=0nxi2][a0a1]=[∑i=0nyi∑i=0nyixi]
写成行列式的形式为:
∣
∑
i
=
0
n
∑
i
=
0
n
x
i
∑
i
=
0
n
y
i
∑
i
=
0
n
x
i
∑
i
=
0
n
x
i
2
∑
i
=
0
n
y
i
x
i
∣
\begin{vmatrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}x_i & \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}y_ix_i\\ \end{vmatrix}
∣∣∣∣∑i=0n∑i=0nxi∑i=0nxi∑i=0nxi2∑i=0nyi∑i=0nyixi∣∣∣∣
第一行乘以系数
∑
i
=
0
n
\sum_{i=0}^{n}
∑i=0n,第二行乘以系数
∑
i
=
0
n
x
i
\sum_{i=0}^{n}x_i
∑i=0nxi
∣
1
∑
i
=
0
n
x
i
∑
i
=
0
n
∑
i
=
0
n
y
i
∑
i
=
0
n
1
∑
i
=
0
n
x
i
2
∑
i
=
0
n
x
i
∑
i
=
0
n
y
i
x
i
∑
i
=
0
n
x
i
∣
\begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 1 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} \end{vmatrix}
∣∣∣∣∣∣11∑i=0n∑i=0nxi∑i=0nxi∑i=0nxi2∑i=0n∑i=0nyi∑i=0nxi∑i=0nyixi∣∣∣∣∣∣
第二行减去第一行得
∣
1
∑
i
=
0
n
x
i
∑
i
=
0
n
∑
i
=
0
n
y
i
∑
i
=
0
n
0
∑
i
=
0
n
x
i
2
∑
i
=
0
n
x
i
−
∑
i
=
0
n
x
i
∑
i
=
0
n
∑
i
=
0
n
y
i
x
i
∑
i
=
0
n
x
i
−
∑
i
=
0
n
y
i
∑
i
=
0
n
∣
\begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 0 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} \end{vmatrix}
∣∣∣∣∣∣10∑i=0n∑i=0nxi∑i=0nxi∑i=0nxi2−∑i=0n∑i=0nxi∑i=0n∑i=0nyi∑i=0nxi∑i=0nyixi−∑i=0n∑i=0nyi∣∣∣∣∣∣
通过上式可得
a
1
a_1
a1为
a
1
=
∑
i
=
0
n
y
i
x
i
∑
i
=
0
n
x
i
−
∑
i
=
0
n
y
i
∑
i
=
0
n
∑
i
=
0
n
x
i
2
∑
i
=
0
n
x
i
−
∑
i
=
0
n
x
i
∑
i
=
0
n
=
∑
i
=
0
n
y
i
x
i
∗
∑
i
=
0
n
−
∑
i
=
0
n
y
i
∗
∑
i
=
0
n
x
i
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
a_1 = \frac{\frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}}{\frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}}} =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}
a1=∑i=0nxi∑i=0nxi2−∑i=0n∑i=0nxi∑i=0nxi∑i=0nyixi−∑i=0n∑i=0nyi=∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi∑i=0nyixi∗∑i=0n−∑i=0nyi∗∑i=0nxi
同理将
a
1
a_1
a1回代入得:
a
0
=
∑
i
=
0
n
y
i
∑
i
=
0
n
−
(
∑
i
=
0
n
y
i
x
i
∗
∑
i
=
0
n
−
∑
i
=
0
n
y
i
∗
∑
i
=
0
n
x
i
)
∗
∑
i
=
0
n
x
i
(
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
)
∗
∑
i
=
0
n
=
(
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
)
∗
∑
i
=
0
n
y
i
−
(
∑
i
=
0
n
y
i
x
i
∗
∑
i
=
0
n
−
∑
i
=
0
n
y
i
∗
∑
i
=
0
n
x
i
)
∗
∑
i
=
0
n
x
i
(
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
)
∗
∑
i
=
0
n
=
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
∑
i
=
0
n
y
i
−
∑
i
=
0
n
y
i
x
i
∗
∑
i
=
0
n
∑
i
=
0
n
x
i
(
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
)
∗
∑
i
=
0
n
=
∑
i
=
0
n
x
i
2
∑
i
=
0
n
y
i
−
∑
i
=
0
n
y
i
x
i
∑
i
=
0
n
x
i
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
a_0 = \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} - \frac{(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}y_i -(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n}\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}
a0=∑i=0n∑i=0nyi−(∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi)∗∑i=0n(∑i=0nyixi∗∑i=0n−∑i=0nyi∗∑i=0nxi)∗∑i=0nxi=(∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi)∗∑i=0n(∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi)∗∑i=0nyi−(∑i=0nyixi∗∑i=0n−∑i=0nyi∗∑i=0nxi)∗∑i=0nxi=(∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi)∗∑i=0n∑i=0nxi2∗∑i=0n∑i=0nyi−∑i=0nyixi∗∑i=0n∑i=0nxi=∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi∑i=0nxi2∑i=0nyi−∑i=0nyixi∑i=0nxi
故最终可得
a
0
=
∑
i
=
0
n
x
i
2
∑
i
=
0
n
y
i
−
∑
i
=
0
n
y
i
x
i
∑
i
=
0
n
x
i
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
a_0 =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}
a0=∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi∑i=0nxi2∑i=0nyi−∑i=0nyixi∑i=0nxi
a
1
=
∑
i
=
0
n
y
i
x
i
∗
∑
i
=
0
n
−
∑
i
=
0
n
y
i
∗
∑
i
=
0
n
x
i
∑
i
=
0
n
x
i
2
∗
∑
i
=
0
n
−
∑
i
=
0
n
x
i
∗
∑
i
=
0
n
x
i
a_1 =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}
a1=∑i=0nxi2∗∑i=0n−∑i=0nxi∗∑i=0nxi∑i=0nyixi∗∑i=0n−∑i=0nyi∗∑i=0nxi