前提条件
- 涉及到高等数学及线性代数相关知识
- 最小二乘说明：
min
φ
∑
i
=
1
n
δ
i
2
=
∑
i
=
1
n
(
φ
(
x
i
)
−
y
i
)
2
\min _{\varphi} \sum_{i=1}^{n} \delta_{i}^{2}=\sum_{i=1}^{n}\left(\varphi\left(x_{i}\right)-y_{i}\right)^{2}
φmini=1∑nδi2=i=1∑n(φ(xi)−yi)2
即函数模型纵坐标与散点值纵坐标的差值平方之和最小时，认为拟合度最好。 - 假设多项式模型为： y = a 0 + a 1 x + … + a k x k y=a_{0}+a_{1} x+\ldots+a_{k} x^{k} y=a0+a1x+…+akxk
理论推导
- 根据最小二乘式子及模型假设，可以得到： R 2 ≡ ∑ i = 1 n [ y i − ( a 0 + a 1 x i + … + a k x i k ) ] 2 R^{2} \equiv \sum_{i=1}^{n}\left[y_{i}-\left(a_{0}+a_{1} x_{i}+\ldots+a_{k} x_{i}^{k}\right)\right]^{2} R2≡i=1∑n[yi−(a0+a1xi+…+akxik)]2
- 要想使结果最小，则参数 a 0 , a 1 , … , a k a_{0},a_{1},\ldots,a_{k} a0,a1,…,ak应满足各项偏导数 ∂ R 2 ∂ a k = 0 \frac{\partial R^2}{\partial a_{k}} = 0 ∂ak∂R2=0，因此得到n个方程： − 2 ∑ i = 1 n [ y − ( a 0 + a 1 x + … + a k x k ) ] = 0 -2 \sum_{i=1}^{n}\left[y-\left(a_{0}+a_{1} x+\ldots+a_{k} x^{k}\right)\right] =0 −2i=1∑n[y−(a0+a1x+…+akxk)]=0 − 2 ∑ i = 1 n [ y − ( a 0 + a 1 x + … + a k x k ) ] x = 0 -2 \sum_{i=1}^{n}\left[y-\left(a_{0}+a_{1} x+\ldots+a_{k} x^{k}\right)\right] x=0 −2i=1∑n[y−(a0+a1x+…+akxk)]x=0 … \ldots … − 2 ∑ i = 1 n [ y − ( a 0 + a 1 x + … + a k x k ) ] x k = 0 -2 \sum_{i=1}^{n}\left[y-\left(a_{0}+a_{1} x+\ldots+a_{k} x^{k}\right)\right] x^{k}=0 −2i=1∑n[y−(a0+a1x+…+akxk)]xk=0
- 将上述方程化简整理可得以下矩阵方程： [ n ∑ i = 1 n x i ⋯ ∑ i = 1 n x i k ∑ i = 1 n x i ∑ i = 1 n x i 2 ⋯ ∑ i = 1 n x i k + 1 ⋮ ⋮ ⋱ ⋮ ∑ i = 1 n x i k ∑ i = 1 n x i k + 1 ⋯ ∑ i = 1 n x i 2 k ] [ a 0 a 1 ⋮ a k ] = [ ∑ i = 1 n y i ∑ i = 1 n x i y i ⋮ ∑ i = 1 n x i k y i ] \left[\begin{array}{cccc} n & \sum_{i=1}^{n} x_{i} & \cdots & \sum_{i=1}^{n} x_{i}^{k} \\ \sum_{i=1}^{n} x_{i} & \sum_{i=1}^{n} x_{i}^{2} & \cdots & \sum_{i=1}^{n} x_{i}^{k+1} \\ \vdots & \vdots & \ddots & \vdots \\ \sum_{i=1}^{n} x_{i}^{k} & \sum_{i=1}^{n} x_{i}^{k+1} & \cdots & \sum_{i=1}^{n} x_{i}^{2 k} \end{array}\right]\left[\begin{array}{c} a_{0} \\ a_{1} \\ \vdots \\ a_{k} \end{array}\right]=\left[\begin{array}{c} \sum_{i=1}^{n} y_{i} \\ \sum_{i=1}^{n} x_{i} y_{i} \\ \vdots \\ \sum_{i=1}^{n} x_{i}^{k} y_{i} \end{array}\right] ⎣⎢⎢⎢⎡n∑i=1nxi⋮∑i=1nxik∑i=1nxi∑i=1nxi2⋮∑i=1nxik+1⋯⋯⋱⋯∑i=1nxik∑i=1nxik+1⋮∑i=1nxi2k⎦⎥⎥⎥⎤⎣⎢⎢⎢⎡a0a1⋮ak⎦⎥⎥⎥⎤=⎣⎢⎢⎢⎡∑i=1nyi∑i=1nxiyi⋮∑i=1nxikyi⎦⎥⎥⎥⎤
- 那么这个方程该如何求解呢？先来看一下范德蒙矩阵，假设矩阵X为范德蒙矩阵： X = [ 1 1 ⋯ 1 x 1 x 2 ⋯ x n ⋮ ⋮ ⋱ ⋮ x 1 k x 2 k ⋯ x n k ] X = \left[\begin{array}{cccc} 1 & 1 & \cdots & 1 \\ x_{1} & x_{2} & \cdots & x_{n} \\ \vdots & \vdots & \ddots & \vdots \\ x_{1}^{k} & x_{2}^{k} & \cdots & x_{n}^{k} \end{array}\right] X=⎣⎢⎢⎢⎡1x1⋮x1k1x2⋮x2k⋯⋯⋱⋯1xn⋮xnk⎦⎥⎥⎥⎤如果让矩阵 X X X与 X T X^T XT相乘就会发现： X X T = [ 1 1 ⋯ 1 x 1 x 2 ⋯ x n ⋮ ⋮ ⋱ ⋮ x 1 k x 2 k ⋯ x n k ] [ 1 x 1 ⋯ x 1 k 1 x 2 ⋯ x 2 k ⋮ ⋮ ⋱ ⋮ 1 x n ⋯ x n k ] = [ n ∑ i = 1 n x i ⋯ ∑ i = 1 n x i k ∑ i = 1 n x i ∑ i = 1 n x i 2 ⋯ ∑ i = 1 n x i k + 1 ⋮ ⋮ ⋱ ⋮ ∑ i = 1 n x i k ∑ i = 1 n x i k + 1 ⋯ ∑ i = 1 n x i 2 k ] XX^T = \left[\begin{array}{cccc}1 & 1 & \cdots & 1 \\ x_{1} & x_{2} & \cdots & x_{n} \\ \vdots & \vdots & \ddots & \vdots \\ x_{1}^{k} & x_{2}^{k} & \cdots & x_{n}^{k}\end{array}\right]\left[\begin{array}{cccc}1 & x_{1} & \cdots & x_{1}^{k} \\ 1 & x_{2} & \cdots & x_{2}^{k} \\ \vdots & \vdots & \ddots & \vdots \\ 1 & x_{n} & \cdots & x_{n}^{k}\end{array}\right] = \left[\begin{array}{cccc} n & \sum_{i=1}^{n} x_{i} & \cdots & \sum_{i=1}^{n} x_{i}^{k} \\ \sum_{i=1}^{n} x_{i} & \sum_{i=1}^{n} x_{i}^{2} & \cdots & \sum_{i=1}^{n} x_{i}^{k+1} \\ \vdots & \vdots & \ddots & \vdots \\ \sum_{i=1}^{n} x_{i}^{k} & \sum_{i=1}^{n} x_{i}^{k+1} & \cdots & \sum_{i=1}^{n} x_{i}^{2 k} \end{array}\right] XXT=⎣⎢⎢⎢⎡1x1⋮x1k1x2⋮x2k⋯⋯⋱⋯1xn⋮xnk⎦⎥⎥⎥⎤⎣⎢⎢⎢⎡11⋮1x1x2⋮xn⋯⋯⋱⋯x1kx2k⋮xnk⎦⎥⎥⎥⎤=⎣⎢⎢⎢⎡n∑i=1nxi⋮∑i=1nxik∑i=1nxi∑i=1nxi2⋮∑i=1nxik+1⋯⋯⋱⋯∑i=1nxik∑i=1nxik+1⋮∑i=1nxi2k⎦⎥⎥⎥⎤同时又有： [ 1 1 ⋯ 1 x 1 x 2 ⋯ x n ⋮ ⋮ ⋱ ⋮ x 1 k x 2 k ⋯ x n k ] [ y 1 y 2 ⋮ y n ] = [ ∑ i = 1 n y i ∑ i = 1 n x i y i ⋮ ∑ i = 1 n x i k y i ] \left[\begin{array}{cccc} 1 & 1 & \cdots & 1 \\ x_{1} & x_{2} & \cdots & x_{n} \\ \vdots & \vdots & \ddots & \vdots \\ x_{1}^{k} & x_{2}^{k} & \cdots & x_{n}^{k} \end{array}\right]\left[\begin{array}{c} y_{1} \\ y_{2} \\ \vdots \\ y_{n} \end{array}\right] = \left[\begin{array}{c}\sum_{i=1}^{n} y_{i} \\ \sum_{i=1}^{n} x_{i} y_{i} \\ \vdots \\ \sum_{i=1}^{n} x_{i}^{k} y_{i}\end{array}\right] ⎣⎢⎢⎢⎡1x1⋮x1k1x2⋮x2k⋯⋯⋱⋯1xn⋮xnk⎦⎥⎥⎥⎤⎣⎢⎢⎢⎡y1y2⋮yn⎦⎥⎥⎥⎤=⎣⎢⎢⎢⎡∑i=1nyi∑i=1nxiyi⋮∑i=1nxikyi⎦⎥⎥⎥⎤
- 由以上可知： X X T A = X Y XX^TA = XY XXTA=XY，因此推得 A = ( X X T ) − 1 X Y A = (XX^T)^{-1}XY A=(XXT)−1XY
381
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
