人工智能教程 - 数学基础课程1.3 - 高等代数(一)-13-16 最小二乘

最小二乘

y = C+ Dt

最佳总误差最小(optimal = overall error as small as I can)

make it

$\{\begin{array}{c}C+D=1\\ C+2D=2\\ C+3D=2\end{array}$

联立无解，但可以有最优解(they don’t have a solution.but they 've got a best solution)

Ax = b

$\left[\begin{array}{cc}1& 1\\ 1& 2\\ 1& 3\end{array}\right]\left[\begin{array}{c}C\\ D\end{array}\right]=\left[\begin{array}{c}1\\ 2\\ 3\end{array}\right]$

Minimize: $||Ax-b|{|}^2=||e|{|}^2$

$=e_1^2+e_2^2+e_3^2$

$=(C+D-1{)}^2+(C+2D-2{)}^2+e_3^2$

实际上做的是线性回归分析
(It’s going to do regression here,linear regression)

最小二乘(Least squares)应用最为广泛，但是有点太容易受到离群量(outliners)的影响

(because it’s a little overcomplensates for outliners)

Find $\hat{x}=\left[\begin{array}{c}\hat{C}\\ \hat{D}\end{array}\right],P$

“^” 为最优的估计(they 're the estimated the best line, not the perfect line)

$\{\begin{array}{c}{A}^T.A.\hat{x}=A^T.b\\ P=A\hat{x}\end{array}$

$\left[\begin{array}{ccc}1& 1& 1\\ 1& 2& 3\end{array}\right]\left[\begin{array}{ccc}1& 1& |1\\ 1& 2& |2\\ 1& 3& |2\end{array}\right]=\left[\begin{array}{ccc}3& 6& |5\\ 6& 14& |11\end{array}\right]$

正规方程组(normal equation): $\{\begin{array}{c}3C+6D=5\\ 6C+14D=11\end{array}$

$\{\begin{array}{c}C=\frac{2}{3}\\ D=\frac{1}{2}\end{array}$

$\therefore$ Best line: $y=\frac{2}{3}+\frac{1}{2}t$

$\{\begin{array}{c}{e}_1=-1/6\\ e_2=+2/6\\ e_3=-1/6\end{array}$

b = P+ e

$\left[\begin{array}{c}1\\ 2\\ 2\end{array}\right]=\left[\begin{array}{c}7/6\\ 5/3\\ 13/6\end{array}\right]+\left[\begin{array}{c}-1/6\\ 2/6\\ -1/6\end{array}\right]$

最小二乘：最典型的应用就是拟合最优直线
(the special but most important example of fitting by straight line)