最小二乘法
J ( w ) = 1 2 ∑ i = 1 N ( w T x i − y i ) 2 J(w)=\Large\frac{1}{2}\normalsize\sum\limits_{i=1}^N(w^Tx_i-y_i)^2 J(w)=21i=1∑N(wTxi−yi)2,其中 w = ( w 1 , w 2 , . . . , w n , b ) T , x i = ( x i 1 , x i 2 , . . . , x i n , 1 ) T w=(w_1,w_2,...,w_n,b)^T,x_i=(x_i^1,x_i^2,...,x_i^n,1)^T w=(w1,w2,...,wn,b)T,xi=(xi1,xi2,...,xin,1)T
设:
X
=
[
x
1
1
x
1
2
.
.
.
x
1
n
x
2
1
x
2
2
.
.
.
x
2
n
.
.
.
.
.
.
.
.
.
.
.
.
x
N
1
x
N
2
.
.
.
x
N
n
]
X= \left[ \begin{matrix} x_1^1 & x_1^2 & ... &x_1^n \\ x_2^1 & x_2^2 & ... &x_2^n \\ ...& ...& ...& ...\\ x_N^1 & x_N^2 & ... &x_N^n \\ \end{matrix} \right]
X=⎣⎢⎢⎡x11x21...xN1x12x22...xN2............x1nx2n...xNn⎦⎥⎥⎤
Y = ( y 1 , y 2 , . . . , y N ) T Y=(y_1,y_2,...,y_N)^T Y=(y1,y2,...,yN)T
则
J ( w ) = 1 2 ∣ ∣ X w − Y ∣ ∣ 2 J(w)=\Large\frac{1}{2}\normalsize||Xw-Y||^2 J(w)=21∣∣Xw−Y∣∣2,其中||为第二范式
J ( w ) = 1 2 ( X w − Y ) T ( X w − Y ) J(w)=\Large\frac{1}{2}\normalsize(Xw-Y)^T(Xw-Y) J(w)=21(Xw−Y)T(Xw−Y)
= 1 2 ( w T X T − Y T ) ( X w − Y ) =\Large\frac{1}{2}\normalsize(w^TX^T-Y^T)(Xw-Y) =21(wTXT−YT)(Xw−Y)
= 1 2 ( w T X T X w + Y T Y − w T X T Y − Y T X w ) =\Large\frac{1}{2}\normalsize(w^TX^TXw+Y^TY-w^TX^TY-Y^TXw) =21(wTXTXw+YTY−wTXTY−YTXw)
则
∂ J ( w ) ∂ w = X T X w − X T Y = 0 \frac{\partial J(w)}{\partial \ w}=X^TXw-X^TY=0 ∂ w∂J(w)=XTXw−XTY=0
w = ( X T X ) − 1 X T Y w=(X^TX)^{-1}X^TY w=(XTX)−1XTY
OK
89
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
