线性模型

公式法求解线性模型

该公式出现在书中的54页, 问题如下: 已知$E(w,b)={\sum }_{i=1}^m(y_i-w{x}_i-b{)}^2$, 求$w$和$b$的最优解.
证明: 首先将$E(w,b)$分别对$w$和$b$求偏导,可得:
$$\begin{gathered} \frac{\partial E(w,b)}{\partial w}=2(w\sum _{i=1}^m{x}_i^2-\sum _{i=1}^m(y_i-b)x_i) \\ \frac{\partial E(w,b)}{\partial b}=2(mb-\sum _{i=1}^m(y_i-w{x}_i)) \end{gathered}$$
令$\frac{\partial E(w,b)}{\partial w}=\frac{\partial E(w,b)}{\partial b}=0$, 可得:
$$\begin{gathered} w\sum _{i=1}^m{x}_i^2-\sum _{i=1}^m(y_i-b)x_i=0,式一 \\ b=\frac{1}{m}\sum _{i=1}^m(y_i-w{x}_i),式二 \end{gathered}$$
将式二带入式一可得:
w ∑ i = 1 m x i 2 − ∑ i = 1 m ( y i − 1 m ∑ j = 1 m ( y j − w x j ) ) x i = 0 w ∑ i = 1 m x i 2 − ∑ i = 1 m x i y i + 1 m ∑ i = 1 m ∑ j = 1 m x i y j − 1 m ∑ i = 1 m ∑ j = 1 m w x i x j = 0 w ( ∑ i = 1 m x i 2 − 1 m ( ∑ i = 1 m x i ) 2 ) = − ∑ i = 1 m x i y i + ∑ i = 1 m y i ∑ j = 1 m 1 m x j w = ∑ i = 1 m y i ( x i − x ˉ ) ∑ i = 1 m x i 2 − 1 m ( ∑ i = 1 m x i ) 2 w\sum_{i=1}^mx_i^2-\sum_{i=1}^m(y_i-\frac{1}{m}\sum_{j=1}^m(y_j-wx_j))x_i=0\\ w\sum_{i=1}^mx_i^2-\sum_{i=1}^mx_iy_i+\frac{1}{m}\sum_{i=1}^m\sum_{j=1}^mx_iy_j-\frac{1}{m}\sum_{i=1}^m\sum_{j=1}^mwx_ix_j=0 \\ w(\sum_{i=1}^mx_i^2-\frac{1}{m}(\sum_{i=1}^mx_i)^2)=-\sum_{i=1}^mx_iy_i+\sum_{i=1}^my_i\sum_{j=1}^m\frac{1}{m}x_j \\w=\frac{\sum_{i=1}^my_i(x_i-\bar x)}{\sum_{i=1}^mx_i^2-\frac{1}{m}(\sum_{i=1}^mx_i)^2} wi=1∑m​xi2​−i=1∑m​(yi​−m1​j=1∑m​(yj​−wxj​))xi​=0wi=1∑m​xi2​−i=1∑m​xi​yi​+m1​i=1∑m​j=1∑m​xi​yj​−m1​i=1∑m​j=1∑m​wxi​xj​=0w(i=1∑m​xi2​−m1​(i=1∑m​xi​)