网上看到的最小二乘法都是互相抄来抄去,一错一大片,连标点符号都错的一样,其实整个推导过程并不是很难的一件事,只需要简单的一步步按照最小化损失函数就可以得到,最终我们需要得到的线性方程为
y
^
=
β
0
+
β
1
x
^
\hat{y} = \beta_0 + \beta_1 \hat{x}
y^=β0+β1x^损失函数采用欧式距离定义,
L
=
∑
i
=
1
n
(
y
i
−
y
^
i
)
2
\mathcal{L} = \sum_{i =1}^n (y_i -\hat{y}_i)^2
L=i=1∑n(yi−y^i)2于是问题变为最优化问题
arg
min
β
0
,
β
1
L
\mathop{\arg\min}_{\beta_0,\beta_1} \quad \mathcal{L}
argminβ0,β1L
于是对
β
0
\beta_0
β0,
β
1
\beta_1
β1求导即为
∂
L
∂
β
0
=
∑
i
=
1
n
(
y
i
−
β
0
−
β
1
x
i
)
\frac{\partial \mathcal{L}}{\partial \beta_0} = \sum_{i=1}^n (y_i-\beta_0 -\beta_1x_i)
∂β0∂L=i=1∑n(yi−β0−β1xi)
对
β
1
求
导
\beta_1求导
β1求导即为
∂
L
∂
β
1
=
∑
i
=
1
n
(
y
1
−
β
0
−
β
1
x
i
)
(
−
x
i
)
\frac{\partial \mathcal{L}}{\partial \beta_1} = \sum_{i=1}^n(y_1 - \beta_0 - \beta_1 x_i)(-x_i)
∂β1∂L=i=1∑n(y1−β0−β1xi)(−xi)由
β
0
\beta_0
β0求导式子得到
β
0
=
1
n
∑
i
=
1
n
y
i
−
1
n
∑
i
=
1
n
β
1
x
i
=
y
‾
−
β
1
x
‾
\beta_0 =\frac{1}{n} \sum_{i=1}^ny_i - \frac{1}{n} \sum_{i=1}^n \beta_1 x_i =\overline{y} -\beta_1\overline{x}
β0=n1i=1∑nyi−n1i=1∑nβ1xi=y−β1x代入第二部分式子,
∑
i
=
1
n
x
i
y
i
−
β
1
∑
i
=
1
n
x
i
2
−
n
β
0
x
‾
=
0
\sum_{i=1}^n x_iy_i -\beta_1\sum_{i=1}^nx_i^2-n\beta_0\overline{x}=0
i=1∑nxiyi−β1i=1∑nxi2−nβ0x=0得到,
β
1
(
∑
i
=
1
n
x
i
2
−
n
(
x
‾
)
2
)
=
∑
i
=
1
n
x
i
y
i
−
n
x
‾
∗
y
‾
\beta_1(\sum_{i=1}^nx_i^2-n(\overline{x})^2)=\sum_{i=1}^nx_iy_i-n\overline{x}*\overline{y}
β1(i=1∑nxi2−n(x)2)=i=1∑nxiyi−nx∗y则得到
β
1
=
∑
i
=
1
n
x
i
y
i
−
n
x
‾
∗
y
‾
∑
i
=
1
n
x
i
2
−
n
(
x
‾
)
2
\beta_1 =\frac{\sum_{i=1}^nx_iy_i-n\overline{x}*\overline{y}}{\sum_{i=1}^nx_i^2-n(\overline{x})^2}
β1=∑i=1nxi2−n(x)2∑i=1nxiyi−nx∗y或者写为
β
1
=
n
∑
i
=
1
n
x
i
y
i
−
∑
i
=
1
n
x
i
∑
i
=
1
n
y
i
n
∑
i
=
1
n
x
i
2
−
(
∑
i
=
1
n
x
)
2
\beta_1 =\frac{n\sum_{i=1}^nx_iy_i-\sum_{i=1}^nx_i\sum_{i=1}^ny_i}{n\sum_{i=1}^nx_i^2-(\sum_{i=1}^n{x})^2}
β1=n∑i=1nxi2−(∑i=1nx)2n∑i=1nxiyi−∑i=1nxi∑i=1nyi代入到
β
0
\beta_0
β0则有,
β
0
=
y
‾
−
β
1
x
‾
=
∑
i
=
1
n
x
i
2
∑
i
=
1
n
y
i
−
∑
i
=
1
n
x
i
∑
i
=
1
n
x
i
y
i
n
∑
i
=
1
n
x
i
2
−
(
∑
i
=
1
n
x
i
)
2
\beta_0 =\overline{y} - \beta_1\overline{x}=\frac{\sum_{i=1}^nx_i^2\sum_{i=1}^ny_i-\sum_{i=1}^nx_i\sum_{i=1}^nx_iy_i}{n\sum_{i=1}^nx_i^2 - (\sum_{i=1}^nx_i)^2}
β0=y−β1x=n∑i=1nxi2−(∑i=1nxi)2∑i=1nxi2∑i=1nyi−∑i=1nxi∑i=1nxiyi这两个式子即为最小二乘法得到的求解结果,求得
β
0
\beta_0
β0和
β
1
\beta_1
β1之后,即可得到回归方程:
y
^
=
β
0
+
β
1
x
^
\hat{y} =\beta_0 +\beta_1\hat{x}
y^=β0+β1x^
最小二乘法的数学推导
最新推荐文章于 2024-06-01 14:04:00 发布