一元线性回归函数:
y
=
ω
x
+
b
y = \omega x+b
y=ωx+b
假设有n组[x1,x2,x3…xn],[y1,y2,y3…yn]数据,它们之间的关系为
f
(
x
)
=
ω
x
+
b
f(x) =ωx+b
f(x)=ωx+b 线性回归的目的之让
f
(
x
)
f(x)
f(x) 与
y
y
y 之间的差值最小当
ω
和
b
\omega和b
ω和b 取何值的时候二者之间的差值最小.设
E
(
ω
,
b
)
Ε_{(\omega,b)}
E(ω,b) 为
f
(
x
)
f(x)
f(x) 与
y
y
y 的差异值
E
(
ω
,
b
)
Ε(\omega,b)
E(ω,b)为损失函数:
E
(
ω
,
b
)
=
∑
i
=
1
n
(
f
(
x
)
−
y
i
)
2
=
∑
i
=
1
n
(
ω
x
i
+
b
−
y
i
)
2
Ε_{(\omega,b)} = \sum_{i=1}^n(f(x)-y_{i})^2 = \sum_{i=1}^n(\omega x_{i}+b-y_{i})^2
E(ω,b)=i=1∑n(f(x)−yi)2=i=1∑n(ωxi+b−yi)2
最小二乘法来确定
ω
和
b
ω和b
ω和b 的值因为
E
(
ω
,
b
)
Ε_{(\omega,b)}
E(ω,b) 为凸函数求其偏导:
∂
E
(
ω
,
b
)
∂
b
=
∑
i
=
1
n
(
ω
x
i
+
b
−
y
i
)
2
∂
b
=
2
∑
i
=
1
n
(
ω
x
i
+
b
−
y
i
)
\frac{∂Ε_{(\omega,b)}}{∂b} = \frac{\sum_{i=1}^n(\omega x_{i}+b-y_{i})^2}{∂b} = 2\sum_{i=1}^n(\omega x_{i}+b-y_{i})
∂b∂E(ω,b)=∂b∑i=1n(ωxi+b−yi)2=2i=1∑n(ωxi+b−yi)
=
2
[
∑
i
=
1
n
ω
x
i
+
∑
i
=
1
n
b
+
−
∑
i
=
1
n
y
i
]
=2[\sum_{i=1}^n\omega x_{i} + \sum_{i=1}^nb +-\sum_{i=1}^ny_{i}]
=2[i=1∑nωxi+i=1∑nb+−i=1∑nyi]
=
2
[
ω
∑
i
=
1
n
x
i
+
n
b
−
∑
i
=
1
n
y
i
]
=2[\omega\sum_{i=1}^nx_i+nb-\sum_{i=1}^ny_i]
=2[ωi=1∑nxi+nb−i=1∑nyi]令
x
ˉ
=
∑
i
=
1
n
x
i
n
,
y
ˉ
=
∑
i
=
1
n
y
i
n
\bar{x}=\frac{\sum_{i=1}^nx_i}{n},\bar{y}=\frac{\sum_{i=1}^ny_i}{n}
xˉ=n∑i=1nxi,yˉ=n∑i=1nyi
∴
∂
E
(
ω
,
b
)
∂
b
=
2
[
ω
n
x
ˉ
+
n
b
−
n
y
ˉ
]
∴\frac{∂Ε_{(\omega,b)}}{∂b} = 2[\omega n \bar{x}+nb-n\bar{y}]
∴∂b∂E(ω,b)=2[ωnxˉ+nb−nyˉ]
=
2
n
[
ω
x
ˉ
+
b
−
y
ˉ
]
=2n[\omega \bar{x}+b-\bar{y}]
=2n[ωxˉ+b−yˉ]
令
b
b
b的导数为0
2
n
[
ω
x
ˉ
+
b
−
y
ˉ
]
=
0
2n[\omega \bar{x}+b-\bar{y}] = 0
2n[ωxˉ+b−yˉ]=0
∵
n
>
=
1
∵n>=1
∵n>=1
∴
ω
x
ˉ
+
b
−
y
ˉ
=
0
∴\omega \bar{x} + b - \bar{y} = 0
∴ωxˉ+b−yˉ=0
∴
b
=
y
ˉ
−
ω
x
ˉ
∴b = \bar{y} - \omega \bar{x}
∴b=yˉ−ωxˉ对
ω
\omega
ω求导
∂
E
(
ω
,
b
)
∂
ω
=
∑
i
=
1
n
(
ω
x
i
+
b
−
y
i
)
2
∂
ω
=
2
∑
i
=
1
n
(
ω
x
i
+
b
−
y
i
)
x
i
\frac{∂Ε_{(\omega,b)}}{∂ \omega } = \frac{\sum_{i=1}^n(\omega x_{i}+b-y_{i})^2}{∂ \omega } = 2\sum_{i=1}^n(\omega x_{i}+b-y_{i})x_i
∂ω∂E(ω,b)=∂ω∑i=1n(ωxi+b−yi)2=2i=1∑n(ωxi+b−yi)xi
=
2
[
∑
i
=
1
n
ω
x
i
2
+
∑
i
=
1
n
b
x
i
−
∑
i
=
1
n
y
i
x
i
]
=2[\sum_{i=1}^n\omega x_i^2 + \sum_{i=1}^nb x_i - \sum_{i=1}^n y_ix_i]
=2[i=1∑nωxi2+i=1∑nbxi−i=1∑nyixi]
=
2
[
∑
i
=
1
n
ω
x
i
2
+
∑
i
=
1
n
(
y
ˉ
−
ω
x
ˉ
)
x
i
−
∑
i
=
1
n
y
i
x
i
]
=2[\sum_{i=1}^n\omega x_i^2 + \sum_{i=1}^n (\bar{y} - \omega \bar{x}) x_i - \sum_{i=1}^n y_ix_i]
=2[i=1∑nωxi2+i=1∑n(yˉ−ωxˉ)xi−i=1∑nyixi]
=
2
[
∑
i
=
1
n
ω
x
i
2
+
n
x
ˉ
(
y
ˉ
−
ω
x
ˉ
)
−
∑
i
=
1
n
y
i
x
i
]
=2[\sum_{i=1}^n\omega x_i^2 + n\bar{x}(\bar{y}- \omega \bar{x})- \sum_{i=1}^n y_ix_i]
=2[i=1∑nωxi2+nxˉ(yˉ−ωxˉ)−i=1∑nyixi]
=
2
[
∑
i
=
1
n
ω
x
i
2
+
n
x
ˉ
y
ˉ
−
n
ω
x
ˉ
2
−
∑
i
=
1
n
y
i
x
i
]
=2[\sum_{i=1}^n\omega x_i^2 +n \bar{x} \bar{y} -n \omega \bar{x}^2- \sum_{i=1}^n y_ix_i]
=2[i=1∑nωxi2+nxˉyˉ−nωxˉ2−i=1∑nyixi]
=
2
[
ω
(
∑
i
=
1
n
x
i
2
−
n
x
ˉ
2
)
+
n
x
ˉ
y
ˉ
−
∑
i
=
1
n
y
i
x
i
]
=2[\omega (\sum_{i=1}^n x_i^2 - n \bar{x}^2) +n \bar{x} \bar{y} - \sum_{i=1}^n y_ix_i]
=2[ω(i=1∑nxi2−nxˉ2)+nxˉyˉ−i=1∑nyixi]
令
ω
\omega
ω的导数为0
ω
(
∑
i
=
1
n
x
i
2
−
n
x
ˉ
2
)
+
n
x
ˉ
y
ˉ
−
∑
i
=
1
n
y
i
x
i
=
0
\omega (\sum_{i=1}^n x_i^2 - n \bar{x}^2) +n \bar{x} \bar{y} - \sum_{i=1}^n y_ix_i = 0
ω(i=1∑nxi2−nxˉ2)+nxˉyˉ−i=1∑nyixi=0
ω
=
∑
i
=
1
n
y
i
x
i
−
n
x
ˉ
y
ˉ
∑
i
=
1
n
x
i
2
−
n
x
ˉ
2
\omega = \frac{\sum_{i=1}^n y_ix_i - n \bar{x} \bar{y}}{\sum_{i=1}^n x_i^2 - n \bar{x}^2}
ω=∑i=1nxi2−nxˉ2∑i=1nyixi−nxˉyˉ
可化简
∵
x
ˉ
=
∑
i
=
1
n
x
i
n
∵\bar{x} = \frac{\sum _{i=1}^n x_i}{n}
∵xˉ=n∑i=1nxi
∴
∑
i
=
1
n
x
i
=
n
x
ˉ
∴\sum _{i=1}^nx_i = n\bar{x}
∴i=1∑nxi=nxˉ
∴
∑
i
=
1
n
x
i
2
−
n
x
ˉ
2
=
∑
i
=
1
n
x
2
−
2
n
x
ˉ
2
+
n
x
ˉ
2
∴\sum_{i=1}^n x_i^2 - n \bar{x}^2 = \sum_{i=1}^n x^2 - 2n\bar{x}^2+n\bar{x}^2
∴i=1∑nxi2−nxˉ2=i=1∑nx2−2nxˉ2+nxˉ2
=
∑
i
=
1
n
x
i
2
−
2
∑
i
=
1
n
x
i
x
ˉ
+
∑
i
=
1
n
x
2
= \sum_{i=1}^n x_i^2 - 2\sum_{i=1}^nx_i\bar{x}+\sum_{i=1}^nx^2
=i=1∑nxi2−2i=1∑nxixˉ+i=1∑nx2
=
∑
i
=
1
n
(
x
i
−
x
ˉ
)
2
=\sum_{i=1}^n(x_i-\bar{x})^2
=i=1∑n(xi−xˉ)2
同理可得
∑
i
=
1
n
y
i
x
i
−
n
x
ˉ
y
ˉ
=
∑
i
=
1
n
y
i
x
i
−
2
n
x
ˉ
y
ˉ
+
n
x
ˉ
y
ˉ
\sum_{i=1}^n y_ix_i - n \bar{x} \bar{y} = \sum_{i=1}^n y_ix_i - 2n\bar{x}\bar{y} + n\bar{x}\bar{y}
i=1∑nyixi−nxˉyˉ=i=1∑nyixi−2nxˉyˉ+nxˉyˉ
=
∑
i
=
1
n
y
i
x
i
−
∑
i
=
1
n
x
i
y
ˉ
−
∑
i
=
1
n
y
i
x
ˉ
+
∑
i
=
1
n
x
ˉ
y
ˉ
=\sum_{i=1}^ny_ix_i - \sum_{i=1}^nx_i\bar{y}-\sum_{i=1}^ny_i\bar{x}+\sum_{i=1}^n\bar{x}\bar{y}
=i=1∑nyixi−i=1∑nxiyˉ−i=1∑nyixˉ+i=1∑nxˉyˉ
=
∑
i
=
1
n
(
x
i
−
x
ˉ
)
(
y
i
−
y
ˉ
)
=\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})
=i=1∑n(xi−xˉ)(yi−yˉ)
∴
ω
=
∑
i
=
1
n
(
x
i
−
x
ˉ
)
(
y
i
−
y
ˉ
)
∑
i
=
1
n
(
x
i
−
x
ˉ
)
2
∴\omega = \frac{\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n(x_i-\bar{x})^2}
∴ω=∑i=1n(xi−xˉ)2∑i=1n(xi−xˉ)(yi−yˉ)
∴
b
=
y
ˉ
−
ω
x
ˉ
,
ω
=
∑
i
=
1
n
(
x
i
−
x
ˉ
)
(
y
i
−
y
ˉ
)
∑
i
=
1
n
(
x
i
−
x
ˉ
)
2
∴b =\bar{y} - \omega \bar{x}, \omega =\frac{\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n(x_i-\bar{x})^2}
∴b=yˉ−ωxˉ,ω=∑i=1n(xi−xˉ)2∑i=1n(xi−xˉ)(yi−yˉ)
x
ˉ
,
y
ˉ
还
原
可
得
\bar{x} , \bar{y}还原可得
xˉ,yˉ还原可得
b
=
∑
i
=
1
n
(
y
i
−
ω
x
i
)
n
,
ω
=
∑
i
=
1
n
y
i
x
i
−
∑
i
=
1
n
x
i
y
i
n
∑
i
=
1
n
x
i
2
+
(
∑
i
=
1
n
x
i
)
2
n
b= \frac{\sum_{i=1}^n(y_i - \omega x_i)}{n} , \omega =\frac{ \sum_{i=1}^ny_ix_i-\frac{\sum_{i=1}^nx_iy_i}{n}}{\sum_{i=1}^nx_i^2+ \frac{(\sum_{i=1}^nx_i)^2}{n} }
b=n∑i=1n(yi−ωxi),ω=∑i=1nxi2+n(∑i=1nxi)2∑i=1nyixi−n∑i=1nxiyi
线性回归之最小二乘法推导
最新推荐文章于 2024-08-21 17:58:45 发布