写在前面的话
《统计学习方法》中与预测有关的知识点,图片拍得有些不清楚,大家可以去查一下这两个知识点。同时还有回归也涉及预测。
隐马尔可夫模型的预测算法
条件随机场的预测算法
回归
线性回归
- 回归模型:
f ( x i ) = w T x + b f\left(x_{i}\right)=\mathbf{w}^{\mathbf{T}} \mathbf{x}+b f(xi)=wTx+b - 权重更新(反向传播):使用梯度下降逼近
w i ∗ = w i + α ∂ L o s s ∂ w i w_{i}^{*}=w_{i}+\alpha \frac{\partial L o s s}{\partial w_{i}} wi∗=wi+α∂wi∂Loss
lr = LinearRegression() #利用已知数据与标签对模型进行训练
lr.fit(train_data, train_label) #对未知数据进行预测
predict = lr.predict(test_data)
对数几率回归(Logistic回归):二分类任务
- 回归模型:
f ( x i ) = σ ( w T x + b ) σ = Sigmoid = 1 1 + e − z z = w T x + b \begin{array}{l} f\left(x_{i}\right)=\sigma\left(\mathbf{w}^{\mathbf{T}} \mathbf{x}+b\right) \\ \sigma=\text { Sigmoid }=\frac{1}{1+e^{-z}} \\ z=\mathbf{w}^{\mathbf{T}} \mathbf{x}+b \end{array} f(xi)=σ(wTx+b)σ= Sigmoid =1+e−z1z=wTx+b - 权重更新 (反向传播):使用梯度下降逼近
对 sigmoid 求导:
Loss
=
∑
i
=
1
n
(
y
ln
f
(
x
i
)
+
(
1
−
y
)
ln
1
−
f
(
x
i
)
)
w
i
∗
=
w
i
+
α
∂
L
o
s
s
∂
w
i
=
w
i
−
α
∑
i
=
1
n
(
f
(
x
i
)
−
y
i
)
x
i
b
∗
=
b
+
α
∂
L
o
s
∂
b
=
b
−
α
∑
i
=
1
n
(
f
(
x
i
)
−
y
i
)
\begin{array}{l} \text { Loss }=\sum_{i=1}^{n}\left(y \ln f\left(\mathbf{x}_{\mathbf{i}}\right)+(1-y) \ln 1-f\left(\mathbf{x}_{\mathbf{i}}\right)\right) \\ w_{i}^{*}=w_{i}+\alpha \frac{\partial L o s s}{\partial w_{i}}=w_{i}-\alpha \sum_{i=1}^{n}\left(f\left(\mathbf{x}_{\mathbf{i}}\right)-y_{i}\right) x_{i} \\ b^{*}=b+\alpha \frac{\partial L o s}{\partial b}=b-\alpha \sum_{i=1}^{n}\left(f\left(\mathbf{x}_{\mathbf{i}}\right)-y_{i}\right) \end{array}
Loss =∑i=1n(ylnf(xi)+(1−y)ln1−f(xi))wi∗=wi+α∂wi∂Loss=wi−α∑i=1n(f(xi)−yi)xib∗=b+α∂b∂Los=b−α∑i=1n(f(xi)−yi)
logreg = LogisticRegression(C=100)
logreg.fit(train_data, train_label)
predict = logreg.predict(test_data)