逻辑回归的推导流程
1、LR推导
P
(
Y
=
1
∣
x
)
=
e
w
x
e
w
x
+
1
=
π
(
x
)
P
(
Y
=
0
∣
x
)
=
1
e
w
x
+
1
=
1
−
π
(
x
)
(1.1)
\begin{aligned} P(Y=1|x) &= \frac{e^{wx}}{e^{wx}+1} = \pi(x) \\ \tag{1.1} P(Y=0|x) &= \frac{1}{e^{wx}+1} = 1 - \pi(x) \\ \end{aligned}
P(Y=1∣x)P(Y=0∣x)=ewx+1ewx=π(x)=ewx+11=1−π(x)(1.1)
2、似然函数
L ( w ) = ∏ i = 1 N [ π ( x i ) ] y i [ π ( 1 − x i ) ] 1 − y i (2.1) L(w) = \prod_{i=1}^N \ [\pi(x_i)]^{y_i} \ [\pi(1-x_i)]^{1-y_i} \tag{2.1} L(w)=i=1∏N [π(xi)]yi [π(1−xi)]1−yi(2.1)
3、对数似然
L ( w ) = ∑ i = 1 N [ y i ⋅ l o g π ( x i ) + ( 1 − y i ) l o g ( 1 − π ( x i ) ) ] = ∑ i = 1 N [ y i ⋅ l o g π ( x i ) − y i l o g ( 1 − π ( x i ) ) + l o g ( 1 − π ( x i ) ) ] = ∑ i = 1 N [ y i ⋅ l o g π ( x i ) 1 − π ( x i ) + l o g ( 1 − π ( x i ) ) ] ( t i p s : 由式子 1.1 可得出 e w x = π ( x ) 1 − π ( x ) ) = ∑ i = 1 N [ y i ⋅ l o g e w x i + l o g 1 e w x i + 1 ] = ∑ i = 1 N [ y i ⋅ w x i − l o g ( e w x i + 1 ) ] (3.1) \begin{aligned} L(w) &= \sum_{i=1}^{N} \ [y_i · log \pi(x_i) \ + \ (1-y_i)log(1-\pi(x_i))] \\ &= \sum_{i=1}^{N} \ [y_i · log \pi(x_i) \ - \ y_i log(1-\pi(x_i)) \ + \ log(1-\pi(x_i))] \\ &= \sum_{i=1}^{N} \ [y_i · log \frac{\pi(x_i)}{1-\pi(x_i)} \ + \ log(1-\pi(x_i))] \\ (tips: &由式子1.1可得出e^{wx} = \frac{\pi(x)}{1-\pi(x)}) \\ &= \sum_{i=1}^{N} \ [y_i·log e^{wx_i} + log\frac{1}{e^{wx_i}+1}] \\ &= \sum_{i=1}^{N} \ [y_i·{wx_i} \ - \ log(e^{wx_i}+1)] \\ \tag{3.1} \end{aligned} L(w)(tips:=i=1∑N [yi⋅logπ(xi) + (1−yi)log(1−π(xi))]=i=1∑N [yi⋅logπ(xi) − yilog(1−π(xi)) + log(1−π(xi))]=i=1∑N [yi⋅log1−π(xi)π(xi) + log(1−π(xi))]由式子1.1可得出ewx=1−π(x)π(x))=i=1∑N [yi⋅logewxi+logewxi+11]=i=1∑N [yi⋅wxi − log(ewxi+1)](3.1)
4、求导
∂ L ( w ) ∂ w = ∑ i = 1 N [ y i x i − e w x i e w x i + 1 x i ] ( t i p s : 由式子 1.1 可得出 e w x i e w x i + 1 = π ( x i ) ) = ∑ i = 1 N [ y i x i − π ( x i ) x i ] (4.1) \begin{aligned} \frac{\partial L(w)}{\partial w} &= \sum_{i=1}^{N} \ [y_ix_i \ - \ \frac{e^{wx_i}}{e^{wx_i}+1}x_i] \\ (tips: 由式子&1.1可得出\frac{e^{wx_i}}{e^{wx_i}+1} = \pi(x_i)) \\ &= \sum_{i=1}^{N} \ [y_ix_i \ - \ \pi(x_i)x_i] \end{aligned} \tag{4.1} ∂w∂L(w)(tips:由式子=i=1∑N [yixi − ewxi+1ewxixi]1.1可得出ewxi+1ewxi=π(xi))=i=1∑N [yixi − π(xi)xi](4.1)
5、更新
w = w 0 + α ∂ L ( w ) ∂ w = w 0 + α ⋅ x i ( y i − π ( x i ) ) (5.1) \begin{aligned} w &= w_0 + \alpha\frac{\partial L(w)}{\partial w} &= w_0 + \alpha · x_i(y_i \ - \ \pi(x_i)) \tag{5.1} \end{aligned} w=w0+α∂w∂L(w)=w0+α⋅xi(yi − π(xi))(5.1)
234
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
