【参数定义】
先验概率分布, P(y=1)=ϕy P ( y = 1 ) = ϕ y
条件概率分布,参数共
2n
2
n
个
P(x1=1|y=1)=ϕ1|y=1P(x2=1|y=1)=ϕ2|y=1⋮P(xn=1|y=1)=ϕn|y=1P(x1=1|y=0)=ϕ1|y=1P(x2=1|y=0)=ϕ2|y=1⋮P(xn=1|y=0)=ϕn|y=1
P
(
x
1
=
1
|
y
=
1
)
=
ϕ
1
|
y
=
1
P
(
x
1
=
1
|
y
=
0
)
=
ϕ
1
|
y
=
1
P
(
x
2
=
1
|
y
=
1
)
=
ϕ
2
|
y
=
1
P
(
x
2
=
1
|
y
=
0
)
=
ϕ
2
|
y
=
1
⋮
⋮
P
(
x
n
=
1
|
y
=
1
)
=
ϕ
n
|
y
=
1
P
(
x
n
=
1
|
y
=
0
)
=
ϕ
n
|
y
=
1
【条件独立性假设】
P(x|y)=P(x1,x2,⋯,xn|y)=P(x1|y)P(x2|y,x1)P(x3|y,x1,x2)⋯P(xn|y,x1,x2,⋯,xn)=∏i=1nP(xi|y) P ( x | y ) = P ( x 1 , x 2 , ⋯ , x n | y ) = P ( x 1 | y ) P ( x 2 | y , x 1 ) P ( x 3 | y , x 1 , x 2 ) ⋯ P ( x n | y , x 1 , x 2 , ⋯ , x n ) = ∏ i = 1 n P ( x i | y )
【参数估计】
ϕy=∑i=1m1{y(i)=1}m ϕ y = ∑ i = 1 m 1 { y ( i ) = 1 } m
ϕj|y=1=∑i=1m1{x(i)j=1∧y(i)=1}∑i=1m1{y(i)=1} ϕ j | y = 1 = ∑ i = 1 m 1 { x j ( i ) = 1 ∧ y ( i ) = 1 } ∑ i = 1 m 1 { y ( i ) = 1 }
ϕj|y=1=∑i=1m1{x(i)j=1∧y(i)=0}∑i=1m1{y(i)=0} ϕ j | y = 1 = ∑ i = 1 m 1 { x j ( i ) = 1 ∧ y ( i ) = 0 } ∑ i = 1 m 1 { y ( i ) = 0 }
【预测未知样本 x^ x ^ ,需要计算后验概率】
P(y=1|x^)=P(x^,y=1)P(x^)=P(y=1)P(x^|y=1)P(x^)∝P(y=1)P(x^|y=1)=P(y=1)∏i=1nP(x^i|y=1) P ( y = 1 | x ^ ) = P ( x ^ , y = 1 ) P ( x ^ ) = P ( y = 1 ) P ( x ^ | y = 1 ) P ( x ^ ) ∝ P ( y = 1 ) P ( x ^ | y = 1 ) = P ( y = 1 ) ∏ i = 1 n P ( x ^ i | y = 1 )
同理
P(y=0|x^)∝P(y=0)∏i=1nP(x^i|y=0)
P
(
y
=
0
|
x
^
)
∝
P
(
y
=
0
)
∏
i
=
1
n
P
(
x
^
i
|
y
=
0
)
如果一定要计算分子 P(x^) P ( x ^ )
P(x^)=∑yP(x^,y)=∑yP(y)P(x^|y)=P(y=1)P(x^|y=1)+P(y=0)P(x^|y=0)=P(y=1)∏i=1nP(x^i|y=1)+P(y=0)∏i=1nP(x^i|y=0) P ( x ^ ) = ∑ y P ( x ^ , y ) = ∑ y P ( y ) P ( x ^ | y ) = P ( y = 1 ) P ( x ^ | y = 1 ) + P ( y = 0 ) P ( x ^ | y = 0 ) = P ( y = 1 ) ∏ i = 1 n P ( x ^ i | y = 1 ) + P ( y = 0 ) ∏ i = 1 n P ( x ^ i | y = 0 )