NLP Homework 03 ——冯煜博
背景信息
本文是沈阳航空航天大学自然语言处理实验室的自然语言处理作业题目与本人的解答。
题目描述
(盒子和球模型)假设有3个盒子,每个盒子里装有红白两种颜色的球,盒子里的红白球有下表列出,初始状态分布。
解答
1. 给出HMM模型 μ = ( A , B , π ) \mu=(A, B,\pi) μ=(A,B,π)
A 如下矩阵:
A
box1
box2
box3
b
o
x
1
0.5
0.2
0.3
b
o
x
2
0.3
0.5
0.2
b
o
x
3
0.2
0.3
0.5
\begin{array}{c|lcr} A & \text{box1} & \text{box2} & \text{box3} \\ \hline box1 & 0.5 & 0.2 & 0.3 \\ box2 & 0.3 & 0.5 & 0.2 \\ box3 & 0.2 & 0.3 & 0.5 \end{array}
Abox1box2box3box10.50.30.2box20.20.50.3box30.30.20.5
B 如下矩阵
B
red
white
b
o
x
1
1
2
1
2
b
o
x
2
2
5
3
5
b
o
x
3
7
10
3
10
\begin{array}{c|lcr} B & \text{red} & \text{white} \\ \hline box1 & \frac{1}{2} & \frac{1}{2} \\ box2 & \frac{2}{5} & \frac{3}{5} \\ box3 & \frac{7}{10} & \frac{3}{10} \end{array}
Bbox1box2box3red2152107white2153103
π
\pi
π 为如下矩阵
box1
box2
box3
π
0.2
0.4
0.4
\begin{array}{c|cr} & \text{box1} & \text{box2} & \text{box3} \\ \hline \pi & 0.2 & 0.4 & 0.4 \end{array}
πbox10.2box20.4box30.4
2. 试用前向算法计算观测序列概率:{红,白,红}
O = {红, 白, 红}
step 1 初始化
a 红 ( b o x 1 ) = π b o x 1 b b o x 1 ( 红 ) = 0.2 × 1 2 = 0.1 a_{红}(box1) = \pi_{box1} b_{box1}(红) = 0.2 \times \frac{1}{2} = 0.1 a红(box1)=πbox1bbox1(红)=0.2×21=0.1
a 红 ( b o x 2 ) = π b o x 2 b b o x 2 ( 红 ) = 0.4 × 2 5 = 0.16 a_{红}(box2) = \pi_{box2} b_{box2}(红) = 0.4 \times \frac{2}{5} = 0.16 a红(box2)=πbox2bbox2(红)=0.4×52=0.16
a 红 ( b o x 3 ) = π b o x 3 b b o x 3 ( 红 ) = 0.4 × 7 10 = 0.28 a_{红}(box3) = \pi_{box3} b_{box3}(红) = 0.4 \times \frac{7}{10} = 0.28 a红(box3)=πbox3bbox3(红)=0.4×107=0.28
step 2 归纳计算
a 红白 ( b o x 1 ) = [ ∑ i = 1 3 a 红 ( b o x i ) a b o x i b o x 1 ] b b o x 1 ( 白 ) = [ a 红 ( b o x 1 ) × 0.5 + a 红 ( b o x 2 ) × 0.3 + a 红 ( b o x 3 ) × 0.2 ] × 1 2 = 0.077 \begin{align} a_{红白}(box1) & = \left[ \sum_{i=1}^{3} a_{红}(box_{i})a_{box_{i} box_{1}} \right] b_{box_{1}}(白) \\ & =\left[ a_{红}(box1) \times 0.5 + a_{红}(box2) \times 0.3 + a_{红}(box3) \times 0.2 \right] \times \frac{1}{2} \\ & = 0.077 \end{align} a红白(box1)=[i=1∑3a红(boxi)aboxibox1]bbox1(白)=[a红(box1)×0.5+a红(box2)×0.3+a红(box3)×0.2]×21=0.077
a 红白 ( b o x 2 ) = [ ∑ i = 1 3 a 红 ( b o x i ) a b o x i b o x 2 ] b b o x 2 ( 白 ) = [ a 红 ( b o x 1 ) × 0.2 + a 红 ( b o x 2 ) × 0.5 + a 红 ( b o x 3 ) × 0.3 ] × 3 5 = 0.1104 \begin{align} a_{红白}(box2) & = \left[ \sum_{i=1}^{3} a_{红}(box_{i})a_{box_{i} box_{2}} \right] b_{box2}(白) \\ & = \left[ a_{红}(box1) \times 0.2 + a_{红}(box2) \times 0.5 + a_{红}(box3) \times 0.3 \right] \times \frac{3}{5} \\ & = 0.1104 \end{align} a红白(box2)=[i=1∑3a红(boxi)aboxibox2]bbox2(白)=[a红(box1)×0.2+a红(box2)×0.5+a红(box3)×0.3]×53=0.1104
a 红白 ( b o x 3 ) = [ ∑ i = 1 3 a 红 ( b o x i ) a b o x i b o x 3 ] b b o x 3 ( 白 ) = [ a 红 ( b o x 1 ) × 0.3 + a 红 ( b o x 2 ) × 0.2 + a 红 ( b o x 3 ) × 0.5 ] × 3 10 = 0.0606 \begin{align} a_{红白}(box3) & = \left[ \sum_{i=1}^{3} a_{红}(box_{i})a_{box_{i} box_{3}} \right] b_{box_{3}}(白) \\ & = \left[ a_{红}(box1) \times 0.3 + a_{红}(box2) \times 0.2 + a_{红}(box3) \times 0.5 \right] \times \frac{3}{10} \\ & = 0.0606 \end{align} a红白(box3)=[i=1∑3a红(boxi)aboxibox3]bbox3(白)=[a红(box1)×0.3+a红(box2)×0.2+a红(box3)×0.5]×103=0.0606
a 红白红 ( b o x 1 ) = [ ∑ i = 1 3 a 红白 ( b o x i ) a b o x i b o x 1 ] b b o x 1 ( 红 ) = [ a 红白 ( b o x 1 ) × 0.5 + a 红白 ( b o x 2 ) × 0.3 + a 红白 ( b o x 3 ) × 0.2 ] × 1 2 = 0.04187 \begin{align} a_{红白红}(box1) & = \left[ \sum_{i=1}^{3} a_{红白}(box_{i}) a_{box_{i}box1} \right] b_{box1}(红) \\ & = \left[ a_{红白}(box1) \times 0.5 + a_{红白}(box2) \times 0.3 + a_{红白}(box3) \times 0.2 \right] \times \frac{1}{2} \\ & = 0.04187 \end{align} a红白红(box1)=[i=1∑3a红白(boxi)aboxibox1]bbox1(红)=[a红白(box1)×0.5+a红白(box2)×0.3+a红白(box3)×0.2]×21=0.04187
a 红白红 ( b o x 2 ) = [ ∑ i = 1 3 a 红白 ( b o x i ) a b o x i b o x 2 ] b b o x 2 ( 红 ) = [ a 红白 ( b o x 1 ) × 0.2 + a 红白 ( b o x 2 ) × 0.5 + a 红白 ( b o x 3 ) × 0.3 ] × 2 5 = 0.035512 \begin{align} a_{红白红}(box2) & = \left[ \sum_{i=1}^{3} a_{红白}(box_{i}) a_{box_{i}box2} \right] b_{box2}(红) \\ & = \left[ a_{红白}(box1) \times 0.2 + a_{红白}(box2) \times 0.5 + a_{红白}(box3) \times 0.3 \right] \times \frac{2}{5} \\ & = 0.035512 \end{align} a红白红(box2)=[i=1∑3a红白(boxi)aboxibox2]bbox2(红)=[a红白(box1)×0.2+a红白(box2)×0.5+a红白(box3)×0.3]×52=0.035512
a 红白红 ( b o x 3 ) = [ ∑ i = 1 3 a 红白 ( b o x i ) a b o x i b o x 3 ] b b o x 3 ( 红 ) = [ a 红白 ( b o x 1 ) × 0.3 + a 红白 ( b o x 2 ) × 0.2 + a 红白 ( b o x 3 ) × 0.5 ] × 7 10 = 0.052836 \begin{align} a_{红白红}(box3) & = \left[ \sum_{i=1}^{3} a_{红白}(box_{i}) a_{box_{i}box3} \right] b_{box3}(红) \\ & = \left[ a_{红白}(box1) \times 0.3 + a_{红白}(box2) \times 0.2 + a_{红白}(box3) \times 0.5 \right] \times \frac{7}{10} \\ & = 0.052836 \end{align} a红白红(box3)=[i=1∑3a红白(boxi)aboxibox3]bbox3(红)=[a红白(box1)×0.3+a红白(box2)×0.2+a红白(box3)×0.5]×107=0.052836
step 3 求和终结
P ( O ∣ μ ) = ∑ i = 1 3 a 红白红 ( b o x i ) = a 红白红 ( b o x 1 ) + a 红白红 ( b o x 2 ) + a 红白红 ( b o x 3 ) = 0.04187 + 0.035512 + 0.052836 = 0.130218 \begin{align} P(O | \mu) & = \sum_{i=1}^{3}a_{红白红} (box_{i}) \\ & = a_{红白红}(box1) + a_{红白红}(box2) + a_{红白红}(box3) \\ & = 0.04187 + 0.035512 + 0.052836 \\ & = 0.130218 \end{align} P(O∣μ)=i=1∑3a红白红(boxi)=a红白红(box1)+a红白红(box2)+a红白红(box3)=0.04187+0.035512+0.052836=0.130218
3. 试用维特比算法,求解观测序列{红,白,红}的最优状态序列。 (要求给详细计算过程)
假设从盒子中取球是放回抽样概率模型。
step 1 初始化
δ 红 ( b o x 1 ) = π b o x 1 b b o x 1 ( 红 ) = 0.2 × 1 2 = 0.1 \delta_{红}(box1) = \pi_{box1}b_{box1}(红) = 0.2 \times \frac{1}{2} = 0.1 δ红(box1)=πbox1bbox1(红)=0.2×21=0.1
δ 红 ( b o x 2 ) = π b o x 2 b b o x 2 ( 红 ) = 0.4 × 2 5 = 0.16 \delta_{红}(box2) = \pi_{box2}b_{box2}(红) = 0.4 \times \frac{2}{5} = 0.16 δ红(box2)=πbox2bbox2(红)=0.4×52=0.16
δ 红 ( b o x 3 ) = π b o x 3 b b o x 3 ( 红 ) = 0.4 × 7 10 = 0.28 \delta_{红}(box3) = \pi_{box3}b_{box3}(红) = 0.4 \times \frac{7}{10} = 0.28 δ红(box3)=πbox3bbox3(红)=0.4×107=0.28
φ 红 ( b o x 1 ) = 0 \varphi_{红}(box1) = 0 φ红(box1)=0
φ 红 ( b o x 2 ) = 0 \varphi_{红}(box2) = 0 φ红(box2)=0
φ 红 ( b o x 3 ) = 0 \varphi_{红}(box3) = 0 φ红(box3)=0
step 2 归纳计算 1
δ 红白 ( b o x 1 ) = m a x [ δ 红 ( b o x 1 ) a b o x 1 b o x 1 , δ 红 ( b o x 2 ) a b o x 2 b o x 1 , δ 红 ( b o x 3 ) a b o x 3 b o x 1 ] b b o x 1 ( 白 ) = 0.28 × 0.2 × 1 2 = 0.028 \begin{align} \delta_{红白}(box1) & = max \left[ \delta_{红}(box1)a_{box1box1} , \delta_{红}(box2)a_{box2box1} , \delta_{红}(box3)a_{box3box1} \right] b_{box1}(白) \\ & = 0.28 \times 0.2 \times \frac{1}{2} \\ & = 0.028 \end{align} δ红白(box1)=max[δ红(box1)abox1box1,δ红(box2)abox2box1,δ红(box3)abox3box1]bbox1(白)=0.28×0.2×21=0.028
δ 红白 ( b o x 2 ) = m a x [ δ 红 ( b o x 1 ) a b o x 1 b o x 2 , δ 红 ( b o x 2 ) a b o x 2 b o x 2 , δ 红 ( b o x 3 ) a b o x 3 b o x 2 ] b b o x 1 ( 白 ) = 0.28 × 0.3 × 3 5 = 0.0504 \begin{align} \delta_{红白}(box2) & = max \left[ \delta_{红}(box1)a_{box1box2} , \delta_{红}(box2)a_{box2box2} , \delta_{红}(box3)a_{box3box2} \right] b_{box1}(白) \\ & = 0.28 \times 0.3 \times \frac{3}{5} \\ & = 0.0504 \end{align} δ红白(box2)=max[δ红(box1)abox1box2,δ红(box2)abox2box2,δ红(box3)abox3box2]bbox1(白)=0.28×0.3×53=0.0504
δ 红白 ( b o x 3 ) = m a x [ δ 红 ( b o x 1 ) a b o x 1 b o x 3 , δ 红 ( b o x 2 ) a b o x 2 b o x 3 , δ 红 ( b o x 3 ) a b o x 3 b o x 3 ] b b o x 3 ( 白 ) = 0.28 × 0.5 × 3 10 = 0.042 \begin{align} \delta_{红白}(box3) & = max \left[ \delta_{红}(box1)a_{box1box3} , \delta_{红}(box2)a_{box2box3} , \delta_{红}(box3)a_{box3box3} \right] b_{box3}(白) \\ & = 0.28 \times 0.5 \times \frac{3}{10} \\ & = 0.042 \end{align} δ红白(box3)=max[δ红(box1)abox1box3,δ红(box2)abox2box3,δ红(box3)abox3box3]bbox3(白)=0.28×0.5×103=0.042
记忆回退路径 1
φ 红白 ( b o x 1 ) = a r g m a x [ δ 红 ( b o x 1 ) a b o x 1 b o x 1 , δ 红 ( b o x 2 ) a b o x 2 b o x 1 , δ 红 ( b o x 3 ) a b o x 3 b o x 1 ] b b o x 1 ( 白 ) = 1 2 δ 红 ( b o x 3 ) a b o x 3 b o x 1 \begin{align} \varphi_{红白}(box1) & = argmax \left[ \delta_{红}(box1)a_{box1box1} , \delta_{红}(box2)a_{box2box1} , \delta_{红}(box3)a_{box3box1} \right] b_{box1}(白) \\ & = \frac{1}{2} \delta_{红}(box3)a_{box3box1} \end{align} φ红白(box1)=argmax[δ红(box1)abox1box1,δ红(box2)abox2box1,δ红(box3)abox3box1]bbox1(白)=21δ红(box3)abox3box1
φ 红白 ( b o x 2 ) = a r g m a x [ δ 红 ( b o x 1 ) a b o x 1 b o x 2 , δ 红 ( b o x 2 ) a b o x 2 b o x 2 , δ 红 ( b o x 3 ) a b o x 3 b o x 2 ] b b o x 2 ( 白 ) = 3 5 δ 红 ( b o x 3 ) a b o x 3 b o x 2 \begin{align} \varphi_{红白}(box2) & = argmax \left[ \delta_{红}(box1)a_{box1box2} , \delta_{红}(box2)a_{box2box2} , \delta_{红}(box3)a_{box3box2} \right] b_{box2}(白) \\ & = \frac{3}{5} \delta_{红}(box3) a_{box3box2} \end{align} φ红白(box2)=argmax[δ红(box1)abox1box2,δ红(box2)abox2box2,δ红(box3)abox3box2]bbox2(白)=53δ红(box3)abox3box2
δ 红白 ( b o x 3 ) = a r g m a x [ δ 红 ( b o x 1 ) a b o x 1 b o x 3 , δ 红 ( b o x 2 ) a b o x 2 b o x 3 , δ 红 ( b o x 3 ) a b o x 3 b o x 3 ] b b o x 3 ( 白 ) = 3 10 δ 红 ( b o x 3 ) a b o x 3 b o x 3 \begin{align} \delta_{红白}(box3) & = argmax \left[ \delta_{红}(box1)a_{box1box3} , \delta_{红}(box2)a_{box2box3} , \delta_{红}(box3)a_{box3box3} \right] b_{box3}(白) \\ & = \frac{3}{10} \delta_{红}(box3)a_{box3box3} \\ \end{align} δ红白(box3)=argmax[δ红(box1)abox1box3,δ红(box2)abox2box3,δ红(box3)abox3box3]bbox3(白)=103δ红(box3)abox3box3
归纳计算 2
δ 红白红 ( b o x 1 ) = m a x [ δ 红白 ( b o x 1 ) a b o x 1 b o x 1 , δ 红白 ( b o x 2 ) a b o x 2 b o x 1 , δ 红白 ( b o x 3 ) a b o x 3 b o x 1 ] b b o x 1 ( 红 ) = 0.0504 × 0.3 × 1 2 = 0.0756 \begin{align} \delta_{红白红}(box1) & = max \left[ \delta_{红白}(box1)a_{box1box1} , \delta_{红白}(box2)a_{box2box1} , \delta_{红白}(box3)a_{box3box1} \right] b_{box1}(红) \\ & = 0.0504 \times 0.3 \times \frac{1}{2} \\ & = 0.0756 \end{align} δ红白红(box1)=max[δ红白(box1)abox1box1,δ红白(box2)abox2box1,δ红白(box3)abox3box1]bbox1(红)=0.0504×0.3×21=0.0756
δ 红白红 ( b o x 2 ) = m a x [ δ 红白 ( b o x 1 ) a b o x 1 b o x 2 , δ 红白 ( b o x 2 ) a b o x 2 b o x 2 , δ 红白 ( b o x 3 ) a b o x 3 b o x 2 ] b b o x 1 ( 红 ) = 0.0504 × 0.5 × 2 5 = 0.01008 \begin{align} \delta_{红白红}(box2) & = max \left[ \delta_{红白}(box1)a_{box1box2} , \delta_{红白}(box2)a_{box2box2} , \delta_{红白}(box3)a_{box3box2} \right] b_{box1}(红) \\ & = 0.0504 \times 0.5 \times \frac{2}{5} \\ & = 0.01008 \end{align} δ红白红(box2)=max[δ红白(box1)abox1box2,δ红白(box2)abox2box2,δ红白(box3)abox3box2]bbox1(红)=0.0504×0.5×52=0.01008
δ 红白红 ( b o x 3 ) = m a x [ δ 红白 ( b o x 1 ) a b o x 1 b o x 3 , δ 红白 ( b o x 2 ) a b o x 2 b o x 3 , δ 红白 ( b o x 3 ) a b o x 3 b o x 3 ] b b o x 3 ( 红 ) = 0.042 × 0.5 × 7 10 = 0.0147 \begin{align} \delta_{红白红}(box3) & = max \left[ \delta_{红白}(box1)a_{box1box3} , \delta_{红白}(box2)a_{box2box3} , \delta_{红白}(box3)a_{box3box3} \right] b_{box3}(红) \\ & = 0.042 \times 0.5 \times \frac{7}{10} \\ & = 0.0147 \end{align} δ红白红(box3)=max[δ红白(box1)abox1box3,δ红白(box2)abox2box3,δ红白(box3)abox3box3]bbox3(红)=0.042×0.5×107=0.0147
记忆回退路径 2
φ 红白红 ( b o x 1 ) = a r g m a x [ δ 红白 ( b o x 1 ) a b o x 1 b o x 1 , δ 红白 ( b o x 2 ) a b o x 2 b o x 1 , δ 红白 ( b o x 3 ) a b o x 3 b o x 1 ] b b o x 1 ( 红 ) = 1 2 δ 红 ( b o x 2 ) a b o x 2 b o x 1 \begin{align} \varphi_{红白红}(box1) & = argmax \left[ \delta_{红白}(box1)a_{box1box1} , \delta_{红白}(box2)a_{box2box1} , \delta_{红白}(box3)a_{box3box1} \right] b_{box1}(红) \\ & = \frac{1}{2} \delta_{红}(box2)a_{box2box1} \end{align} φ红白红(box1)=argmax[δ红白(box1)abox1box1,δ红白(box2)abox2box1,δ红白(box3)abox3box1]bbox1(红)=21δ红(box2)abox2box1
φ 红白红 ( b o x 2 ) = a r g m a x [ δ 红白 ( b o x 1 ) a b o x 1 b o x 2 , δ 红白 ( b o x 2 ) a b o x 2 b o x 2 , δ 红白 ( b o x 3 ) a b o x 3 b o x 2 ] b b o x 2 ( 红 ) = 2 5 δ 红白 ( b o x 2 ) a b o x 2 b o x 2 \begin{align} \varphi_{红白红}(box2) & = argmax \left[ \delta_{红白}(box1)a_{box1box2} , \delta_{红白}(box2)a_{box2box2} , \delta_{红白}(box3)a_{box3box2} \right] b_{box2}(红) \\ & = \frac{2}{5} \delta_{红白}(box2) a_{box2box2} \end{align} φ红白红(box2)=argmax[δ红白(box1)abox1box2,δ红白(box2)abox2box2,δ红白(box3)abox3box2]bbox2(红)=52δ红白(box2)abox2box2
δ 红白红 ( b o x 3 ) = a r g m a x [ δ 红白 ( b o x 1 ) a b o x 1 b o x 3 , δ 红白 ( b o x 2 ) a b o x 2 b o x 3 , δ 红白 ( b o x 3 ) a b o x 3 b o x 3 ] b b o x 3 ( 白 ) = 7 10 δ 红白 ( b o x 3 ) a b o x 3 b o x 3 \begin{align} \delta_{红白红}(box3) & = argmax \left[ \delta_{红白}(box1)a_{box1box3} , \delta_{红白}(box2)a_{box2box3} , \delta_{红白}(box3)a_{box3box3} \right] b_{box3}(白) \\ & = \frac{7}{10} \delta_{红白}(box3)a_{box3box3} \\ \end{align} δ红白红(box3)=argmax[δ红白(box1)abox1box3,δ红白(box2)abox2box3,δ红白(box3)abox3box3]bbox3(白)=107δ红白(box3)abox3box3
step 3 终结
Q ^ 红白红 = a r g m a x [ δ 红白红 ( b o x 1 ) , δ 红白红 ( b o x 2 ) , δ 红白红 ( b o x 3 ) ] = δ 红白红 ( b o x 3 ) \begin{align} \hat{Q}_{红白红} & = argmax \left[ \delta_{红白红}(box1) , \delta_{红白红}(box2) , \delta_{红白红}(box3) \right] \\ & = \delta_{红白红}(box3) \end{align} Q^红白红=argmax[δ红白红(box1),δ红白红(box2),δ红白红(box3)]=δ红白红(box3)
P ^ ( Q ^ 红白红 ) = m a x [ δ 红白红 ( b o x 1 ) , δ 红白红 ( b o x 2 ) , δ 红白红 ( b o x 3 ) ] = 0.0147 \begin{align} \hat{P}(\hat{Q}_{红白红}) & = max \left[ \delta_{红白红}(box1) , \delta_{红白红}(box2) , \delta_{红白红}(box3) \right] \\ & = 0.0147 \end{align} P^(Q^红白红)=max[δ红白红(box1),δ红白红(box2),δ红白红(box3)]=0.0147
step 4 路径(状态序列)回溯
q ^ 红白 = φ 红白红 ( q ^ 红白红 ) = φ 红白红 [ δ 红白红 ( b o x 3 ) ] \begin{align} \hat{q}_{红白} = \varphi_{红白红}(\hat{q}_{红白红}) = \varphi_{红白红} \left[ \delta_{红白红}(box3) \right] \end{align} q^红白=φ红白红(q^红白红)=φ红白红[δ红白红(box3)]
q ^ 红 = φ 红白 ( q ^ 红白 ) = φ 红白 [ δ 红白 ( b o x 2 ) ] \begin{align} \hat{q}_{红} = \varphi_{红白}(\hat{q}_{红白}) = \varphi_{红白} \left[ \delta_{红白}(box2) \right] \end{align} q^红=φ红白(q^红白)=φ红白[δ红白(box2)]
q ^ = φ 红 ( q ^ 红 ) = φ 红 [ δ 红 ( b o x 3 ) ] \begin{align} \hat{q} = \varphi_{红}(\hat{q}_{红}) = \varphi_{红} \left[ \delta_{红}(box3) \right] \end{align} q^=φ红(q^红)=φ红[δ红(box3)]