# 序列的算法（一·b）隐马尔可夫模型

47 篇文章 21 订阅

##序言

a. 湿货部分要浅入浅出，形象生动，读得明白。
b. 干货部分要一文以蔽之，公式罗列，看得通透；

#Unigram
Unigram模型认为序列中的每一项都是独立发生的，所以很自然，假设我们有N个序列，每个序列长度是 M n M_n ，那么整个序列的联合概率分布就是：
P ( X ) = ∏ i N ∏ j M i p ( X i j ) P(X) = \prod_i^N\prod_j^{M_i}p\left(X_i^j\right)

P ( X ) = ∏ v p ( x = v ) c o u n t ( v ) P(X) = \prod_vp(x=v)^{count(v)}

L = l n P ( X ) = ∑ v c o u n t ( v ) ∗ l n p ( x = v ) 并 且 ∑ v p ( v ) = 1 L = lnP(X) = \sum_vcount(v)*ln^{p(x=v)}\\ 并且 \sum_vp\left(v\right) = 1

L ′ = l n P ( X ) = ∑ v c o u n t ( v ) ∗ l n p ( v ) + λ ( 1 − ∑ v p ( v ) ) ∂ L ′ ∂ p ( v ) = c o u n t ( v ) p ( v ) − λ ∂ L ′ ∂ λ = 1 − ∑ v p ( v ) \begin{aligned} L&#x27; = lnP(X) &amp;= \sum_vcount(v)*ln^{p(v)} + \lambda(1-\sum_vp\left(v\right))\\ \frac{\partial L&#x27;}{\partial p(v)} &amp;= \frac{count(v)}{p(v)}-\lambda\\ \frac{\partial L&#x27;}{\partial \lambda} &amp;= 1-\sum_vp\left(v\right) \end{aligned}

p ( v ) = c o u n t ( v ) λ 并 且 ∑ v p ( v ) = ∑ v c o u n t ( v ) λ = 1 所 以 λ = c o u n t ( ∗ ) p(v)=\frac{count(v)}{\lambda} 并且\\ \sum_vp\left(v\right) = \sum_v\frac{count(v)}{\lambda} =1\\ 所以\lambda = count(*)

p ( v ) = c o u n t ( v ) c o u n t ( ∗ ) p(v) = \frac{count(v)}{count(*)}

#马尔可夫模型

p ( x t = v i , x t + 1 = v j ) = c o u n t ( v i , v j ) c o u n t ( v i , ∗ ) p(x^{t}=v_i,x^{t+1}=v_j) = \frac{count(v_i,v_j)}{count(v_i,*)}

#隐马尔可夫模型HMM

P ( o , h ∣ θ ) = p ( h 1 ∣ θ ) ∏ t = 2 T p ( h t ∣ h t − 1 , θ ) ∏ t = 1 T p ( o t ∣ h t , θ ) P(o,h|\theta) = p(h_1|\theta)\prod_{t=2}^Tp(h_t|h_{t-1},\theta)\prod_{t=1}^Tp(o_t|h_t,\theta)

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \pi_{q_i} &= p…

P ( o , h ∣ θ ) = π h 1 ∏ t = 2 T A h t − 1 h t ∏ t = 1 T B h t o t P(o,h|\theta) = \pi_{h_1}\prod_{t=2}^TA_{h_{t-1}h_t}\prod_{t=1}^TB_{h_to_t}

p ( o ∣ θ ) = ∑ h p ( o , h ∣ θ ) p(o|\theta) = \sum_hp(o,h|\theta)

#Baum-Welch算法（EM算法）

EM算法作为十大数据挖掘算法之一，后面我们可以单独开一篇博客学习，我自己现在也没有理解得很深入，但在这里你只需要知道它有两步，Q&M，Q步骤中计算Q函数，M步骤中最大化Q函数

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ Q(\theta,\thet…

F = ∑ h p ( h ∣ o , θ o l d ) ∗ l n   π h 1 F = \sum_h p(h|o,\theta^{old})*ln\ \pi_{h_1}

∑ h ∈ H q ′ p ( h ∣ o , θ o l d ) ∗ l n   π q \sum_{h \in H&#x27;_q} p(h|o,\theta^{old})*ln\ \pi_{q}
H q ′ H&#x27;_q H H h 1 = q h_1=q 的所有可能的 h h 序列构成的集合，此时 ∑ h ∈ H ′ p ( h ∣ o , θ o l d ) \sum_{h \in H&#x27;} p(h|o,\theta^{old}) 发生了微妙的变化：
∑ h ∈ H ′ p ( h ∣ o , θ o l d ) = p ( h 1 = q ∣ o , θ o l d ) \sum_{h \in H&#x27;} p(h|o,\theta^{old}) = p(h_1=q|o,\theta^{old})

F = ∑ q p ( h 1 = q ∣ o , θ o l d ) l n   π q F = \sum_qp(h_1=q|o,\theta^{old})ln\ \pi_q

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ Q(\theta,\…

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \gamma_t &= p(…

Q ( θ , θ o l d ) = ∑ q γ 1 ( q ) l n   π q + ∑ t = 2 ∑ q 1 ∈ Q ∑ q 2 ∈ Q ξ t ( q 1 , q 2 ) ∗ l n A q 1 , q 2 + ∑ t ∑ q γ t ( q ) l n   B q , o t Q(\theta,\theta^{old}) = \sum_q\gamma_1(q)ln\ \pi_q + \sum_{t=2}\sum_{q_1 \in Q}\sum_{q_2 \in Q}\xi_t(q_1,q_2)*lnA_{q_1,q_2} + \sum_{t}\sum_q\gamma_t(q)ln\ B_{q,o_t}

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \frac{\partial…

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ A_{q_1,q_2} &=…

#前向后向算法（forward-backward）

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \gamma_t &…

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \alpha(h_t…

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \alpha(h_t…

α ( h 1 ) q = π q ∗ p ( o 1 ∣ q ) \alpha(h_1)^q = \pi_q*p(o_1|q)

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \beta(h_t)…

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \gamma_t(q…

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ \xi_t(q_1,…

KaTeX parse error: No such environment: align at position 8: \begin{̲a̲l̲i̲g̲n̲}̲ p(o) &= \s…

#Viterbi算法

q t ∗ = arg ⁡ max ⁡ q i [ γ t ( q i ) ] q_t^* = \mathop{\arg\max}_{q_i}[\gamma_t(q_i)]

f t ( q ) = max ⁡ h 1... t − 1 p ( h t = q , o 1... t ) f_t(q) = \mathop{\max}_{h_{1...t-1}}p(h_t=q,o_{1...t})

f t ( q ) = ( max ⁡ q ′ f t − 1 ( q ′ ) ∗ A q ′ , q ) ∗ B q , o t f_t(q) = \left(\mathop{\max}_q&#x27;f_{t-1}(q&#x27;)*A_{q&#x27;,q}\right)*B_{q,o_t}

f 1 ( q ) = π q ∗ B q , o 1 f_1(q) = \pi_q*B_{q,o_1}

g t ( q ) = arg ⁡ max ⁡ q ′ f t − 1 ( q ′ ) ∗ A q ′ , q g_t(q) = \mathop{\arg\max}_{q&#x27;}f_{t-1}(q&#x27;)*A_{q&#x27;,q}
g t ( q ) g_t(q) 表示当t时刻隐藏状态确认是q的情况下，它的上一时刻状态应该是什么，我们最后在 f T ( q ) f_T(q) 找到一个概率最大的q，只需要要根据这个q一步步回溯回去就可以找到对应的序列了,如图e所示，我们在最后一步挑选了q1。

#尾巴

之前在不同的书上都看过多次隐马尔可夫模型，但记忆不深刻，时间稍微一长就忘掉了，所以借着这个机会从头到尾梳理一次，按照概率表示)->(EM算法)->(前向后向算法)->(Viterbi)这个顺序理解相当酣畅，都是有需求才进入下一步的介绍，承上启下，相信这次要记得久一点了。


• 3
点赞
• 11
收藏
觉得还不错? 一键收藏
• 5
评论
04-23 266
07-20 4840
03-23 239
04-17 607
08-05 1036
07-25 1653
02-21 6万+
04-16 2万+

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。