HHM(forward法求观察序列的概率)

UTF8gbsn

The direct method of computing $P(O|\lambda )$ is ultra expensive.
Fortunately, there are some other ways to get an appropriate
performance. Let’s look at the first improvement called forward method.

• Definition: The forward probability of
  $O=\left(o_1,o_2,\cdots ,o_t\right)$ and $i_t=q_i$ is:
  $${\alpha }_t(i)=P(o_1,o_2,\cdots ,o_t,i_t=q_i|\lambda )$$

$i_t$ means the state of step $t$, and $q_i\in Q$ means the exact state
of $i_t$

Alg:

• Input: $\lambda =(A,B,\pi )$ and observation sequence $O$

• Output: The probability of the observation sequence $P(O|\lambda )$

Steps:

1. initialization: $${\alpha }_1(i)={\pi }_i{b}_i(o_1),i\in (1,2,\cdots ,N)$$

2. recursion:
   $${\alpha }_{t+1}(i)=\sum _{j=1}^N({\alpha }_t(j)a_{ji})b_i(o_{t+1}),i\in (1,2,\cdots ,N)$$

3. stop: $$P(O|\lambda )=\sum _{i=1}^N{\alpha }_T(i)$$

The complexity of this algorithm is $O(N^{2}T)$, which is lower than the
direct method’s $O(N^{N}T)$.

Example:

Let’s consider a HMM $\lambda =(A,B,\pi )$, Q = { 1 , 2 , 3 } Q=\{1,2,3\} Q={1,2,3},
V = { r e d , w h i t e } V=\{red, white\} V={red,white}.

$$A=\begin{array}{r}\left[\begin{array}{ccc}0.5& 0.2& 0.3\\ 0.3& 0.5& 0.2\\ 0.2& 0.3& 0.5\end{array}\right]\end{array}B=\begin{array}{r}\left(\begin{array}{cc}0.5& 0.5\\ 0.4& 0.6\\ 0.7& 0.3\end{array}\right)\end{array}\pi =\begin{array}{r}\left(\begin{array}{c}0.3\\ 0.4\\ 0.4\end{array}\right)\end{array}$$ Let, T = 3 , O = { R e d , W h i t e , R e d } T=3, O=\{Red, White, Red\} T=3,O={Red,White,Red}. How to use forward
method to calculate $P(O|\lambda )$.

1. initialization $$\begin{array}{rl}{\alpha }_1(1)& ={\pi }_1{b}_1(o_1)=0.2\times 0.5=0.1\\ {\alpha }_1(2)& ={\pi }_2{b}_2(o_1)=0.4\times 0.4=0.1\\ a_1(3)& ={\pi }_3{b}_3(o_1)=0.4\times 0.7=0.28\end{array}$$

2. forward $$\begin{array}{rl}& {\alpha }_2(1)=\left[\sum _{i=1}^3{\alpha }_1(i)a_{i1}\right]b_1(o_2)=0.154\times 0.5=0.077\\ & {\alpha }_2(2)=\left[\sum _{i=1}^3{\alpha }_1(i)a_{i2}\right]b_2\left(o_2\right)=0.184\times 0.6=0.1104\\ & {\alpha }_2(3)=\left[\sum _{i=1}^3{\alpha }_1(i)a_{i3}\right]b_3\left(o_2\right)=0.202\times 0.3=0.0606\\ & {\alpha }_3(1)=\left[\sum _{i=1}^3{\alpha }_2(i)a_{i1}\right]b_1\left(o_3\right)=0.04187\\ & {\alpha }_3(2)=\left[\sum _{i=1}^3{\alpha }_2(i)a_{i2}\right]b_2\left(o_3\right)=0.03551\\ & {\alpha }_3(3)=\left[\sum _{i=1}^3{\alpha }_2(i)a_{i3}\right]b_3\left(o_3\right)=0.05284\end{array}$$

3. stop: $$P(O\mid \lambda )=\sum _{i=1}^3{\alpha }_3(i)=0.13022$$