解析kaldi中yesno的语音分帧和模型

看了很多内容，还是不明白kaldi是如何处理每一段语音的。可以做下面的实验

为了结合kaldi，可以将kaldi中yesno的对齐结果解析出来。步骤如下：
１、跑完yesno的run.sh脚本，获得模型0.mdl、HCLG.fst和ali.1.gz
２、命令行cd kaldi/egs/yesno/s5/exp/mono0a
３、执行命令~/kaldi/src/bin/copy-int-vector "ark:gunzip -c ali.1.gz|" ark,t:ali.txt
４、上一步生成了ali.txt，这就是单音素对齐之后的结果。就是上文中的step中的内容。每个语音文件都被分成帧，每一帧对应一个Transition-id ，注意并不是状态。

５、执行命令~/kaldi/src/bin/show-transitions phones.txt 0.mdl
这个命令的输出结果就是Transition-state，phone，hmm-state，pdf和Transition-id之间的关系。通过这些信息就可以找到上面语音每一帧对应的phone。

boystray@boystray-All-Series:~/kaldi/egs/aishell/s5/exp/mono$ ~/kaldi/src/bin/show-transitions phones.txt 0.mdl
/home/boystray/kaldi/src/bin/show-transitions phones.txt 0.mdl 
Transition-state 1: phone = sil hmm-state = 0 pdf = 0
 Transition-id = 1 p = 0.25 [self-loop]
 Transition-id = 2 p = 0.25 [0 -> 1]
 Transition-id = 3 p = 0.25 [0 -> 2]
 Transition-id = 4 p = 0.25 [0 -> 3]
Transition-state 2: phone = sil hmm-state = 1 pdf = 1
 Transition-id = 5 p = 0.25 [self-loop]
 Transition-id = 6 p = 0.25 [1 -> 2]
 Transition-id = 7 p = 0.25 [1 -> 3]
 Transition-id = 8 p = 0.25 [1 -> 4]
Transition-state 3: phone = sil hmm-state = 2 pdf = 2
 Transition-id = 9 p = 0.25 [2 -> 1]
 Transition-id = 10 p = 0.25 [self-loop]
 Transition-id = 11 p = 0.25 [2 -> 3]
 Transition-id = 12 p = 0.25 [2 -> 4]
Transition-state 4: phone = sil hmm-state = 3 pdf = 3
 Transition-id = 13 p = 0.25 [3 -> 1]
 Transition-id = 14 p = 0.25 [3 -> 2]
 Transition-id = 15 p = 0.25 [self-loop]
 Transition-id = 16 p = 0.25 [3 -> 4]
Transition-state 5: phone = sil hmm-state = 4 pdf = 4
 Transition-id = 17 p = 0.75 [self-loop]
 Transition-id = 18 p = 0.25 [4 -> 5]
Transition-state 6: phone = a1 hmm-state = 0 pdf = 5
 Transition-id = 19 p = 0.75 [self-loop]
 Transition-id = 20 p = 0.25 [0 -> 1]
Transition-state 7: phone = a1 hmm-state = 1 pdf = 6
 Transition-id = 21 p = 0.75 [self-loop]
 Transition-id = 22 p = 0.25 [1 -> 2]

６、进入graph_tgpr目录，执行~/kaldi/tools/openfst-1.6.7/bin/fstdraw  HCLG.fst  | dot -Tps |ps2pdf - HCLG.pdf
就可以看到图形化的HCLG.

7、使用ali-to-phones打印出每段语音phone序列

 ~/kaldi/src/bin/ali-to-phones --write-lengths=true final.mdl ark:ali.txt ark,t:3.txt
/home/boystray/kaldi/src/bin/ali-to-phones --write-lengths=true final.mdl ark:ali.txt ark,t:3.txt
LOG (ali-to-phones[5.5]:main():ali-to-phones.cc:134) Done 31 utterances.

3.txt的内容如下，每个语音后面是分号隔开的一对数字。第一个数字是phone编号，第二个数字是重复次数。计算第一行的语音0_0_0_0_1_1_1_1一共是633帧（就是把第一行所有分组的第二个数字都加起来）

0_0_0_0_1_1_1_1 1 85 ; 3 37 ; 3 57 ; 3 60 ; 1 20 ; 3 42 ; 1 29 ; 2 26 ; 1 34 ; 2 27 ; 1 38 ; 2 28 ; 1 40 ; 2 22 ; 1 88 
0_0_0_1_0_0_0_1 1 109 ; 3 37 ; 3 53 ; 3 51 ; 1 16 ; 2 26 ; 1 21 ; 3 38 ; 3 53 ; 3 55 ; 1 20 ; 2 23 ; 1 172 
0_0_0_1_0_1_1_0 1 104 ; 3 37 ; 3 56 ; 3 57 ; 1 19 ; 2 28 ; 1 21 ; 3 36 ; 1 20 ; 2 27 ; 1 29 ; 2 24 ; 1 25 ; 3 25 ; 1 101 
0_0_1_0_0_0_1_0 1 93 ; 3 37 ; 3 57 ; 1 16 ; 2 30 ; 1 20 ; 3 38 ; 3 52 ; 3 53 ; 1 14 ; 2 27 ; 1 19 ; 3 33 ; 1 116 
0_0_1_0_0_1_1_0 1 79 ; 3 36 ; 3 49 ; 1 31 ; 2 25 ; 1 22 ; 3 38 ; 3 57 ; 1 24 ; 2 23 ; 1 33 ; 2 27 ; 1 27 ; 3 24 ; 1 105 
0_0_1_0_0_1_1_1 1 100 ; 3 31 ; 3 58 ; 1 17 ; 2 27 ; 1 25 ; 3 37 ; 3 53 ; 1 20 ; 2 28 ; 1 34 ; 2 26 ; 1 35 ; 2 21 ; 1 116 
0_0_1_0_1_0_0_0 1 128 ; 3 34 ; 3 3 ; 1 24 ; 2 22 ; 1 28 ; 3 35 ; 1 33 ; 2 25 ; 1 33 ; 3 34 ; 3 61 ; 1 26 ; 3 32 ; 1 102 
0_0_1_0_1_0_0_1 1 76 ; 3 39 ; 3 63 ; 1 14 ; 2 27 ; 1 29 ; 3 36 ; 1 19 ; 2 29 ; 1 26 ; 3 38 ; 3 52 ; 1 15 ; 2 22 ; 1 189 
0_0_1_0_1_0_1_1 1 101 ; 3 31 ; 3 55 ; 1 28 ; 2 27 ; 1 24 ; 3 31 ; 1 26 ; 2 24 ; 1 25 ; 3 32 ; 1 28 ; 2 24 ; 1 28 ; 2 21 ; 1 117 
0_0_1_1_0_0_0_1 1 64 ; 3 38 ; 3 51 ; 1 21 ; 2 25 ; 1 30 ; 2 26 ; 1 29 ; 3 36 ; 3 53 ; 1 25 ; 3 38 ; 1 18 ; 2 23 ; 1 104 
0_0_1_1_0_1_0_0 1 77 ; 3 38 ; 3 56 ; 1 22 ; 2 27 ; 1 27 ; 2 27 ; 1 29 ; 3 41 ; 1 22 ; 2 29 ; 1 27 ; 3 41 ; 1 8 ; 3 29 ; 1 94 
0_0_1_1_0_1_1_0 1 98 ; 3 36 ; 3 58 ; 1 19 ; 2 29 ; 1 28 ; 2 29 ; 1 26 ; 3 37 ; 1 24 ; 2 29 ; 1 34 ; 2 29 ; 1 23 ; 3 27 ; 1 153 
0_0_1_1_0_1_1_1 1 88 ; 3 35 ; 3 57 ; 1 21 ; 2 24 ; 1 31 ; 2 25 ; 1 26 ; 3 35 ; 1 20 ; 2 27 ; 1 28 ; 2 27 ; 1 28 ; 2 20 ; 1 111 
0_0_1_1_1_0_0_0 1 74 ; 3 39 ; 1 17 ; 3 17 ; 1 48 ; 2 25 ; 1 34 ; 2 27 ; 1 39 ; 2 24 ; 1 38 ; 3 35 ; 1 22 ; 3 36 ; 1 26 ; 3 31 ; 1 97 
0_0_1_1_1_0_0_1 1 84 ; 3 36 ; 3 51 ; 1 19 ; 2 27 ; 1 29 ; 2 25 ; 1 25 ; 2 26 ; 1 50 ; 3 31 ; 3 54 ; 1 21 ; 2 23 ; 1 137 

 

8、进入yesno/s5/mfcc目录，执行

~/kaldi/src/featbin/copy-feats ark:raw_mfcc_train_yesno.1.ark ark,t:1.txt

将每一帧的MFCC特征打印出来，可以看到0_0_0_0_1_1_1_1这个语音一共分成了633帧(跟上一步中的帧数是一致的)，每一行是13列，就是13维的MFCC特征向量。

    1 0_0_0_0_1_1_1_1  [
    2   48.97441 -14.08838 -0.1344408 4.717922 21.6918 -0.2593727 -8.379629 8.9065 4.354929 17.00239 0.8865671 9.878274 2.10597
    3   53.68612 -10.14593 -1.394655 -2.119213 13.08846 6.172102 8.67521 19.2422 0.4617043 5.210238 3.242958 2.333473 -0.5913754
    4   55.30577 -10.3102 2.783288 6.130808 18.00465 0.1580257 -5.361832 5.867401 6.992275 3.769726 0.3255215 4.97998 6.144581
    5   56.4837 -16.38814 -2.418082 8.250138 5.304474 5.584198 -14.83413 2.809654 10.13197 19.37797 -4.723887 2.218409 4.529137
    6   59.04967 -6.238425 -3.889257 -4.782249 1.989491 8.229766 -3.262495 -3.118021 -2.30123 12.84513 -23.23007 4.634783 -2.480999
    7   61.0052 -5.754354 -2.929795 -1.887645 9.401306 6.466054 3.297932 5.754842 6.992275 13.73597 -2.704123 -3.764996 -11.14876
    8   61.16816 -6.399782 -4.08115 -1.308723 0.9340172 1.201521 1.067387 3.180134 5.485221 14.03292 -2.367496 -0.4280972 4.259896
    9   61.98296 -7.206567 -1.714476 2.512154 2.200584 6.760006 -7.461169 -3.488502 2.219934 8.297047 -3.826214 9.39221 -4.559584


  631   56.4837 -7.851995 -1.202763 2.97529 10.63036 7.935814 6.117779 15.99773 7.745803 11.95428 -11.79164 6.47583 -5.88232
  632   57.07267 -11.29581 -3.121687 5.777586 6.943207 14.69671 -1.425576 2.809654 8.876094 16.11155 -8.090159 10.60737 -2.953404
  633   57.80887 -9.981664 2.053856 -1.540293 -3.287874 -0.6767693 -6.673918 -0.1541843 0.8384681 19.97186 -4.275051 -2.38421 8.298507
  634   56.18922 -9.981664 0.5949914 5.777586 12.26909 6.466054 -6.4115 1.698215 2.973462 8.091259 -14.81946 6.245699 7.490785 ]

 

于是一段语音分割成一帧，每一帧有１３维度的特征向量，确定每一帧的Transition-id，最后确定每一帧的phone，最后根据HMM确定一段语音的word。

 

参考

https://blog.csdn.net/chinatelecom08/article/details/81252140

https://blog.csdn.net/asrgreek/article/details/89070301#commentsedit

https://www.cnblogs.com/welen/p/7485151.html

https://blog.csdn.net/Xwei1226/article/details/80517046

https://blog.csdn.net/Xwei1226/article/details/80488335

http://www.voidcn.com/article/p-ocqffbfj-oo.html