声学模型计算到phone上后验概率和强制对其结果匹配

通过声学模型计算出来到pdf-id上的后验概率，可以通过转换计算得到phone上的概率。

kaldi提供了多个方法，甚至还提供了直接到有y(标注结果)上的对齐上的phone上的概率输出

---

1 通过post-to-phone-post来计算

先需要通过nnet3-compute计算chain前向传播概率矩阵(声学模型输出)、计算chain声学模型帧对齐上的＜pdf-id,后验概率＞(不算语言模型)得到声学模型chain的输出概率即可。

# exp/chain/nnet3_ali/xent.nnet3.1.ark 由xent-output的声学模型输出产生
logprob-to-post ark:exp/chain/nnet3_ali/xent.nnet3.1.ark ark:-|\
post-to-phone-post --transition-id-counts=exp/chain/tdnn1a_sp_post_eval/tacc \
exp/chain/tdnn_1a_sp/final.mdl ark:- ark:exp/chain/nnet3_ali/xent.nnet3.1.phone.post.ark

exp/chain/tdnn1a_sp_post_eval/tacc这个转换矩阵是通过在所有训练数据集上汇总得到，这个是固定的，永远不变

可以用steps/nnet3/chain/get_phone_post.sh得到的中间产物，可以用get_phone_post计算phone上概率并产生tacc和transform.mat文章中得到

可以很明显的和计算chain声学模型帧对齐上的＜pdf-id,后验概率＞(不算语言模型)对比出，这里面输出的肯定是phone-id了

HAO0007501-000000 [ 1 0.9999405 ] [ 1 0.9999996 ] [ 1 0.9999999 ] [ 1 0.9999998 ] [ 1 0.9999992 ] [ 1 0.9999951 ] [ 1 0.9999954 ] [ 1 0.9999985 ] [ 1 0.9999326 ] [ 1 0.9991617 ] [ 1 0.9999516 ] [ 1 0.9999683 ] [ 1 0.9999915 ] [ 1 0.9998736 ] [ 1 0.9999506 ] [ 1 0.9999948 ] [ 1 0.9999986 ] [ 1 0.9999992 ] [ 1 0.9999981 ] [ 1 0.9999894 ] [ 1 0.9997845 ] [ 149 0.01 144 0.01191371 1 0.9591256 122 0.01 148 0.02 ] [ 144 0.01 1 0.9541238 148 0.01 ] [ 1 0.9965102 ] [ 1 0.9996038 ] [ 1 0.999707 ] [ 1 0.9998472 ] [ 1 0.9999455 ] [ 1 0.9999617 ] [ 1 0.9999813 ] [ 1 0.9999716 ] [ 1 0.9999858 ] [ 1 0.9998567 ] [ 1 0.9996137 ] [ 1 0.9998506 ] [ 1 0.3605962 122 0.01 145 0.01 90 0.5729466 148 0.04218972 126 0.01 ] [ 1 0.01195761 145 0.01 90 0.9361535 148 0.04467029 ] [ 148 0.06616876 126 0.01 145 0.01 90 0.920313 ] [ 97 0.01 108 0.01 90 0.8801629 148 0.1025384 ] [ 83 0.002435766 106 0.001080241 110 0.0003070737 71 0.01 98 0.02 148 0.1402076 62 0.03122567 108 0.5165952 131 0.0002496055 91 0.06477136 90 0.07312825 97 0.02935434 191 0.01 103 0.01 135 0.0003001691 133 0.009611049 132 8.878241e-05 82 0.007564234 129 6.168412e-05 128 0.009477836 127 0.0002108736 60 0.0100035 107 0.06034594 96 0.01110402 194 0.01 100 0.000645661 92 0.01 ] [ 103 0.03158043 37 0.03068992 106 0.0002066734 31 0.01 80 0.01 107 0.08145306 58 0.01 92 0.01 98 0.01 145 0.0412131 108 0.4614727 100 0.001077326 194 0.04248523 96 0.02618811 124 0.01 7 0.02190897 148 0.1109542 97 0.1048043 191 0.01 ] [ 214 0.01 81 0.007536624 82 0.002262723 84 0.0002006526 37 0.1764187 133 0.01 107 0.04104028 166 0.01 145 0.1513312 59 0.01 124 0.03829373 108 0.05997065 100 0.000645661 194 0.1524241 96 0.0219443 7 0.1305882 148 0.1290507 97 0.03785951 103 0.03664133 ] [ 124 0.01 151 0.01 37 0.100707 185 0.00935025 44 0.01148633 186 0.0006497495 45 7.376472e-05 107 0.02294665 145 0.4645545 191 0.01 214 0.03 32 0.0001465773 80 0.01 33 0.03983349 34 0.001429128 35 0.01276269 108 0.04664025 194 0.04187197 182 0.01091357 148 0.1259216 209 0.0002174511 210 0.002004299 211 0.0001762103 212 0.007602039 ] [ 37 0.0660127 106 0.0002066734 45 6.380978e-05 53 0.02320934 47 0.01287099 145 0.1126374 96 0.01 107 0.05803323 201 0.01 191 0.01259284 126 0.01 32 6.505944e-05 33 0.1583167 34 0.0006343295 35 0.01566482 43 0.009853801 184 0.008377855 44 0.00993619 185 0.007797525 46 0.0001461988 187 0.003824619 108 0.02601308 151 0.01 205 0.000591181 208 0.009408819 30 0.01 124 0.06549517 180 0.01 148 0.2793551 56 0.05295399 ] [ 31 0.01 126 0.02602065 11 0.0001588559 196 0.01 145 0.03 44 0.01878857 45 0.0001139836 47 0.01109744 33 0.01 124 0.58605 148 0.22147 9 0.01743004 56 0.02538405 ] [ 15 0.008519953 16 0.001480046 44 0.00993619 45 6.380978e-05 9 0.3225382 11 0.002848446 167 0.01 14 0.01 78 0.01 31 0.02 124 0.3067466 12 0.04997119 148 0.1351716 60 0.01 111 0.01 36 0.03700791 83 0.04370646 ] [ 71 0.01281069 9 0.4311641 11 0.00286742 14 0.01 44 0.05591929 45 0.0003432539 47 0.002606788 38 5.350112e-05 39 0.004071869 40 0.0005046457 41 0.006578228 42 0.0006940686 12 0.2434108 148 0.08150821 36 0.01005394 83 0.05134147 49 0.04414922 ] [ 12 0.2079409 144 0.01 9 0.4874247 49 0.07186691 14 0.02402355 30 0.01 44 0.0379246 45 0.0002149501 47 0.004701567 71 0.01770077 148 0.0685931 83 0.01 1 0.01 ] [ 12 0.07891624 73 0.01 15 0.008302138 16 0.001697861 9 0.1125657 49 0.01528228 44 0.008852383 45 5.017379e-05 47 0.001097443 149 0.02 40 0.01 29 0.02 144 0.3354859 148 0.316108 11 0.01 60 0.01 30 0.04853029 ] [ 29 0.01 147 0.01 122 0.01 130 0.01 144 0.6713294 148 0.2720779 30 0.07999999 ] [ 217 0.01 215 0.01 30 0.03235225 144 0.5223752 148 0.4322894 ] [ 78 0.0005749124 19 0.009978957 22 2.104238e-05 15 0.01 20 0.02008883 76 0.03888487 30 0.02 77 0.03504472 80 0.001232223 81 0.0164556 83 0.003662308 84 7.388808e-05 2 0.01 82 0.01036641 144 0.03 148 0.2694041 75 0.553112 ] [ 76 0.02784788 77 0.02058059 19 0.009978957 22 2.104238e-05 15 0.01 78 0.01010548 148 0.1560424 75 0.7425741 ] [ 76 0.01039372 77 0.007941972 20 0.01210282 31 0.01 148 0.05062214 75 0.8966265 ] [ 17 8.94825e-05 19 0.01803712 76 0.01703374 217 0.01 15 0.01991052 77 0.03091959 79 4.271264e-06 122 0.04583983 75 0.6596824 20 0.07321092 148 0.1309246 1 0.02 ] [ 17 0.0003713563 19 0.0612474 15 0.05112909 58 0.03 77 0.009997615 126 0.01 79 2.385092e-06 21 0.01523768 20 0.1147943 217 0.09146246 216 0.04969933 75 0.01591494 122 0.2622186 148 0.1570705 31 0.1417072 ] [ 216 0.02715104 122 0.5519449 217 0.2020029 148 0.04131977 31 0.1141258 ] [ 217 0.1241771 31 0.02 86 0.1009233 89 0.01 206 0.08588398 32 5.501658e-06 35 3.080929e-05 33 0.02108144 36 0.01 96 0.008184228 99 0.001781659 100 3.411314e-05 43 0.01359555 216 0.04230849 122 0.1564546 72 0.01 148 0.1783555 85 0.01121683 179 0.0002162607 182 0.009783739 73 0.00998403 120 0.01 205 8.810421e-06 111 0.01 70 0.01001597 62 0.053353 8 3.150737e-05 10 4.568876e-05 58 0.01 11 0.01580363 12 3.607008e-06 ] [ 36 0.009606792 183 2.185575e-05 89 0.02 122 0.01 32 4.948535e-06 113 0.01 80 0.01 33 0.01993513 206 0.1246988 86 0.07981877 85 0.01608069 179 0.0002748945 182 0.02100736 93 0.01 123 0.01 63 0.01 35 0.01045313 148 0.3964792 60 0.03168161 205 1.834859e-05 111 0.02893803 70 0.01 62 0.02171947 8 0.0001258324 10 0.0001824693 11 0.08979625 12 1.440547e-05 ] [ 61 0.0004155406 63 0.009496978 183 2.185575e-05 33 0.02859035 86 0.1028348 37 0.01 206 0.03195778 65 0.009773748 116 0.01 69 0.0002262513 85 0.03164958 179 0.0003859788 182 0.02546424 64 0.0002605043 205 4.928841e-06 111 0.03069419 32 0.01 149 0.01 8 0.01037794 10 3.903151e-05 12 1.332783e-05 36 0.02 73 0.009975767 70 0.1161868 35 0.01815157 31 0.02 62 0.01579003 91 0.01 148 0.2277254 60 0.05552791 11 0.1041873 58 0.02 ] [ 88 0.01 120 0.01 86 0.01 216 0.02410122 194 0.01 31 0.01 35 0.02 148 0.03856676 123 0.01 217 0.01 11 0.01113104 58 0.8133896 ] [ 145 0.01 18 0.01 8 0.008039106 31 0.01 43 0.05585452 46 3.634469e-06 32 0.01 70 0.01 148 0.1228723 122 0.08977131 15 0.02148998 62 0.01694421 11 0.001960894 58 0.6017663 ] [ 16 0.01 35 0.01614692 18 0.01 11 0.001960894 8 0.01786838 43 0.02697896 46 1.755528e-06 10 0.01565025 25 0.01 33 0.01 148 0.03474962 15 0.8102539 62 0.01 ] [ 18 0.01236373 22 0.002040552 34 0.01 126 0.01 14 0.01 58 0.01 10 0.02190688 13 0.01362452 35 0.02 15 0.750669 62 0.01112317 43 0.02571377 46 6.888509e-07 148 0.04387978 ] [ 144 0.02 149 0.01 124 0.01 126 0.1561277 13 0.01 122 0.6803347 43 0.009999732 46 2.678846e-07 148 0.07781513 15 0.01514589 62 0.01 ] [ 149 0.02 94 0.01 92 0.01080758 96 0.04510323 100 0.0001046342 122 0.04431532 93 0.1378145 91 0.03382169 31 0.01 97 0.01578689 99 0.01595046 90 0.01 148 0.05755223 126 0.2023494 62 0.3903895 ] [ 60 2.325394e-05 95 2.708688e-05 94 0.02 126 0.01 92 0.03381744 100 9.149681e-05 97 0.05361801 93 0.01594165 99 0.06082574 31 0.01 62 0.1288107 96 0.386044 91 0.1149365 148 0.1160418 70 0.03096328 ] [ 60 4.265076e-05 95 1.609489e-05 97 0.02 92 0.01452686 126 0.01 93 0.01 99 0.01962319 217 0.01 62 0.1316573 96 0.04729896 70 0.01 216 0.01 91 0.03708282 148 0.1375558 31 0.610695 ] [ 91 0.009686784 92 0.0003074592 95 5.756703e-06 4 0.01 148 0.0987638 31 0.8626936 ] [ 3 0.0346339 31 0.03 4 0.8908502 148 0.0107911 ] [ 4 0.9846131 ] [ 4 0.8913195 148 0.01 31 0.07074773 ] [ 148 0.01 31 0.9932162 ] [ 90 0.01 70 0.0003937249 73 0.2461585 31 0.7435221 ] [ 70 0.001593696 73 0.9493701 148 0.03100051 ] [ 148 0.02 70 0.002358824 73 0.9710169 59 0.01 ] [ 70 0.001329224 73 0.5471791 59 0.4245575 ] [ 148 0.03 59 0.9552596 ] [ 59 0.9654052 148 0.02 ] [ 148 0.02 158 0.01 59 0.9775106 ] [ 165 0.001551429 167 0.0003200586 168 0.008128512 158 0.811124 159 0.0001321396 59 0.146971 148 0.02494159 ] [ 168 0.01 155 0.009906314 2 0.01 158 0.9401076 159 0.006123928 124 0.01 148 0.04377646 ] [ 37 0.01 158 0.9388704 159 0.006915936 148 0.02221034 ] [ 158 0.8815921 159 0.006494011 7 0.05589128 148 0.043571 ] [ 155 0.009906314 6 1.253866e-05 158 0.6991624 159 0.005243877 4 0.0241184 7 0.2141241 148 0.02478459 ] [ 6 0.01003859 158 0.590925 159 0.00435289 116 0.01 4 0.1249963 7 0.2420065 148 0.03261494 ] [ 168 0.009754212 169 0.0002457879 155 0.004956838 157 0.0005086602 5 0.01 191 0.01 158 0.4234056 159 0.003085501 4 0.4509007 6 0.0104142 7 0.08723826 148 0.04894784 ] [ 155 0.004956838 157 0.0005086602 16 0.01 3 8.804455e-05 5 0.01991196 2 0.009792217 158 0.259436 159 0.001877663 4 0.5651596 6 0.05836448 7 0.01 148 0.04574308 83 0.01 ] [ 155 0.004956838 68 0.01 5 0.02473928 157 0.0005086602 16 0.02 2 0.08826517 158 0.06596951 159 0.0004525445 163 0.01 116 0.01 11 0.02385216 3 0.000683858 56 0.01 6 0.1384727 7 0.01 148 0.06034796 83 0.02091217 4 0.4986894 ] [ 5 0.009911955 2 0.01712082 3 0.0002694247 158 0.3756474 159 0.002767106 116 0.01 11 0.01 6 0.04340445 148 0.06732794 36 0.01 4 0.4472219 ] [ 5 0.0175209 2 0.03694228 68 0.02055859 158 0.1586442 159 0.001168609 11 0.01877053 3 0.0005844404 6 0.101567 69 0.02436536 148 0.02565941 83 0.02244435 4 0.4897307 60 0.01 ] [ 5 0.01 155 0.004956838 157 0.0005086602 68 0.01 2 0.02695943 158 0.3698652 159 0.00269111 3 0.0005844404 6 0.131132 69 0.01 54 0.01 148 0.05368072 83 0.01490745 4 0.3415616 ] [ 16 0.01 5 0.01137469 2 0.02205233 158 0.3447703 159 0.002539658 3 0.009446777 56 0.01 171 0.01 6 0.1856879 69 0.01 148 0.037667 83 0.01770419 4 0.2882899 156 0.01 1 0.02 ] [ 5 0.01 66 0.0001477973 67 0.009852203 16 0.01 25 0.01 158 0.2335108 159 0.001720095 2 0.05164245 3 0.002093956 4 0.2970836 56 0.01 27 0.01 6 0.2255345 69 0.01449655 148 0.04063069 83 0.0330938 60 0.01 1 0.02 ] [ 82 0.009754551 84 0.0002454498 23 1.058169e-06 26 0.01305903 25 0.04638296 5 0.01754236 172 0.01 2 0.02378571 158 0.09787345 3 0.03520851 56 0.01026751 159 0.0007209584 65 2.942685e-05 20 0.01 67 0.01879826 68 0.02717525 27 0.0409409 6 0.0538711 69 0.04036652 40 0.015436 148 0.04280965 83 0.03714984 36 0.01 4 0.389772 60 0.01 1 0.02 ] [ 23 2.303027e-06 26 0.02842202 155 0.009906314 25 0.07323251 158 0.1192371 2 0.01974019 3 0.02124057 56 0.01 159 0.0009720133 65 1.531331e-05 67 0.009782347 68 0.0002023399 27 0.02805366 6 0.05922324 69 0.01 148 0.02 83 0.03873808 36 0.01 4 0.226403 60 0.01 1 0.2889895 ] [ 4 0.01998746 158 0.009926876 159 7.312366e-05 6 0.01001254 1 0.9684973 ] [ 1 0.9978083 ] [ 1 1.008561 ] [ 1 0.9996926 ] [ 1 0.9999599 ] [ 1 0.9999925 ] [ 1 0.9999983 ] [ 1 0.9999989 ] [ 1 0.999999 ] [ 1 0.9999995 ] [ 1 0.9999993 ] [ 1 0.9999985 ] [ 1 0.9999961 ] [ 1 0.9999987 ] [ 1 0.9999989 ] [ 1 0.999999 ] [ 1 0.9999998 ] [ 1 0.9999996 ] [ 1 0.9999996 ] [ 1 0.9999986 ] [ 1 0.9999985 ] [ 1 0.9999989 ] [ 1 0.9999988 ] [ 1 0.999999 ] [ 1 0.9999983 ] [ 1 0.9999985 ] [ 1 0.9999994 ] [ 1 0.9999726 ] [ 1 0.999929 ]

---

2 通过get-post-on-ali将phone上概率对齐

在前文chain模型音素对齐我们已经计算出了在这个测试集上的对齐结果ali.1.gz，这里直接拿来用。
【这里不能乱用GMM模型的对齐结果，是不一致的，比如frame-shift等等之类，下采样帧数也不同】，所有的必须保持上下数据源一致

#注释命令用来观察，最后是用的ark,t:-
#ali-to-phones --frame-shift=0.03 --per-frame=true exp/chain/tdnn_1a_sp/final.mdl \
#ark:"gunzip -c exp/chain/nnet3_ali/ali.1.gz|" ark,t:-|head -n 1
ali-to-phones --frame-shift=0.03 --per-frame=true exp/chain/tdnn_1a_sp/final.mdl \
ark:"gunzip -c exp/chain/nnet3_ali/ali.1.gz|" ark:exp/chain/nnet3_ali/ali.1.force_phones.tra

ali-to-phones --frame-shift=0.03 --per-frame=true exp/chain/tdnn_1a_sp/final.mdl
ark:“gunzip -c exp/tri5a_test_ali/ali.1.gz|” ark:

# 注释命令结果，对齐到音素
HAO0007501-000000 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 90 90 90 90 108 108 145 145 44 124 124 9 9 144 144 144 75 75 75 75 122 122 86 86 86 58 58 15 15 122 93 93 93 31 4 4 4 31 31 73 73 59 59 59 59 158 158 158 158 158 158 158 158 158 158 158 158 158 158 158 158 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
...

接下来用这个对齐结果去匹配phone上的概率：

# exp/chain/nnet3_ali/xent.nnet3.1.phone.post.ark 由 logprob-to-post|post-to-phone-post 产生
# exp/chain/nnet3_ali/ali.1.force_phones.tra 由 ali-to-phones 产生
get-post-on-ali ark:exp/chain/nnet3_ali/xent.nnet3.1.phone.post.ark \
ark:exp/chain/nnet3_ali/ali.1.force_phones.tra \
ark,t:exp/chain/nnet3_ali/ali.1.force_phones.conf

在音频id=HAO0007501-000000上，维度也是125的标量，这样就将声学模型的输出和对齐结果进行了强制匹配

HAO0007501-000000  [ 0.9999405 0.9999996 0.9999999 0.9999998 0.9999992 0.9999951 0.9999954 0.9999985 0.9999326 0.9991617 0.9999516 0.9999683 0.9999915 0.9998736 0.9999506 0.9999948 0.9999986 0.9999992 0.9999981 0.9999894 0.9997845 0.9591256 0.9541238 0.9965102 0.9996038 0.999707 0.9998472 0.9999455 0.9999617 0.9999813 0.9999716 0.9999858 0.9998567 0.9996137 0.9998506 0.5729466 0.9361535 0.920313 0.8801629 0.5165952 0.4614727 0.1513312 0.4645545 0.00993619 0.58605 0.3067466 0.4311641 0.4874247 0.3354859 0.6713294 0.5223752 0.553112 0.7425741 0.8966265 0.6596824 0.2622186 0.5519449 0.1009233 0.07981877 0.1028348 0.8133896 0.6017663 0.8102539 0.750669 0.6803347 0.1378145 0.01594165 0.01 0.8626936 0.8908502 0.9846131 0.8913195 0.9932162 0.7435221 0.9493701 0.9710169 0.4245575 0.9552596 0.9654052 0.9775106 0.811124 0.9401076 0.9388704 0.8815921 0.6991624 0.590925 0.4234056 0.259436 0.06596951 0.3756474 0.1586442 0.3698652 0.3447703 0.2335108 0.09787345 0.1192371 0.9684973 0.9978083 1.008561 0.9996926 0.9999599 0.9999925 0.9999983 0.9999989 0.999999 0.9999995 0.9999993 0.9999985 0.9999961 0.9999987 0.9999989 0.999999 0.9999998 0.9999996 0.9999996 0.9999986 0.9999985 0.9999989 0.9999988 0.999999 0.9999983 0.9999985 0.9999994 0.9999726 0.999929 ]
HAO0007501-000001  [ 0.9999721 1 0.9999999 1 0.9999999 0.9999993 0.999999 0.9999986 0.9993761 0.9616253 0.3832772 0.871968 0.8392693 0.8158693 0.9773068 0.9722463 0.9972401 0.9973797 0.9964901 0.9955683 0.9921992 0.9953927 0.9944957 0.9943523 0.9959607 0.991245 0.9944284 0.9939294 0.9979447 0.9997677 0.9988025 0.9991241 0.9986967 0.9982828 0.9980387 0.9987798 0.997874 0.9875883 0.9910284 0.9673002 0.9624189 0.8107919 0.7323484 0.8177719 0.9638796 0.8089926 0.9636203 0.9537491 0.5649223 0.2648355 0.2735341 0.5375443 0.8768235 0.9905785 0.9805479 0.7431432 0.6869053 0.7304654 0.8209313 0.6783789 0.4264788 0.3315727 0.08334089 0.02926807 0.02552721 0.1481441 0.3696633 0.8402868 0.9604737 0.9901516 0.4284833 0.5597259 0.7457447 0.7546557 0.7703914 0.3240199 0.1428408 0.0183886 0 0 0 0 0 0 0 0 0.329183 0.1889566 0.4567453 0.4947765 0.3569115 0.7425327 0.7600626 0.6285472 0.5171931 0.3878079 0.8085836 0.9758302 0.9847373 0.9977538 0.994993 0.9974444 0.9873499 0.9773967 0.9713246 0.9796193 0.9682311 0.9511322 0.9459198 0.8949922 0.8176433 0.8653414 0.8996071 0.9528953 0.9793271 0.9683178 0.9801671 0.9872707 1.006233 0.9979505 0.9980234 0.9979487 0.9987153 0.9995419 0.9976562 0.9969479 0.9978821 0.9995698 0.9995114 0.9959491 0.9995416 0.9998056 0.9999326 0.9999903 0.9999367 0.9983549 0.9992328 0.9994054 ]
...

References

How to get the acoustic probability from chain model？
ow can we calculate the posterior probabilities of phone according to force alignment
kaldi chain模型的序列鉴别性训练代码分析
kaldi ASR: DNN训练
Add option to convert pdf-posteriors to phones