小白都能看懂的马尔可夫链详解

在机器学习算法中，马尔可夫链(Markov chain)是个很重要的概念。马尔可夫链（Markov chain），又称离散时间马尔可夫链（discrete-time Markov chain），因俄国数学家安德烈·马尔可夫（俄语：Андрей Андреевич Марков）得名，为状态空间中经过从一个状态到另一个状态的转换的随机过程。该过程要求具备“无记忆”的性质：下一状态的概率分布只能由当前状态决定，在时间序列中它前面的事件均与之无关。这种特定类型的“无记忆性”称作马尔可夫性质。马尔科夫链作为实际过程的统计模型具有许多应用。
在马尔可夫链的每一步，系统根据概率分布，可以从一个状态变到另一个状态，也可以保持当前状态。状态的改变叫做转移，与不同的状态改变相关的概率叫做转移概率。随机漫步就是马尔可夫链的例子。随机漫步中每一步的状态是在图形中的点，每一步可以移动到任何一个相邻的点，在这里移动到每一个点的概率都是相同的（无论之前漫步路径是如何的）。
(来自参考文献1)

2.一个经典的马尔科夫链实例

用一句话来概括马尔科夫链的话，那就是某一时刻状态转移的概率只依赖于它的前一个状态。举个简单的例子，假如每天的天气是一个状态的话，那个今天是不是晴天只依赖于昨天的天气，而和前天的天气没有任何关系。这么说可能有些不严谨，但是这样做可以大大简化模型的复杂度，因此马尔科夫链在很多时间序列模型中得到广泛的应用，比如循环神经网络RNN，隐式马尔科夫模型HMM等。
假设状态序列为$\cdots xt-2,xt-1,xt,xt+1,xt+2,\cdots \cdots xt-2,xt-1,xt,xt+1,xt+2,\cdots \cdots x_{t-2},x_{t-1},x_t,x_{t+1},x_{t+2},\cdots$t0​=[0.1,0.2,0.7]，然后算之后的状态。

def markov():
    init_array = np.array([0.1, 0.2, 0.7])
    transfer_matrix = np.array([[0.9, 0.075, 0.025],
                               [0.15, 0.8, 0.05],
                               [0.25, 0.25, 0.5]])
    restmp = init_array
    for i in range(25):
        res = np.dot(restmp, transfer_matrix)
        print i, "\t", res
        restmp = res

markov()

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12

最终输出的结果：

0 	[ 0.295   0.3425  0.3625]
1 	[ 0.4075   0.38675  0.20575]
2 	[ 0.4762  0.3914  0.1324]
3 	[ 0.52039   0.381935  0.097675]
4 	[ 0.55006   0.368996  0.080944]
5 	[ 0.5706394  0.3566873  0.0726733]
6 	[ 0.58524688  0.34631612  0.068437  ]
7 	[ 0.59577886  0.33805566  0.06616548]
8 	[ 0.60345069  0.33166931  0.06487999]
9 	[ 0.60907602  0.32681425  0.06410973]
10 	[ 0.61321799  0.32315953  0.06362248]
11 	[ 0.61627574  0.3204246   0.06329967]
12 	[ 0.61853677  0.31838527  0.06307796]
13 	[ 0.62021037  0.31686797  0.06292166]
14 	[ 0.62144995  0.31574057  0.06280949]
15 	[ 0.62236841  0.31490357  0.06272802]
16 	[ 0.62304911  0.31428249  0.0626684 ]
17 	[ 0.62355367  0.31382178  0.06262455]
18 	[ 0.62392771  0.31348008  0.06259221]
19 	[ 0.624205   0.3132267  0.0625683]
20 	[ 0.62441058  0.31303881  0.06255061]
21 	[ 0.624563    0.31289949  0.06253751]
22 	[ 0.624676   0.3127962  0.0625278]
23 	[ 0.62475978  0.31271961  0.06252061]
24 	[ 0.6248219   0.31266282  0.06251528]

 
 

 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

从第18次开始，状态就开始收敛至$[0.624,0.312,0.0625][0.624,0.312,0.0625][0.624,0.312,0.0625]$P的幂次方有什么有意思的地方？废话不多说，直接上代码。

def matrixpower():
    transfer_matrix = np.array([[0.9, 0.075, 0.025],
                               [0.15, 0.8, 0.05],
                               [0.25, 0.25, 0.5]])
    restmp = transfer_matrix
    for i in range(25):
        res = np.dot(restmp, transfer_matrix)
        print i, "\t", res
        restmp = res

matrixpower()

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11

代码运行的结果：

0 	[[ 0.8275   0.13375  0.03875]
 [ 0.2675   0.66375  0.06875]
 [ 0.3875   0.34375  0.26875]]
1 	[[ 0.7745   0.17875  0.04675]
 [ 0.3575   0.56825  0.07425]
 [ 0.4675   0.37125  0.16125]]
2 	[[ 0.73555   0.212775  0.051675]
 [ 0.42555   0.499975  0.074475]
 [ 0.51675   0.372375  0.110875]]
3 	[[ 0.70683   0.238305  0.054865]
 [ 0.47661   0.450515  0.072875]
 [ 0.54865   0.364375  0.086975]]
4 	[[ 0.685609   0.2573725  0.0570185]
 [ 0.514745   0.4143765  0.0708785]
 [ 0.570185   0.3543925  0.0754225]]
5 	[[ 0.6699086  0.2715733  0.0585181]
 [ 0.5431466  0.3878267  0.0690267]
 [ 0.585181   0.3451335  0.0696855]]
6 	[[ 0.65828326  0.28213131  0.05958543]
 [ 0.56426262  0.36825403  0.06748335]
 [ 0.5958543   0.33741675  0.06672895]]
7 	[[ 0.64967099  0.28997265  0.06035636]
 [ 0.5799453   0.35379376  0.06626094]
 [ 0.60356362  0.33130471  0.06513167]]
8 	[[ 0.64328888  0.29579253  0.06091859]
 [ 0.59158507  0.34309614  0.06531879]
 [ 0.60918588  0.32659396  0.06422016]]
9 	[[ 0.63855852  0.30011034  0.06133114]
 [ 0.60022068  0.33517549  0.06460383]
 [ 0.61331143  0.32301915  0.06366943]]
10 	[[ 0.635052    0.30331295  0.06163505]
 [ 0.60662589  0.3293079   0.06406621]
 [ 0.61635051  0.32033103  0.06331846]]
11 	[[ 0.63245251  0.30568802  0.06185947]
 [ 0.61137604  0.32495981  0.06366415]
 [ 0.61859473  0.31832073  0.06308454]]
12 	[[ 0.63052533  0.30744922  0.06202545]
 [ 0.61489845  0.32173709  0.06336446]
 [ 0.6202545   0.31682232  0.06292318]]
13 	[[ 0.62909654  0.30875514  0.06214832]
 [ 0.61751028  0.31934817  0.06314155]
 [ 0.62148319  0.31570774  0.06280907]]
14 	[[ 0.62803724  0.30972343  0.06223933]
 [ 0.61944687  0.3175772   0.06297594]
 [ 0.6223933   0.3148797   0.062727  ]]
15 	[[ 0.62725186  0.31044137  0.06230677]
 [ 0.62088274  0.31626426  0.062853  ]
 [ 0.62306768  0.31426501  0.06266732]]
16 	[[ 0.62666957  0.31097368  0.06235675]
 [ 0.62194736  0.31529086  0.06276178]
 [ 0.62356749  0.31380891  0.0626236 ]]
17 	[[ 0.62623785  0.31136835  0.0623938 ]
 [ 0.6227367   0.31456919  0.06269412]
 [ 0.62393798  0.31347059  0.06259143]]
18 	[[ 0.62591777  0.31166097  0.06242126]
 [ 0.62332193  0.31403413  0.06264394]
 [ 0.62421263  0.31321968  0.0625677 ]]
19 	[[ 0.62568045  0.31187792  0.06244162]
 [ 0.62375584  0.31363743  0.06260672]
 [ 0.62441624  0.31303361  0.06255015]]
20 	[[ 0.6255045   0.31203878  0.06245672]
 [ 0.62407756  0.31334332  0.06257913]
 [ 0.62456719  0.31289565  0.06253716]]
21 	[[ 0.62537405  0.31215804  0.06246791]
 [ 0.62431608  0.31312525  0.06255867]
 [ 0.62467911  0.31279335  0.06252754]]
22 	[[ 0.62527733  0.31224646  0.06247621]
 [ 0.62449293  0.31296357  0.0625435 ]
 [ 0.62476209  0.3127175   0.06252042]]
23 	[[ 0.62520562  0.31231202  0.06248236]
 [ 0.62462404  0.3128437   0.06253225]
 [ 0.62482361  0.31266126  0.06251514]]
24 	[[ 0.62515245  0.31236063  0.06248692]
 [ 0.62472126  0.31275483  0.06252391]
 [ 0.62486922  0.31261956  0.06251122]]

 
 

 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

从第20次开始，结果开始收敛，并且每一行都为$[0.625,0.312,0.0625][0.625,0.312,0.0625][0.625,0.312,0.0625]$π是方程