【深度学习】马尔科夫链

马尔科夫链

一、常见的马尔可夫过程：
(1) 独立随机过程为马尔可夫过程。
(2) 独立增量过程为马尔可夫过程：没{X(t)，t∈[0，+∞)}为一独立增量过程，且有P(X(0)=x0)=1，x0为常数，则X(t)为马尔可夫过程。
(3) 泊松过程为马尔可夫过程。
(4) 维纳过程为马尔可夫过程。
(5) 质点随机游动过程为马尔可夫过程。

二、模型的创立条件

import numpy as np
def markov():
    init_array = np.array([0.1,0.2,0.7])
    transfer_matrix = np.array([
        [0.9, 0.075, 0.025],
        [0.15,0.8,0.05],
        [0.25,0.25,0.5],
    ])
    restmp = init_array 
    for i in range(25):
        res = np.dot(restmp, transfer_matrix)
        print(i,"\t",res)
        restmp = res

markov()

0        [0.295  0.3425 0.3625]
1        [0.4075  0.38675 0.20575]
2        [0.4762 0.3914 0.1324]
3        [0.52039  0.381935 0.097675]
4        [0.55006  0.368996 0.080944]
5        [0.5706394 0.3566873 0.0726733]
6        [0.58524688 0.34631612 0.068437  ]
7        [0.59577886 0.33805566 0.06616548]
8        [0.60345069 0.33166931 0.06487999]
9        [0.60907602 0.32681425 0.06410973]
15       [0.62236841 0.31490357 0.06272802]
16       [0.62304911 0.31428249 0.0626684 ]
17       [0.62355367 0.31382178 0.06262455]
18       [0.62392771 0.31348008 0.06259221]
19       [0.624205  0.3132267 0.0625683]
20       [0.62441058 0.31303881 0.06255061]
21       [0.624563   0.31289949 0.06253751]
22       [0.624676  0.3127962 0.0625278]
23       [0.62475978 0.31271961 0.06252061]
24       [0.6248219  0.31266282 0.06251528]

会收敛于：
[0.6248219 0.31266282 0.06251528]

如果我们换一个初始状态t0​，比如[0.2,0.3.0.5]，继续运行上面的代码，只是将init_array变一下，最后结果为：

    init_array = np.array([0.2,0.3.0.5])
0        [0.35 0.38 0.27]
1        [0.4395  0.39775 0.16275]
2        [0.4959  0.39185 0.11225]
3        [0.53315  0.378735 0.088115]
4        [0.558674 0.365003 0.076323]
5        [0.5766378 0.3529837 0.0703785]
6        [0.5895162  0.34322942 0.06725438]
7        [0.59886259 0.33561085 0.06552657]
8        [0.6056996  0.32978501 0.06451539]
9        [0.61072624 0.32538433 0.06388944]
10       [0.61443362 0.32208429 0.06348209]
11       [0.61717343 0.31962047 0.0632061 ]
12       [0.61920068 0.31778591 0.06301341]
13       [0.62070185 0.31642213 0.06287602]
14       [0.62181399 0.31540935 0.06277666]
15       [0.62263816 0.31465769 0.06270415]
16       [0.62324903 0.31410005 0.06265091]
17       [0.62370187 0.31368645 0.06261168]
18       [0.62403757 0.31337972 0.06258271]
19       [0.62428645 0.31315227 0.06256128]
20       [0.62447096 0.31298362 0.06254542]
21       [0.62460776 0.31285857 0.06253366]
22       [0.62470919 0.31276586 0.06252495]
23       [0.62478439 0.31269711 0.0625185 ]
24       [0.62484014 0.31264614 0.06251372]

24 [0.6248219 0.31266282 0.06251528]
也就是状态转移的概率确定的前提下，和初始概率无关，