利用Mathpix和Typora编辑markov链的证明过程

Markov链的证明过程：

什么是markov链：

马尔可夫链（Markov Chain, MC）是概率论和数理统计中具有马尔可夫性质（Markov property）且存在于离散的指数集（index set）和状态空间（state space）内的随机过程（stochastic process）[1][2]。适用于连续指数集的马尔可夫链被称为马尔可夫过程（Markov process），但有时也被视为马尔可夫链的子集，即连续时间马尔可夫链（Continuous-Time MC, CTMC），与离散时间马尔可夫链（Discrete-Time MC, DTMC）相对应，因此马尔可夫链是一个较为宽泛的概念[2]。

设两个信道串接构成随机变量序列（XYZ）,则：
$$I(XY;Z)⩾I(Y;Z)$$

当且仅当：
$$P(Z/XY)=P(Z/Y)$$
即（XYZ）是Markov链时，才有：
$$I(XY;Z)=I(Y;Z)$$

证明如下：

根据平均交互信息量的定义：
$$\begin{gathered} I(a_i;b_j)=\sum _{i=1}^{r}p(a_i/b_j)I(a_i;b_j) \\ =\sum _{i=1}^{r}p(a_i/b_j)\log \frac{p(a_i/b_j)}{p(a_i)}........(j=1,2,...,s) \end{gathered}$$

所以：
$$I(XY;Z)=\sum _{i=1}^r\sum _{j=1}^n\sum _{k=1}^{L}p\left(a_i{b}_j{c}_k\right)\log \frac{p\left(c_k/a_i{b}_j\right)}{p\left(c_k\right)}$$
而：
$$I(Y;Z)=\sum _{j=1}^s\sum _{i=1}^{L}p\left(b_j{c}_k\right)\log \frac{p\left(c_k/b_j\right)}{p\left(c_k\right)}=\sum _{i=1}^r\sum _{j=1}^s\sum _{k=1}^{L}p\left(a_i{b}_j{c}_k\right)\log \frac{p\left(c_k/b_j\right)}{p\left(c_k\right)}$$
考虑到：
$$\sum _{j=1}^s\sum _{i=1}^{L}p\left(b_j{c}_k\right)=\sum _{i=1}^r\sum _{j=1}^s\sum _{k=1}^{L}p\left(a_i{b}_j{c}_k\right)=1,$$
以及“底”大于1的对数是倒U型凸函数，由不等式：
$$f\left\{\sum _{i=1}^r{p}_i{x}_i\right\}⩾\sum _{i=1}^r{p}_{i}f\left(x_i\right)$$
可得下式：

$$\begin{array}{rl}I(Y;Z)-I(XY;Z)& =\sum _{i=1}^r\sum _{j=1}^s\sum _{k=1}^{L}p\left(a_i,b_j{c}_k\right)\log \frac{p\left(c_k/b_j\right)}{p\left(c_k/a_i,b_j\right)}.........1\\ & ⩽\log \left\{\sum _i\sum _j\sum _{k}p\left(a_i,b_j,c_k\right)\frac{p\left(c_k/b_j\right)}{p\left(c_k/a_i{b}_j\right)}\right\}.......2\\ & =\log \left\{\sum _i\sum _j\sum _{k}p\left(a_i{b}_j\right)p\left(c_k/b_j\right)\right\}......3\\ ////////////////////////////=\log 1=0& \end{array}$$

下面是上式中的一些难懂解释：1式解释、：
$$\begin{gathered} \sum _{i=1}^r\sum _{j=1}^s\sum _{k=1}^{L}p\left(a_i,b_j{c}_k\right)\log \frac{p\left(c_k/b_j\right)}{p\left(c_k/a_i,b_j\right)} \\ =\sum _{i=1}^r\sum _{j=1}^s\sum _{k=1}^{L}p\left(a_i{b}_j{c}_k\right)\log \frac{p\left(c_k/b_j\right)}{p\left(c_k\right)}-\sum _{i=1}^r\sum _{j=1}^n\sum _{k=1}^{L}p\left(a_i{b}_j{c}_k\right)\log \frac{p\left(c_k/a_i{b}_j\right)}{p\left(c_k\right)} \end{gathered}$$
上式提出公共部分，后面的进行log进行减法

2式到3式得证明过程：
$$\begin{gathered} \because p\left(a_i,b_j,c_k\right)=p\left(a_i{b}_j\right)p\left(c_k/a_i{b}_j\right) \\ \therefore p\left(a_i,b_j,c_k\right)\frac{p\left(c_k/b_j\right)}{p\left(c_k/a_i{b}_j\right)}=p\left(a_i{b}_j\right)p\left(c_k/b_j\right) \end{gathered}$$

3式可以写成如下：
$$\begin{gathered} \sum _i\sum _j\sum _{k}p\left(a_i{b}_j\right)p\left(c_k/b_j\right)=\sum _i\sum _{j}p\left(a_i{b}_j\right)\sum _{k}p\left(c_k/b_j\right) \\ 又因为.......\sum _{k}p\left(c_k/b_j\right)....当且仅当k=1,2,....,L都有\sum _{k}p\left(c_k/b_j\right)=\sum _{k}p\left(c_k\right) \\ 又因为....\sum _i\sum _{j}p\left(a_i{b}_j\right)=1,,,,\sum _{k}p\left(c_k\right)=1 \\ 所以....\sum _i\sum _j\sum _{k}p\left(a_i{b}_j\right)p\left(c_k/b_j\right)=\sum _i\sum _{j}p\left(a_i{b}_j\right)\sum _{k}p\left(c_k/b_j\right)=1 \end{gathered}$$

综上所述，上式不等式成立

当且仅当：

$$\frac{p\left(c_k,b_j\right)}{p\left(c_k/a_i{b}_j\right)}=1\left(i=1,2,\cdots ,r;j=1,2,\cdots ,s;k=1,2,\cdots L\right)$$

则上式不等式等于：
$$\begin{array}{rl}I(Y;Z)-I(XY;Z)& =\sum _{i=1}^r\sum _{j=1}^s\sum _{k=1}^{L}p\left(a_i{b}_j{c}_k\right)\log \frac{p\left(c_k/b_j\right)}{p\left(c_k/a_i{b}_j\right)}\\ & =\sum _{i=1}^r\sum _{j=1}^s\sum _{k=1}^{L}p\left(a_i{b}_j{c}_k\right)\log 1=0\\ I(XY;Z)& =I(Y;Z)\end{array}$$