有向概率图模型(bayesian network)的条件独立性

顺序结构

下证$A\perp B|C$
$p(A,B,C)=p(A)p(C|A)p(B|C)$
$p(A,B|C)=\frac{p(A,B,C)}{p(C)}=\frac{p(A,B,C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{p(A)p(C|A)p(B|C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(C|A^{\prime })p(B^{\prime }|C)}=\frac{p(A)p(C|A)p(B|C)}{\left(\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime })\right)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}=\frac{p(A)p(C|A)p(B|C)}{\left(\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime })\right)}$
$p(A|C)=\frac{p(A,C)}{p(C)}=\frac{\sum _{B^{\prime }}p(A,B^{\prime },C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{\sum _{B^{\prime }}p(A)p(C|A)p(B^{\prime }|C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(C|A^{\prime })p(B^{\prime }|C)}=\frac{p(A)p(C|A)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}{\left(\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime })\right)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}=\frac{p(A)p(C|A)}{\left(\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime })\right)}$
$p(B|C)=\frac{p(B,C)}{p(C)}=\frac{\sum _{A^{\prime }}p(A^{\prime },B,C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime })p(B|C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(C|A^{\prime })p(B^{\prime }|C)}=\frac{\left(\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime })\right)p(B|C)}{\left(\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime })\right)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}=p(B|C)$
$p(A,B|C)=p(A|C)p(B|C)$恰好成立

下证$B\perp A|C$
$p(B,A,C)=p(B)p(C|B)p(A|C)$
$p(B,A|C)=\frac{p(B,A,C)}{p(C)}=\frac{p(B,A,C)}{\sum _{B^{\prime }}\sum _{A^{\prime }}p(B^{\prime },A^{\prime },C)}=\frac{p(B)p(C|B)p(A|C)}{\sum _{B^{\prime }}\sum _{A^{\prime }}p(B^{\prime })p(C|B^{\prime })p(A^{\prime }|C)}=\frac{p(B)p(C|B)p(A|C)}{\left(\sum _{B^{\prime }}p(B^{\prime })p(C|B^{\prime })\right)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)}=\frac{p(B)p(C|B)p(A|C)}{\left(\sum _{B^{\prime }}p(B^{\prime })p(C|B^{\prime })\right)}$
$p(B|C)=\frac{p(B,C)}{p(C)}=\frac{\sum _{A^{\prime }}p(B,A^{\prime },C)}{\sum _{B^{\prime }}\sum _{A^{\prime }}p(B^{\prime },A^{\prime },C)}=\frac{\sum _{A^{\prime }}p(B)p(C|B)p(A^{\prime }|C)}{\sum _{B^{\prime }}\sum _{A^{\prime }}p(B^{\prime })p(C|B^{\prime })p(A^{\prime }|C)}=\frac{p(B)p(C|B)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)}{\left(\sum _{B^{\prime }}p(B^{\prime })p(C|B^{\prime })\right)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)}=\frac{p(B)p(C|B)}{\left(\sum _{B^{\prime }}p(B^{\prime })p(C|B^{\prime })\right)}$
$p(A|C)=\frac{p(A,C)}{p(C)}=\frac{\sum _{B^{\prime }}p(B^{\prime },A,C)}{\sum _{B^{\prime }}\sum _{A^{\prime }}p(B^{\prime },A^{\prime },C)}=\frac{\sum _{B^{\prime }}p(B^{\prime })p(C|B^{\prime })p(A|C)}{\sum _{B^{\prime }}\sum _{A^{\prime }}p(B^{\prime })p(C|B^{\prime })p(A^{\prime }|C)}=\frac{\left(\sum _{B^{\prime }}p(B^{\prime })p(C|B^{\prime })\right)p(A|C)}{\left(\sum _{B^{\prime }}p(B^{\prime })p(C|B^{\prime })\right)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)}=p(A|C)$
$p(B,A|C)=p(B|C)p(A|C)$恰好成立

同父结构

下证$A\perp B|C$
$p(A,B,C)=p(C)p(A|C)p(B|C)$

$p(A,B|C)=\frac{p(A,B,C)}{p(C)}=\frac{p(A,B,C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{p(C)p(A|C)p(B|C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(C)p(A^{\prime }|C)p(B^{\prime }|C)}=\frac{p(C)p(A|C)p(B|C)}{p(C)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}=p(A|C)p(B|C)$
$p(A|C)=\frac{p(A,C)}{p(C)}=\frac{\sum _{B^{\prime }}p(A,B^{\prime },C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{\sum _{B^{\prime }}p(C)p(A|C)p(B^{\prime }|C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(C)p(A^{\prime }|C)p(B^{\prime }|C)}=\frac{p(C)p(A|C)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}{p(C)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}=p(A|C)$
$p(B|C)=\frac{p(B,C)}{p(C)}=\frac{\sum _{A^{\prime }}p(A^{\prime },B,C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{\sum _{A^{\prime }}p(C)p(A^{\prime }|C)p(B|C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(C)p(A^{\prime }|C)p(B^{\prime }|C)}=\frac{p(C)p(B|C)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)}{p(C)\left(\sum _{A^{\prime }}p(A^{\prime }|C)\right)\left(\sum _{B^{\prime }}p(B^{\prime }|C)\right)}=p(B|C)$
$p(A,B|C)=p(A|C)p(B|C)$恰好成立

V形结构

下证$A\perp B$
$p(A,B,C)=p(A)p(B)p(C|A,B)$
$p(A,B)=\sum _{C^{\prime }}p(A,B,C^{\prime })=\sum _{C^{\prime }}p(A)p(B)p(C^{\prime }|A,B)=p(A)p(B)\left(\sum _{C^{\prime }}p(C^{\prime }|A,B)\right)=p(A)p(B)$
$p(A)=\sum _{B^{\prime }}\sum _{C^{\prime }}p(A,B^{\prime },C^{\prime })=\sum _{B^{\prime }}\sum _{C^{\prime }}p(A)p(B^{\prime })p(C^{\prime }|A,B^{\prime })=p(A)\left[\sum _{B^{\prime }}p(B^{\prime })\left(\sum _{C^{\prime }}p(C^{\prime }|A,B^{\prime })\right)\right]=p(A)$
$p(B)=\sum _{A^{\prime }}\sum _{C^{\prime }}p(A^{\prime },B,C^{\prime })=\sum _{A^{\prime }}\sum _{C^{\prime }}p(A^{\prime })p(B)p(C^{\prime }|A^{\prime },B)=p(B)\left[\sum _{A^{\prime }}p(A^{\prime })\left(\sum _{C^{\prime }}p(C^{\prime }|A^{\prime },B)\right)\right]=p(B)$
$p(A,B)=p(A)p(B)$恰好成立

下证$A/\perp B|C$
$p(A,B,C)=p(A)p(B)p(C|A,B)$
$p(A,B|C)=\frac{p(A,B,C)}{p(C)}=\frac{p(A,B,C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{p(A)p(B)p(C|A,B)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(B^{\prime })p(C|A^{\prime },B^{\prime })}$
$p(A|C)=\frac{p(A,C)}{p(C)}=\frac{\sum _{B^{\prime }}p(A,B^{\prime },C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{\sum _{B^{\prime }}p(A)p(B^{\prime })p(C|A,B^{\prime })}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(B^{\prime })p(C|A^{\prime },B^{\prime })}=\frac{p(A)\sum _{B^{\prime }}p(B^{\prime })p(C|A,B^{\prime })}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(B^{\prime })p(C|A^{\prime },B^{\prime })}$
$p(B|C)=\frac{p(B,C)}{p(C)}=\frac{\sum _{A^{\prime }}p(A^{\prime },B,C)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime },B^{\prime },C)}=\frac{\sum _{A^{\prime }}p(A^{\prime })p(B)p(C|A^{\prime },B)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(B^{\prime })p(C|A^{\prime },B^{\prime })}=\frac{p(B)\sum _{A^{\prime }}p(A^{\prime })p(C|A^{\prime },B)}{\sum _{A^{\prime }}\sum _{B^{\prime }}p(A^{\prime })p(B^{\prime })p(C|A^{\prime },B^{\prime })}$
$p(A,B|C)=p(A|C)p(B|C)$可能成立, 也可能不成立, no guarantee.

反例

A代表𝐂𝐍’s realty market上涨的概率

A=0	A=1
0.5	0.5

B代表𝐔𝐒’s stock market上涨的概率

B=0	B=1
0.5	0.5

C代表投资组合盈利的概率

A=0 B=0	A=0 B=1	A=1 B=0	A=1 B=1
C=0,C=1	C=0,C=1	C=0,C=1	C=0,C=1
1.0,0.0	1.0,0.0	1.0,0.0	0.0,1.0

这个例子的现实意义是, 中美金融市场满足有效市场假设, 这个投资组合是风险对冲的, 盈利当且仅当𝐂𝐍’s realty market和𝐔𝐒’s stock market同时上涨.

注意p(A=1,B=1|C=0) = 0, 即这个投资组合不盈利时𝐂𝐍’s realty market和𝐔𝐒’s stock market不可能同时上涨.
但是p(A=1|C=0) > 0, p(B=1|C=0) > 0, 即这个投资组合不盈利时𝐂𝐍’s realty market可能上涨(但是此时𝐔𝐒’s stock market会跌), 这个投资组合不盈利时𝐔𝐒’s stock market可能上涨(但是此时𝐂𝐍’s realty market会跌).
这意味着, 这个投资组合不盈利时, 𝐂𝐍’s realty market和𝐔𝐒’s stock market不独立.
有趣的是, 这个投资组合盈利时, 𝐂𝐍’s realty market和𝐔𝐒’s stock market又独立了.
这就是no guarantee的含义.

上述例子使用0,1这种极端值是为了方便直观理解.
实际上, 将0全部替换为$\epsilon$, 将1全部替换为$1-\epsilon$, 上述例子仍然成立, 只是不够直观.

对于$\epsilon >0$的情形, 我们不妨这样思考, 随着$\epsilon \to 0$, p(A,B|C=0)趋向于不独立, 而p(A,B|C=1)趋向于独立.

例如

A=0	A=1
0.5	0.5

B=0	B=1
0.5	0.5

A=0 B=0	A=0 B=1	A=1 B=0	A=1 B=1
C=0,C=1	C=0,C=1	C=0,C=1	C=0,C=1
0.9,0.1	0.9,0.1	0.9,0.1	0.1,0.9

from itertools import product

A = [.5, .5,]
B = [.5, .5,]
C_AB = {
  (0,0,) : [1., 0.,],
  (0,1,) : [1., 0.,],
  (1,0,) : [1., 0.,],
  (1,1,) : [0., 1.,],
}

def ABC(a,b,c):
    Pa = A[a]
    Pb = A[b]
    Pc_ab = C_AB[a,b][c]
    return Pa * Pb * Pc_ab

def AB_C(a,b,c):
    Pabc = ABC(a,b,c)
    Pc = 0.
    for a,b, in product(range(2), range(2),):
        Pc += ABC(a,b,c)
    return Pabc / Pc

def A_C(a,c):
    Pac = 0.
    for b in range(2):
        Pac += ABC(a,b,c)
    Pc = 0.
    for a,b, in product(range(2), range(2),):
        Pc += ABC(a,b,c)
    return Pac / Pc

def B_C(b,c):
    Pbc = 0.
    for a in range(2):
        Pbc += ABC(a,b,c)
    Pc = 0.
    for a,b, in product(range(2), range(2),):
        Pc += ABC(a,b,c)
    return Pbc / Pc

if __name__ == '__main__':
    for c in range(2):
        print('='*50)
        for a,b, in product(range(2), range(2),):
            print('- '*25)
            print(f'p(A={a}|C={c}) p(B={b}|C={c}) = {A_C(a,c) * B_C(b,c):.4f}'
                + f' # = {A_C(a,c):.4f} x {B_C(b,c):.4f}')
            print(f'p(A={a},B={b}|C={c})        = {AB_C(a,b,c):.4f}')
#

==================================================
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=0|C=0) p(B=0|C=0) = 0.4444 # = 0.6667 x 0.6667
p(A=0,B=0|C=0)        = 0.3333
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=0|C=0) p(B=1|C=0) = 0.2222 # = 0.6667 x 0.3333
p(A=0,B=1|C=0)        = 0.3333
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=1|C=0) p(B=0|C=0) = 0.2222 # = 0.3333 x 0.6667
p(A=1,B=0|C=0)        = 0.3333
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=1|C=0) p(B=1|C=0) = 0.1111 # = 0.3333 x 0.3333
p(A=1,B=1|C=0)        = 0.0000
==================================================
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=0|C=1) p(B=0|C=1) = 0.0000 # = 0.0000 x 0.0000
p(A=0,B=0|C=1)        = 0.0000
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=0|C=1) p(B=1|C=1) = 0.0000 # = 0.0000 x 1.0000
p(A=0,B=1|C=1)        = 0.0000
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=1|C=1) p(B=0|C=1) = 0.0000 # = 1.0000 x 0.0000
p(A=1,B=0|C=1)        = 0.0000
- - - - - - - - - - - - - - - - - - - - - - - - -
p(A=1|C=1) p(B=1|C=1) = 1.0000 # = 1.0000 x 1.0000
p(A=1,B=1|C=1)        = 1.0000

V形子结构

A代表𝐂𝐍’s realty market上涨的概率

A=0	A=1
0.5	0.5

B代表𝐔𝐒’s stock market上涨的概率

B=0	B=1
0.5	0.5

C代表投资组合盈利的概率

A=0 B=0	A=0 B=1	A=1 B=0	A=1 B=1
C=0,C=1	C=0,C=1	C=0,C=1	C=0,C=1
1.0,0.0	1.0,0.0	1.0,0.0	0.0,1.0

D代表操作该投资组合的交易员加薪的概率

C=0	C=1
D=0,D=1	D=0,D=1
1.0,0.0	0.0,1.0

这个极端的例子中, C恒等于D, D观测到了意味着C也被完全观测到了.
由上一问我们知道, 一旦C被观测到为0, A,B之间将不再独立.
因此, 一旦D被观测到为0, A,B之间将不再独立.

上述例子使用0,1这种极端值是为了方便直观理解.
实际上, 将0全部替换为$\epsilon$, 将1全部替换为$1-\epsilon$, 上述例子仍然成立, 只是不够直观.

对于$\epsilon >0$的情形, 我们不妨这样思考, 随着$\epsilon \to 0$, p(A,B|D=0)趋向于不独立, 而p(A,B|D=1)趋向于独立.

例如

A=0	A=1
0.5	0.5

B=0	B=1
0.5	0.5

A=0 B=0	A=0 B=1	A=1 B=0	A=1 B=1
C=0,C=1	C=0,C=1	C=0,C=1	C=0,C=1
0.9,0.1	0.9,0.1	0.9,0.1	0.1,0.9

C=0	C=1
D=0,D=1	D=0,D=1
0.9,0.1	0.1,0.9