例1：均匀分布
注意，本文所举实例中对数运算均采用自然对数底数
e
e
e，并且约定：
0
×
log
0
=
0
0\times \log 0=0
0×log0=0
假设随机变量 X X X， Y Y Y， Z Z Z：
| X X X | 1 1 1 | 1 1 1 | 1 1 1 | 0 0 0 | 0 0 0 | 0 0 0 |
|---|---|---|---|---|---|---|
| Y Y Y | 0 0 0 | 1 1 1 | 1 1 1 | 1 1 1 | 0 0 0 | 0 0 0 |
| Z Z Z | 0 0 0 | 0 0 0 | 1 1 1 | 1 1 1 | 1 1 1 | 0 0 0 |
例1.1 信息熵
由信息熵的定义，有：
H
(
X
)
=
−
∑
x
∈
X
p
(
x
)
⋅
log
p
(
x
)
=
−
1
2
log
1
2
−
1
2
log
1
2
=
log
2
=
0.693147
H
(
Y
)
=
−
∑
y
∈
Y
p
(
y
)
⋅
log
p
(
y
)
=
−
1
2
log
1
2
−
1
2
log
1
2
=
log
2
=
0.693147
H
(
Z
)
=
−
∑
z
∈
Z
p
(
z
)
⋅
log
p
(
z
)
=
−
1
2
log
1
2
−
1
2
log
1
2
=
log
2
=
0.693147
\begin{aligned} H\left( X \right) &=-\sum_{x\in X}{p\left( x \right)}\cdot \log p\left( x \right) =-\frac{1}{2}\log \frac{1}{2}-\frac{1}{2}\log \frac{1}{2} \\ &=\log 2=0.693147 \\ H\left( Y \right) &=-\sum_{y\in Y}{p\left( y \right)}\cdot \log p\left( y \right) =-\frac{1}{2}\log \frac{1}{2}-\frac{1}{2}\log \frac{1}{2} \\ &=\log 2=0.693147 \\ H\left( Z \right) &=-\sum_{z\in Z}{p\left( z \right)}\cdot \log p\left( z \right) =-\frac{1}{2}\log \frac{1}{2}-\frac{1}{2}\log \frac{1}{2} \\ &=\log 2=0.693147 \end{aligned}
H(X)H(Y)H(Z)=−x∈X∑p(x)⋅logp(x)=−21log21−21log21=log2=0.693147=−y∈Y∑p(y)⋅logp(y)=−21log21−21log21=log2=0.693147=−z∈Z∑p(z)⋅logp(z)=−21log21−21log21=log2=0.693147
例1.2 联合熵
由概率可知：
p
(
x
=
0
,
y
=
0
)
=
1
3
，
p
(
x
=
0
,
y
=
1
)
=
1
6
p
(
x
=
1
,
y
=
0
)
=
1
6
，
p
(
x
=
1
,
y
=
1
)
=
1
3
p\left( x=0,y=0 \right) =\frac{1}{3}\text{，}p\left( x=0,y=1 \right) =\frac{1}{6} \\ \ \ \\ \\p\left( x=1,y=0 \right) =\frac{1}{6}\text{，}p\left( x=1,y=1 \right) =\frac{1}{3}
p(x=0,y=0)=31，p(x=0,y=1)=61 p(x=1,y=0)=61，p(x=1,y=1)=31则：
H
(
X
,
Y
)
=
−
∑
x
∈
X
∑
y
∈
Y
p
(
x
,
y
)
log
p
(
x
,
y
)
=
−
1
6
log
1
6
−
1
3
log
1
3
−
1
3
log
1
3
−
1
6
log
1
6
=
1
3
×
(
log
6
+
log
9
)
=
1.32966
\begin{aligned} H\left( X,Y \right) &=-\sum_{x\in X}{\sum_{y\in Y}{p\left( x,y \right)}\log}p\left( x,y \right) \\ \\ &=-\frac{1}{6}\log \frac{1}{6}-\frac{1}{3}\log \frac{1}{3}-\frac{1}{3}\log \frac{1}{3}-\frac{1}{6}\log \frac{1}{6} \\ \\ &=\frac{1}{3}\times \left( \log 6+\log 9 \right) =1.32966 \end{aligned}
H(X,Y)=−x∈X∑y∈Y∑p(x,y)logp(x,y)=−61log61−31log31−31log31−61log61=31×(log6+log9)=1.32966
例1.3 条件熵
计算可得
p
(
x
=
0
∣
y
=
0
)
=
p
(
x
=
0
,
y
=
0
)
p
(
y
=
0
)
=
1
3
1
2
=
2
3
p
(
x
=
1
∣
y
=
0
)
=
p
(
x
=
1
,
y
=
0
)
p
(
y
=
0
)
=
1
6
1
2
=
1
3
p
(
x
=
0
∣
y
=
1
)
=
p
(
x
=
0
,
y
=
1
)
p
(
y
=
1
)
=
1
6
1
2
=
1
3
p
(
x
=
1
∣
y
=
1
)
=
p
(
x
=
1
,
y
=
1
)
p
(
y
=
1
)
=
1
3
1
2
=
2
3
p\left( x=0\left| y=0 \right. \right) =\frac{p\left( x=0,y=0 \right)}{p\left( y=0 \right)}=\frac{\frac{1}{3}}{\frac{1}{2}}=\frac{2}{3} \\ \ \ \\ p\left( x=1\left| y=0 \right. \right) =\frac{p\left( x=1,y=0 \right)}{p\left( y=0 \right)}=\frac{\frac{1}{6}}{\frac{1}{2}}=\frac{1}{3} \\ \ \ \\ p\left( x=0\left| y=1 \right. \right) =\frac{p\left( x=0,y=1 \right)}{p\left( y=1 \right)}=\frac{\frac{1}{6}}{\frac{1}{2}}=\frac{1}{3} \\ \ \ \\ p\left( x=1\left| y=1 \right. \right) =\frac{p\left( x=1,y=1 \right)}{p\left( y=1 \right)}=\frac{\frac{1}{3}}{\frac{1}{2}}=\frac{2}{3}
p(x=0∣y=0)=p(y=0)p(x=0,y=0)=2131=32 p(x=1∣y=0)=p(y=0)p(x=1,y=0)=2161=31 p(x=0∣y=1)=p(y=1)p(x=0,y=1)=2161=31 p(x=1∣y=1)=p(y=1)p(x=1,y=1)=2131=32则
H
(
X
∣
Y
=
0
)
=
−
p
(
x
=
0
∣
y
=
0
)
⋅
log
p
(
x
=
0
∣
y
=
0
)
−
p
(
x
=
1
∣
y
=
0
)
⋅
log
p
(
x
=
1
∣
y
=
0
)
=
−
2
3
log
2
3
−
1
3
log
1
3
=
log
3
−
2
3
log
2
\begin{aligned} H\left( X\left| Y \right. =0 \right) &=-p\left( x=0\left| y=0 \right. \right) \cdot \log p\left( x=0\left| y=0 \right. \right) -p\left( x=1\left| y=0 \right. \right) \cdot \log p\left( x=1\left| y=0 \right. \right) \\ \ \ \\ &=-\frac{2}{3}\log \frac{2}{3}-\frac{1}{3}\log \frac{1}{3} \\ \ \ \\ &=\log 3-\frac{2}{3}\log 2 \end{aligned}
H(X∣Y=0) =−p(x=0∣y=0)⋅logp(x=0∣y=0)−p(x=1∣y=0)⋅logp(x=1∣y=0)=−32log32−31log31=log3−32log2
H
(
X
∣
Y
=
1
)
=
−
p
(
x
=
0
∣
y
=
1
)
⋅
log
p
(
x
=
0
∣
y
=
1
)
−
p
(
x
=
1
∣
y
=
1
)
⋅
log
p
(
x
=
1
∣
y
=
1
)
=
−
1
3
log
1
3
−
2
3
log
2
3
=
log
3
−
2
3
log
2
\begin{aligned} H\left( X\left| Y \right. =1 \right) &=-p\left( x=0\left| y=1 \right. \right) \cdot \log p\left( x=0\left| y=1 \right. \right) -p\left( x=1\left| y=1 \right. \right) \cdot \log p\left( x=1\left| y=1 \right. \right) \\ \ \ \\ &=-\frac{1}{3}\log \frac{1}{3}-\frac{2}{3}\log \frac{2}{3} \\ \ \ \\ &=\log 3-\frac{2}{3}\log 2 \end{aligned}
H(X∣Y=1) =−p(x=0∣y=1)⋅logp(x=0∣y=1)−p(x=1∣y=1)⋅logp(x=1∣y=1)=−31log31−32log32=log3−32log2
通过以上计算可到条件熵为：
H
(
X
∣
Y
)
=
p
(
y
=
0
)
⋅
H
(
X
∣
Y
=
0
)
+
p
(
y
=
1
)
⋅
H
(
X
∣
Y
=
1
)
=
1
2
×
(
log
3
−
2
3
log
2
)
+
1
2
×
(
log
3
−
2
3
log
2
)
=
log
3
−
2
3
log
2
=
0.636514
\begin{aligned} H\left( X\left| Y \right. \right) &=p\left( y=0 \right) \cdot H\left( X\left| Y=0 \right. \right) +p\left( y=1 \right) \cdot H\left( X\left| Y=1 \right. \right) \\ \ \ \\ &=\frac{1}{2}\times \left( \log 3-\frac{2}{3}\log 2 \right) +\frac{1}{2}\times \left( \log 3-\frac{2}{3}\log 2 \right) \\ \ \ \\ &=\log 3-\frac{2}{3}\log 2=0.636514 \end{aligned}
H(X∣Y) =p(y=0)⋅H(X∣Y=0)+p(y=1)⋅H(X∣Y=1)=21×(log3−32log2)+21×(log3−32log2)=log3−32log2=0.636514同理可计算出
H
(
Y
∣
X
)
=
log
3
−
2
3
log
2
=
0.636514
H\left( Y\left| X \right. \right) =\log 3-\frac{2}{3}\log 2=0.636514
H(Y∣X)=log3−32log2=0.636514
例1.4 互信息
p ( x = 0 , y = 0 ) = 1 3 ， p ( x = 0 , y = 1 ) = 1 6 ， p ( x = 1 , y = 0 ) = 1 6 ， p ( x = 1 , y = 1 ) = 1 3 p\left( x=0,y=0 \right) =\frac{1}{3}\text{，}p\left( x=0,y=1 \right) =\frac{1}{6}\text{，}p\left( x=1,y=0 \right) =\frac{1}{6}\text{，}p\left( x=1,y=1 \right) =\frac{1}{3} p(x=0,y=0)=31，p(x=0,y=1)=61，p(x=1,y=0)=61，p(x=1,y=1)=31
则
X
X
X和
Y
Y
Y的互信息为：
I
(
X
;
Y
)
=
p
(
x
=
1
,
y
=
0
)
⋅
log
p
(
x
=
1
,
y
=
0
)
p
(
x
=
1
)
⋅
p
(
y
=
0
)
+
p
(
x
=
1
,
y
=
1
)
⋅
log
p
(
x
=
1
,
y
=
1
)
p
(
x
=
1
)
⋅
p
(
y
=
1
)
+
p
(
x
=
0
,
y
=
0
)
⋅
log
p
(
x
=
0
,
y
=
0
)
p
(
x
=
0
)
⋅
p
(
y
=
0
)
+
p
(
x
=
0
,
y
=
1
)
⋅
log
p
(
x
=
0
,
y
=
1
)
p
(
x
=
0
)
⋅
p
(
y
=
1
)
=
1
6
×
log
1
6
1
2
×
1
2
+
1
3
×
log
1
3
1
2
×
1
2
+
1
3
×
log
1
3
1
2
×
1
2
+
1
6
×
log
1
6
1
2
×
1
2
=
1
3
×
(
5
log
2
−
3
log
3
)
=
0.056633
\begin{aligned} I\left( X;Y \right) &=p\left( x=1,y=0 \right) \cdot \log \frac{p\left( x=1,y=0 \right)}{p\left( x=1 \right) \cdot p\left( y=0 \right)}+p\left( x=1,y=1 \right) \cdot \log \frac{p\left( x=1,y=1 \right)}{p\left( x=1 \right) \cdot p\left( y=1 \right)} \\ \\ &\,\, +p\left( x=0,y=0 \right) \cdot \log \frac{p\left( x=0,y=0 \right)}{p\left( x=0 \right) \cdot p\left( y=0 \right)}+p\left( x=0,y=1 \right) \cdot \log \frac{p\left( x=0,y=1 \right)}{p\left( x=0 \right) \cdot p\left( y=1 \right)} \\ \\ &=\frac{1}{6}\times \log \frac{\small{\frac{1}{6}}}{\frac{1}{2}\times \frac{1}{2}}+\frac{1}{3}\times \log \frac{\frac{1}{3}}{\frac{1}{2}\times \frac{1}{2}}+\frac{1}{3}\times \log \frac{\frac{1}{3}}{\frac{1}{2}\times \frac{1}{2}}+\frac{1}{6}\times \log \frac{\frac{1}{6}}{\frac{1}{2}\times \frac{1}{2}} \\ \\ &=\frac{1}{3}\times \left( 5\log 2-3\log 3 \right) =0.056633 \end{aligned}
I(X;Y)=p(x=1,y=0)⋅logp(x=1)⋅p(y=0)p(x=1,y=0)+p(x=1,y=1)⋅logp(x=1)⋅p(y=1)p(x=1,y=1)+p(x=0,y=0)⋅logp(x=0)⋅p(y=0)p(x=0,y=0)+p(x=0,y=1)⋅logp(x=0)⋅p(y=1)p(x=0,y=1)=61×log21×2161+31×log21×2131+31×log21×2131+61×log21×2161=31×(5log2−3log3)=0.056633
例1.5 条件互信息
p
(
z
=
0
)
=
1
2
，
p
(
z
=
1
)
=
1
2
p
(
x
=
1
∣
z
=
0
)
=
2
3
，
p
(
x
=
0
∣
z
=
0
)
=
1
3
，
p
(
x
=
1
∣
z
=
1
)
=
1
3
，
p
(
x
=
0
∣
z
=
1
)
=
2
3
p
(
y
=
1
∣
z
=
0
)
=
1
3
，
p
(
y
=
0
∣
z
=
0
)
=
2
3
，
p
(
y
=
1
∣
z
=
1
)
=
1
3
，
p
(
y
=
0
∣
z
=
1
)
=
2
3
p\left( z=0 \right) =\frac{1}{2}，p\left( z=1 \right) =\frac{1}{2} \\ \ \ \\ p\left( \left. x=1 \right|z=0 \right) =\frac{2}{3}\text{，}p\left( \left. x=0 \right|z=0 \right) =\frac{1}{3}\text{，}p\left( \left. x=1 \right|z=1 \right) =\frac{1}{3}\text{，}p\left( \left. x=0 \right|z=1 \right) =\frac{2}{3} \\ \ \ \\ p\left( \left. y=1 \right|z=0 \right) =\frac{1}{3}\text{，}p\left( \left. y=0 \right|z=0 \right) =\frac{2}{3}\text{，}p\left( \left. y=1 \right|z=1 \right) =\frac{1}{3}\text{，}p\left( \left. y=0 \right|z=1 \right) =\frac{2}{3} \\
p(z=0)=21，p(z=1)=21 p(x=1∣z=0)=32，p(x=0∣z=0)=31，p(x=1∣z=1)=31，p(x=0∣z=1)=32 p(y=1∣z=0)=31，p(y=0∣z=0)=32，p(y=1∣z=1)=31，p(y=0∣z=1)=32记
p
11
=
p
(
x
=
1
,
y
=
0
∣
z
=
0
)
=
1
3
，
p
21
=
p
(
x
=
1
,
y
=
1
∣
z
=
0
)
=
1
3
，
p
31
=
p
(
x
=
0
,
y
=
0
∣
z
=
0
)
=
1
3
，
p
41
=
p
(
x
=
0
,
y
=
1
∣
z
=
0
)
=
0
p
12
=
p
(
x
=
1
,
y
=
0
∣
z
=
1
)
=
0
，
p
22
=
p
(
x
=
1
,
y
=
1
∣
z
=
1
)
=
1
3
，
p
32
=
p
(
x
=
0
,
y
=
0
∣
z
=
1
)
=
1
3
，
p
42
=
p
(
x
=
0
,
y
=
1
∣
z
=
1
)
=
1
3
p_{11}=p\left( \left. x=1,y=0 \right|z=0 \right) =\frac{1}{3}\text{，}p_{21}=p\left( \left. x=1,y=1 \right|z=0 \right) =\frac{1}{3}\text{，}p_{31}=p\left( \left. x=0,y=0 \right|z=0 \right) =\frac{1}{3}\text{，}p_{41}=p\left( \left. x=0,y=1 \right|z=0 \right) =0 \\ \mathrm{ } \\ p_{12}=p\left( \left. x=1,y=0 \right|z=1 \right) =0\text{，}p_{22}=p\left( \left. x=1,y=1 \right|z=1 \right) =\frac{1}{3}\text{，}p_{32}=p\left( \left. x=0,y=0 \right|z=1 \right) =\frac{1}{3}\text{，}p_{42}=p\left( \left. x=0,y=1 \right|z=1 \right) =\frac{1}{3}
p11=p(x=1,y=0∣z=0)=31，p21=p(x=1,y=1∣z=0)=31，p31=p(x=0,y=0∣z=0)=31，p41=p(x=0,y=1∣z=0)=0p12=p(x=1,y=0∣z=1)=0，p22=p(x=1,y=1∣z=1)=31，p32=p(x=0,y=0∣z=1)=31，p42=p(x=0,y=1∣z=1)=31
P
11
=
p
(
x
=
1
,
y
=
0
∣
z
=
0
)
p
(
x
=
1
∣
z
=
0
)
⋅
p
(
y
=
0
∣
z
=
0
)
=
1
3
2
3
×
2
3
=
3
4
，
P
12
=
p
(
x
=
1
,
y
=
0
∣
z
=
1
)
p
(
x
=
1
∣
z
=
1
)
⋅
p
(
y
=
0
∣
z
=
1
)
=
0
P
21
=
p
(
x
=
1
,
y
=
1
∣
z
=
0
)
p
(
x
=
1
∣
z
=
0
)
⋅
p
(
y
=
1
∣
z
=
0
)
=
1
3
2
3
×
1
3
=
3
2
，
P
22
=
p
(
x
=
1
,
y
=
1
∣
z
=
1
)
p
(
x
=
1
∣
z
=
1
)
⋅
p
(
y
=
1
∣
z
=
1
)
=
1
3
1
3
×
2
3
=
3
2
P
31
=
p
(
x
=
0
,
y
=
0
∣
z
=
0
)
p
(
x
=
0
∣
z
=
0
)
⋅
p
(
y
=
0
∣
z
=
0
)
=
1
3
1
3
×
2
3
=
3
2
，
P
32
=
p
(
x
=
0
,
y
=
0
∣
z
=
1
)
p
(
x
=
0
∣
z
=
1
)
⋅
p
(
y
=
0
∣
z
=
1
)
=
1
3
2
3
×
1
3
=
3
2
，
P
41
=
p
(
x
=
0
,
y
=
1
∣
z
=
0
)
p
(
x
=
0
∣
z
=
0
)
⋅
p
(
y
=
1
∣
z
=
0
)
=
0
，
P
42
=
p
(
x
=
0
,
y
=
1
∣
z
=
1
)
p
(
x
=
0
∣
z
=
1
)
⋅
p
(
y
=
1
∣
z
=
1
)
=
1
3
2
3
×
2
3
=
3
4
\begin{aligned} P_{11}&=\frac{p\left( x=1,y=0\left| z=0 \right. \right)}{p\left( x=1\left| z=0 \right. \right) \cdot p\left( y=0\left| z=0 \right. \right)}=\frac{\frac{1}{3}}{\frac{2}{3}\times \frac{2}{3}}=\frac{3}{4}\text{，}P_{12}=\frac{p\left( x=1,y=0\left| z=1 \right. \right)}{p\left( x=1\left| z=1 \right. \right) \cdot p\left( y=0\left| z=1 \right. \right)}=0 \\ \ \ \\ P_{21}&=\frac{p\left( x=1,y=1\left| z=0 \right. \right)}{p\left( x=1\left| z=0 \right. \right) \cdot p\left( y=1\left| z=0 \right. \right)}=\frac{\frac{1}{3}}{\frac{2}{3}\times \frac{1}{3}}=\frac{3}{2}\text{，}P_{22}=\frac{p\left( x=1,y=1\left| z=1 \right. \right)}{p\left( x=1\left| z=1 \right. \right) \cdot p\left( y=1\left| z=1 \right. \right)}=\frac{\frac{1}{3}}{\frac{1}{3}\times \frac{2}{3}}=\frac{3}{2} \\ \ \ \\ P_{31}&=\frac{p\left( x=0,y=0\left| z=0 \right. \right)}{p\left( x=0\left| z=0 \right. \right) \cdot p\left( y=0\left| z=0 \right. \right)}=\frac{\frac{1}{3}}{\frac{1}{3}\times \frac{2}{3}}=\frac{3}{2}\text{，}P_{32}=\frac{p\left( x=0,y=0\left| z=1 \right. \right)}{p\left( x=0\left| z=1 \right. \right) \cdot p\left( y=0\left| z=1 \right. \right)}=\frac{\frac{1}{3}}{\frac{2}{3}\times \frac{1}{3}}=\frac{3}{2}\text{，} \\ \ \ \\ P_{41}&=\frac{p\left( x=0,y=1\left| z=0 \right. \right)}{p\left( x=0\left| z=0 \right. \right) \cdot p\left( y=1\left| z=0 \right. \right)}=0\text{，} P_{42}=\frac{p\left( x=0,y=1\left| z=1 \right. \right)}{p\left( x=0\left| z=1 \right. \right) \cdot p\left( y=1\left| z=1 \right. \right)}=\frac{\frac{1}{3}}{\frac{2}{3}\times \frac{2}{3}}=\frac{3}{4} \end{aligned}
P11 P21 P31 P41=p(x=1∣z=0)⋅p(y=0∣z=0)p(x=1,y=0∣z=0)=32×3231=43，P12=p(x=1∣z=1)⋅p(y=0∣z=1)p(x=1,y=0∣z=1)=0=p(x=1∣z=0)⋅p(y=1∣z=0)p(x=1,y=1∣z=0)=32×3131=23，P22=p(x=1∣z=1)⋅p(y=1∣z=1)p(x=1,y=1∣z=1)=31×3231=23=p(x=0∣z=0)⋅p(y=0∣z=0)p(x=0,y=0∣z=0)=31×3231=23，P32=p(x=0∣z=1)⋅p(y=0∣z=1)p(x=0,y=0∣z=1)=32×3131=23，=p(x=0∣z=0)⋅p(y=1∣z=0)p(x=0,y=1∣z=0)=0，P42=p(x=0∣z=1)⋅p(y=1∣z=1)p(x=0,y=1∣z=1)=32×3231=43故条件互信息为：
I
(
X
;
Y
∣
Z
)
=
∑
z
∈
Z
p
(
z
)
∑
x
∈
X
∑
y
∈
Y
p
(
x
,
y
∣
z
)
⋅
log
p
(
x
,
y
∣
z
)
p
(
x
∣
z
)
p
(
y
∣
z
)
=
p
(
z
=
0
)
×
[
p
11
⋅
log
P
11
+
p
21
⋅
log
P
21
+
p
31
⋅
log
P
31
+
p
41
⋅
log
P
41
]
+
p
(
z
=
1
)
×
[
p
12
⋅
log
P
12
+
p
22
⋅
log
P
22
+
p
32
⋅
log
P
32
+
p
42
⋅
log
P
42
]
=
1
2
×
[
1
3
log
3
4
+
1
3
log
3
2
+
1
3
log
3
2
+
0
]
+
1
2
×
[
0
+
1
3
log
3
2
+
1
3
log
3
2
+
1
3
log
3
4
]
=
log
3
−
4
3
log
2
=
0.174416
\begin{aligned} I\left( \left. X;Y \right|Z \right) &=\sum_{z\in Z}{p\left( z \right)}\sum_{x\in X}{\sum_{y\in Y}{p\left( \left. x,y \right|z \right)}}·\log \frac{p\left( \left. x,y \right|z \right)}{p\left( \left. x \right|z \right) p\left( \left. y \right|z \right)} \\ \ \ \\ &=p\left( z=0 \right) \times \left[ p_{11}\cdot \log P_{11}+p_{21}\cdot \log P_{21}+p_{31}\cdot \log P_{31}+p_{41}\cdot \log P_{41} \right] \\ \ \ \\ &+p\left( z=1 \right) \times \left[ p_{12}\cdot \log P_{12}+p_{22}\cdot \log P_{22}+p_{32}\cdot \log P_{32}+p_{42}\cdot \log P_{42} \right] \\ \ \ \\ &=\frac{1}{2}\times \left[ \frac{1}{3}\log \frac{3}{4}+\frac{1}{3}\log \frac{3}{2}+\frac{1}{3}\log \frac{3}{2}+0 \right] +\frac{1}{2}\times \left[ 0+\frac{1}{3}\log \frac{3}{2}+\frac{1}{3}\log \frac{3}{2}+\frac{1}{3}\log \frac{3}{4} \right] \\ \ \ \\ &=\log 3-\frac{4}{3}\log 2=0.174416 \end{aligned}
I(X;Y∣Z) =z∈Z∑p(z)x∈X∑y∈Y∑p(x,y∣z)⋅logp(x∣z)p(y∣z)p(x,y∣z)=p(z=0)×[p11⋅logP11+p21⋅logP21+p31⋅logP31+p41⋅logP41]+p(z=1)×[p12⋅logP12+p22⋅logP22+p32⋅logP32+p42⋅logP42]=21×[31log43+31log23+31log23+0]+21×[0+31log23+31log23+31log43]=log3−34log2=0.174416
2233
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
