T1
在一个二进制的信道中，信源消息集 X={0,1}且 p(1)=p(0),信宿的消息集 Y={0},1,
信道传输概率
p
(
y
=
1
∣
x
=
0
)
=
1
/
4
p( y= 1| x= 0) = 1/ 4
p(y=1∣x=0)=1/4,
p
(
y
=
0
∣
x
=
1
)
=
1
/
8
p( y= 0| x= 1) = 1/ 8
p(y=0∣x=1)=1/8。求：
(1)在接收端收到 y=0 后，所提供的关于传输消息 x 的平均条件互信息 I(X; y=0);
(2) 该情况下所能提供的平均互信息量 I(X; Y)。
解：
(1)
p
(
y
∣
x
)
=
[
3
4
1
4
1
8
7
8
]
,
p
(
x
y
)
=
[
3
8
1
8
1
16
7
16
]
,
p
(
x
∣
y
)
=
[
6
7
2
9
1
7
7
9
]
\mathrm{p(y\mid x)=\begin{bmatrix}\dfrac{3}{4}&\dfrac{1}{4}\\\dfrac{1}{8}&\dfrac{7}{8}\end{bmatrix},~p(xy)=~\begin{bmatrix}\dfrac{3}{8}&\dfrac{1}{8}\\\dfrac{1}{16}&\dfrac{7}{16}\end{bmatrix},~p(x\mid y)=~\begin{bmatrix}\dfrac{6}{7}&\dfrac{2}{9}\\\dfrac{1}{7}&\dfrac{7}{9}\end{bmatrix}}
p(y∣x)=
43814187
, p(xy)=
8316181167
, p(x∣y)=
76719297
p
(
y
=
0
)
=
3
8
+
1
16
=
7
16
p( y= 0) = \frac 38+ \frac 1{16}= \frac 7{16}
p(y=0)=83+161=167
p
(
y
=
1
)
=
1
8
+
7
16
=
9
16
p( y= 1) = \frac 18+ \frac 7{16}= \frac 9{16}
p(y=1)=81+167=169
(
X
:
y
=
0
)
=
H
(
X
)
−
H
(
X
∣
0
)
=
1
+
(
6
7
log
6
7
+
1
7
log
1
7
)
=
0.408
(
b
i
t
/
s
y
m
b
o
l
)
(X: y= 0) = H( X) - H( X| 0) = 1+ ( \frac 67\log \frac 67+ \frac 17\log \frac 17) = 0. 408( bit/ symbol)
(X:y=0)=H(X)−H(X∣0)=1+(76log76+71log71)=0.408(bit/symbol)
(2)
I
(
X
:
y
=
1
)
=
H
(
X
)
−
H
(
X
∣
1
)
=
1
+
(
2
9
log
2
9
+
7
9
log
7
9
)
=
0.236
(
b
i
t
/
s
y
m
b
o
l
)
I(X:y= 1) = H( X) - H( X| 1) = 1+ ( \frac 29\log \frac 29+ \frac 79\log \frac 79) = 0. 236( bit/ symbol)
I(X:y=1)=H(X)−H(X∣1)=1+(92log92+97log97)=0.236(bit/symbol)
I ( X ; Y ) = ∑ Y p ( y ) I ( X ; y ) = 7 16 × 0.408 + 9 16 × 0.236 ≈ 0.311 ( b i t / s y m b o l ) I(X;Y)=\sum_Yp(y)I(X;y)=\frac7{16}\times0.408+\frac9{16}\times0.236\approx0.311(bit/symbol) I(X;Y)=Y∑p(y)I(X;y)=167×0.408+169×0.236≈0.311(bit/symbol)
or: I ( X ; Y ) = H ( X ) − H ( X / Y ) I( X; Y) = H( X) - H( X/ Y) I(X;Y)=H(X)−H(X/Y)
= 1 + ( 3 8 log 6 7 + 1 8 log 2 9 + 1 16 log 1 7 + 7 16 log 7 9 ) ≈ 1 − 0.6883 = 0.3117 ( b i t / s y m b o l ) =1+(\frac38\log\frac67+\frac18\log\frac29+\frac1{16}\log\frac17+\frac7{16}\log\frac79)\approx1-0.6883=0.3117(bit/symbol) =1+(83log76+81log92+161log71+167log97)≈1−0.6883=0.3117(bit/symbol)
T2
一个信源发出二重符号序列消息
(
X
1
,
X
2
)
(X_1,X_2)
(X1,X2),其中第一个符号
X
1
X_1
X1可以是A、B、C中的任一个，第二个符号
X
2
X_2
X2可以是D、E、F、G中的任一个。已知各个
p
(
x
1
i
)
p(x_{_{1i}})
p(x1i)为：
p
(
A
)
=
1
/
2
,
p
(
B
)
=
1
/
3
p(A)=1/2,p(B)=1/3
p(A)=1/2,p(B)=1/3,
p
(
C
)
=
1
/
6
p(C)=1/6
p(C)=1/6; 各个
p
(
x
2
j
∣
x
ı
i
)
p(x_{2j}|x_{\text{\i}i})
p(x2j∣xıi)值如下表所示。求这个信源的熵(联合熵
H
(
X
1
X
2
)
)
H(X_1X_2))
H(X1X2))。
| X_{1i}X_{2i} | A | B | C |
|---|---|---|---|
| D | 1/4 | 3/10 | 1/6 |
| E | 1/4 | 1/5 | 1/2 |
| F | 1/4 | 1/5 | 1/6 |
| G | 1/4 | 3/10 | 1/6 |
解：
P
(
x
2
∣
x
1
)
=
[
1
4
1
4
1
4
1
4
3
10
1
5
1
5
3
10
1
6
1
3
1
6
1
6
]
P
(
x
1
)
=
[
1
2
1
3
1
6
]
P
(
x
1
x
2
)
=
[
1
8
1
8
1
8
1
8
1
10
1
15
1
15
1
10
1
36
1
12
1
36
1
36
]
\begin{gathered}P(x_2\mid x_1)=\begin{bmatrix}\dfrac{1}{4}&\dfrac{1}{4}&\dfrac{1}{4}&\dfrac{1}{4}\\\dfrac{3}{10}&\dfrac{1}{5}&\dfrac{1}{5}&\dfrac{3}{10}\\\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}\end{bmatrix}\quad P(x_1)=\begin{bmatrix}\dfrac{1}{2}\\\dfrac{1}{3}\\\dfrac{1}{6}\end{bmatrix}\quad P(x_1x_2)=\begin{bmatrix}\dfrac{1}{8}&\dfrac{1}{8}&\dfrac{1}{8}&\dfrac{1}{8}\\\dfrac{1}{10}&\dfrac{1}{15}&\dfrac{1}{15}&\dfrac{1}{10}\\\dfrac{1}{36}&\dfrac{1}{12}&\dfrac{1}{36}&\dfrac{1}{36}\end{bmatrix}\end{gathered}
P(x2∣x1)=
41103614151314151614110361
P(x1)=
213161
P(x1x2)=
81101361811511218115136181101361
H
(
X
1
X
2
)
=
4
⋅
1
8
⋅
L
o
g
(
8
)
+
2
⋅
1
10
⋅
L
o
g
(
10
)
+
2
⋅
1
15
⋅
L
o
g
(
15
)
+
3
⋅
1
36
⋅
L
o
g
(
36
)
+
1
12
⋅
L
o
g
(
12
)
=
3.415
\mathrm{H}(X_{1}X_{2})=\:4\cdot\frac{1}{8}\cdot\mathrm{Log}(8)+2\cdot\frac{1}{10}\cdot\mathrm{Log}(10)+2\cdot\frac{1}{15}\cdot\mathrm{Log}(15)+3\cdot\frac{1}{36}\cdot\mathrm{Log}(36)+\frac{1}{12}\cdot\mathrm{Log}(12)=3.415
H(X1X2)=4⋅81⋅Log(8)+2⋅101⋅Log(10)+2⋅151⋅Log(15)+3⋅361⋅Log(36)+121⋅Log(12)=3.415
扫一扫
2630
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
