Distributed Source Coding_souce coding-CSDN博客

本文链接：https://blog.csdn.net/qq_39599295/article/details/115652131

Reference:

Elements of Information Theory, 2nd Edition

Slides of EE4560, TUD

Content

Introduction

We know how to encode a source $X$ . A rate $R\ge H(X)$ is sufficient.
If there are two sources $(X, Y)$ , a rate $R\ge H(X,Y)$ is sufficient.

在这里插入图片描述

But what if the $X$ and $Y$ sources must be described separately for some user who wishes to reconstruct both $X$ and $Y$ ?
Clearly, by separately encoding $X$ and $Y$ , it is seen that a rate $R=R_x+R_y\ge H(X)+H(Y)$ is sufficient.
However, in a surprising and fundamental paper by Slepian and Wolf, it is shown that a total rate $R = H (X, Y)$ is sufficient even for separate encoding of correlated sources.
Intuitively, since $H (X, Y) = H (X) + H (Y ∣ X)$ , we can first encode source $X$ at a rate $R_1\ge H(X)$ after which we encode source $Y$ , given $X$ , at a rate $R_2\ge H(Y|X)$ .

More specifically,
- Using $n H (X)$ bits we can encode $X^{n}$ efficiently, so that the decoder can reconstruct $X^{n}$ with arbitrarily low probability of error
- Associated with every $x^{n}$ is a typical “fan” of $y^{n}$ sequences that are jointly typical with the given $x^{n}, 2^{n H(Y | X)}$ in total
- The encoder can send the index of the $y^{n}$ within this typical fan for which he needs $n H (Y ∣ X)$ bits
- The decoder, also knowing $x^{n}$ , can then construct the typical fan and hence reconstruct $y^{n}$

在这里插入图片描述

The graph of the whole process can be presented as

在这里插入图片描述

But what if the $Y$ encoder does not know which sequence $x^n$ is encoded?

在这里插入图片描述

Slepian-Wolf Coding

Let $\left(X_{1}, Y_{1}\right),\left(X_{2}, Y_{2}\right), \ldots$ be a sequence of jointly distributed random variables i.i.d. $\sim p(x, y)$

Definition 1 (Distributed source code):

A $\left(\left(2^{n R_{1}}, 2^{n R_{2}}\right), n\right)$ distributed source code for the joint sources $(X, Y)$ consists of two encoder maps
$\begin{array}{l} f_{1}: \mathcal{X}^{n} \rightarrow\left\{1,2, \ldots, 2^{n R_{1}}\right\} \\ f_{2}: \mathcal{Y}^{n} \rightarrow\left\{1,2, \ldots, 2^{n R_{2}}\right\} \end{array}$
and a decoder map
$g:\left\{1,2, \ldots, 2^{n R_{1}}\right\} \times\left\{1,2, \ldots, 2^{n R_{2}}\right\} \rightarrow \mathcal{X}^{n} \times \mathcal{Y}^{n}$
Definition 2 (Probability of error):

The probability of error for a distributed source code is defined as
$P_{\epsilon}^{(n)}=\operatorname{Pr}\left(g\left(f_{1}\left(X^{n}\right), f_{2}\left(Y^{n}\right)\right) \neq\left(X^{n}, Y^{n}\right)\right)$