Huffman Codes and Data Compression

哈夫曼编码让不同字符之间代码长度不同，保证经常出现的字符的代码要短。

Prefix Code

浅层定义：没有字符代码是别的字符代码的前缀。

定义：for a set $S$ of letters, a prefix code is a function $\gamma$ that maps each letter $x\in S$ to some sequence of zeros and ones, in such way that for distinct $x,y\in S$, the sequence $\gamma (x)$ is not a prefix of the sequence $\gamma (y)$.

Optimal Prefix Codes

假设对于任意一个字母 $x\in S$, $f_x$ 代表字母 $x$ 出现的频率。

• 假设总共有 $n$ 个字母，其中 $n{f}_x$ 个字母是 $x$.
• ${\sum }_{x\in S}{f}_x=1$

用 $|\gamma (x)|$ 表示 $\gamma (x)$ 的长度，那么
$\text{encoding length}={\sum }_{x\in S}n{f}_x|\gamma (x)|=n{\sum }_{x\in S}{f}_x|\gamma (x)|$

定义： the average number of bits required per letter $ABL(\gamma )={\sum }_{x\in S}{f}_x|\gamma (x)|$

目标：minimize $ABL(\gamma )$

算法设计

A binary tree $T$ with the number of leaves equal to the size of the alphabet $S$, and we label each leaf with a distinct letter in $S$.

定理 (4.27): The encoding of $S$ constructed from $T$ is a prefix code.
证明：（反证法）
若 $\gamma (x)$ 是 $\gamma (y)$ 的 prefix，the path from the root to $x$ would have to be a prefix of the path from
the root to $y$
但是，这等同于说 $x$ would lie on the path from root to $y$
与 $x$ 是 leaf 矛盾

$ABL(T)={\sum }_{x\in S}{f}_x\cdot {\text{depth}}_T(x)$

定理 (4.28): The binary tree corresponding to the optimal prefix code is full.
证明：（exchange argument）
Let $T$ denote the binary tree corresponding to the optimal prefix code, and suppose it contains
a node $u$ with exactly one child $v$.
\quad
Convert $T$ into a tree $T^{\prime }$ by replacing node $u$ with $v$.
分两类情况讨论：
1.$u$ 是 root: 删除 $u$, 并将 $v$ 变成 root。
2.$u$ 不是 root:
\quad 令 $w$ 为 $u$ 的 parent in $T$.
\quad 删除 $u$，并让 $v$ 成为 $w$ 的 child.
\quad 这个改变 decrease the number of bits needed to encode any leaf in the subtree rooted at node $u$,
\quad and it does not affect other leaves
$\therefore ABL(T^{\prime })<ABL(T)$
\quad 与 optimality of $T$ 矛盾

定理 (4.29): 假设已知 optimal binary tree $T^{\ast }$。Suppose that $u$ and $v$ are lives of $T^{\ast }$, such that $depth(u)<depth(v)$. Further, suppose that in a labeling of $T^{\ast }$ corresponding to an optimal prefix code, leaf $u$ is labeled with $y\in S$ and leaf $v$ is labeled with $z\in S$. Then, $f_y\ge f_z$.
证明：用 exchange argument 证明，交换 $u,v$ 的 label。

定理 (4.30): Consider a leaf $v$ in $T^{\ast }$ with largest depth. Leaf $v$ has a parent $u$, and by (4.28) $T^{\ast }$ is a full binary tree, so $u$ has another child $w$. Then, $w$ is a leaf of $T^{\ast }$. 反证法

定理 (4.31): There is an optimal prefix code, with corresponding tree $T^{\ast }$, in which the two lowest-frequency letters are assigned to leaves that are siblings in $T^{\ast }$.

算法伪代码

To construct a prefix code for an alphabet $S$, with given frequencies:

1. If $S$ has two letters then
2. \quad Encode one letter using 0 and the other letter using 1
3. Else
4. \quad Let $y^{\ast }$ and $z^{\ast }$ be the two lowest-frequency letters
5. \quad Form a new alphabet $S^{\prime }$ by deleting $y\ast$ and $z^{\ast }$ and replacing them with a new letter $w$ of frequency $f_{y^{\ast }}+f_{z^{\ast }}$
6. \quad Recursively construct a prefix code ${\gamma }^{\prime }$ for $S^{\prime }$, with tree $T^{\prime }$
7. \quad Define a prefix code for $S$ as follows:
8. \quad \quad Start with $T^{\prime }$
9. \quad \quad Take the leaf labeled $w$ and add two children below it, labeled $y^{\ast }$ and $z^{\ast }$
10. EndIf