1.简要介绍
对于PCM量化在平稳随机中的应用具有无限振幅区间的过程,我们选择了量化步长为给定量化器大小K的最小值失真。这个概念的自然延伸是在给定K的时候,最小化关于标量量化器的所有参数的失真优化变量。参数变量为K-1个边界ui,K个映射值
s
i
′
s_i^{'}
si′,
0
≤
i
<
K
0\le i \lt K
0≤i<K。得到的量化器称为pdf优化标量固定长度编码的量化器。
通常,为量子化定义一个失真度量是否恰当的标准,比如量化的统计错误。我们通过选择一个恰当的边界值
u
i
u_i
ui与映射值
s
i
′
s_i^{'}
si′可以使得损失最小。如果我们定义失真
D
D
D为
f
(
ϵ
)
f(\epsilon)
f(ϵ)的期望值,其中f是某个函数(可微),
ϵ
\epsilon
ϵ是量化误差,输入的概率密度函数为
p
(
x
)
p(x)
p(x)。此时,我们定义distortion D:
D
=
E
[
f
(
s
i
n
−
s
o
u
t
)
]
=
∑
i
=
1
N
∫
−
u
i
u
i
+
1
f
(
s
−
s
i
′
)
p
(
x
)
d
x
\begin{aligned} D & = E[f(s_{in}-s_{out})] \\ & = \sum_{i=1}^{N} \int_{-u_i}^{u_{i+1}} f(s-s_i^{'})\ p(x)\ dx \\ \end{aligned}
D=E[f(sin−sout)]=i=1∑N∫−uiui+1f(s−si′) p(x) dx
x
N
+
1
=
∞
,
x
1
=
−
∞
x_{N+1}=\infty,x_1=-\infty
xN+1=∞,x1=−∞
2.证明
如果我们想在N固定的情况下最小化D,我们需要D分别对
u
i
u_i
ui和
s
i
′
s_i^{'}
si′求偏导:
∂
D
∂
u
i
=
f
(
u
i
−
s
i
−
1
′
)
p
(
x
)
−
f
(
u
i
−
s
i
′
)
p
(
u
i
)
=
0
i
=
2
,
.
.
.
,
N
(
1
)
\begin{aligned} \frac{\partial D}{\partial u_i}=f(u_i -s_{i-1}^{'})p(x)-f(u_i-s_i^{'})p(u_i)=0 \\ i=2,...,N\ \ \ \ \ \ \ (1) \end{aligned}
∂ui∂D=f(ui−si−1′)p(x)−f(ui−si′)p(ui)=0i=2,...,N (1)
∂
D
∂
s
i
′
=
−
∫
u
i
u
i
+
1
f
′
(
s
−
s
i
′
)
p
(
x
)
d
x
=
0
i
=
1
,
.
.
.
,
N
(
2
)
\begin{aligned} \frac{\partial D}{\partial s_i^{'}} = -\int_{u_i}^{u_{i+1}} f^{'}(s-s_i^{'})\ p(x)\ dx=0 \\ i=1,...,N\ \ \ \ \ \ \ (2) \end{aligned}
∂si′∂D=−∫uiui+1f′(s−si′) p(x) dx=0i=1,...,N (2)
(
1
)
b
e
c
o
m
e
s
(
f
o
r
p
(
x
i
)
≠
0
)
f
(
u
i
−
s
i
−
1
′
)
=
f
(
u
i
−
s
i
′
)
i
=
2
,
.
.
.
,
N
(
3
)
\begin{aligned} (1)becomes (for\ p(x_i) \neq 0) \\ & f(u_i -s_{i-1}^{'})=f(u_i-s_i^{'}) & &i=2,...,N\ \ \ \ \ \ \ (3) \end{aligned}
(1)becomes(for p(xi)=0)f(ui−si−1′)=f(ui−si′)i=2,...,N (3)
( 2 ) b e c o m e s ∫ u i u i + 1 f ′ ( s − s i ′ ) p ( x ) d x = 0 i = 2 , . . . , N ( 4 ) \begin{aligned} (2)becomes \\ &&&&&&&&& \int_{u_i}^{u_{i+1}} f^{'}(s-s_i^{'})\ p(x)\ dx=0 & &i=2,...,N\ \ \ \ \ \ \ (4) \end{aligned} (2)becomes∫uiui+1f′(s−si′) p(x) dx=0i=2,...,N (4)
我们可以在详细的计算以下:
2.1 求 s i ′ s_i^{'} si′
D = E [ f ( s i n − s o u t ) ] = ∑ i = 1 N ∫ − u i u i + 1 f ( s − s i ′ ) p ( x ) d x \begin{aligned} D & = E[f(s_{in}-s_{out})] \\ & = \sum_{i=1}^{N} \int_{-u_i}^{u_{i+1}} f(s-s_i^{'})\ p(x)\ dx \\ \end{aligned} D=E[f(sin−sout)]=i=1∑N∫−uiui+1f(s−si′) p(x) dx
函数
f
(
s
)
f(s)
f(s)我们采用MSE。
方法一:
E
{
(
S
−
s
i
′
)
2
}
=
E
{
(
S
−
E
(
s
)
+
E
(
s
)
−
s
i
′
)
2
}
=
E
{
(
S
−
E
(
s
)
)
)
2
}
+
(
E
(
s
)
−
s
i
′
)
2
≥
E
{
(
S
−
E
(
s
)
)
)
2
}
\begin{aligned} & E \{ (S-s_i{'})^2 \} \\ & =E\{(S-E(s)+E(s)-s_i{'})^2\} \\ & =E\{(S-E(s)))^2\}+(E(s)-s_i{'})^2 \\ & \ge E\{(S-E(s)))^2\} \end{aligned}
E{(S−si′)2}=E{(S−E(s)+E(s)−si′)2}=E{(S−E(s)))2}+(E(s)−si′)2≥E{(S−E(s)))2}
当等号成立时,
E
{
S
}
=
=
s
i
′
E\{S\} == s_i^{'}
E{S}==si′,即:
s
i
′
=
E
{
S
}
=
∫
u
i
u
i
+
1
s
f
(
s
)
d
s
∫
u
i
u
i
+
1
f
(
s
)
d
s
s_i^{'}=E\{S\}=\frac{\int_{u_i}^{u_{i+1}}sf(s)\ ds}{\int_{u_i}^{u_{i+1}}f(s)\ ds}
si′=E{S}=∫uiui+1f(s) ds∫uiui+1sf(s) ds
方法二:
D
=
E
{
f
(
S
,
s
i
′
)
}
=
∫
u
i
u
i
+
1
(
s
−
s
i
′
)
2
f
(
s
)
d
s
=
∫
u
i
u
i
+
1
s
2
f
(
s
)
d
s
‾
−
∫
u
i
u
i
+
1
2
s
s
i
′
f
(
s
)
d
s
+
∫
u
i
u
i
+
1
s
i
′
2
f
(
s
)
d
s
‾
\begin{aligned} D&= E\{f(S,s_i^{'})\}=\int_{u_i}^{u_{i+1}}(s-s_i^{'})^2f(s)\ ds \\ & = \underline{\int_{u_i}^{u_{i+1}}s^2f(s)\ ds}-\underline{\int_{u_i}^{u_{i+1}}2ss_i^{'}f(s)\ ds+\int_{u_i}^{u_{i+1}}s_i^{'2}f(s)\ ds } \end{aligned}
D=E{f(S,si′)}=∫uiui+1(s−si′)2f(s) ds=∫uiui+1s2f(s) ds−∫uiui+12ssi′f(s) ds+∫uiui+1si′2f(s) ds
上述第一项为定值,所需对于D最小化来说,我们需要将第二项最小化,即:
M
i
n
s
i
′
∫
u
i
u
i
+
1
s
i
′
2
f
(
s
)
d
s
−
∫
u
i
u
i
+
1
2
s
i
′
s
f
(
s
)
d
s
Min_{s_i^{'}}\ \int_{u_i}^{u_{i+1}}s_i^{'2}f(s)\ ds-\int_{u_i}^{u_{i+1}}2s_i^{'}sf(s)\ ds
Minsi′ ∫uiui+1si′2f(s) ds−∫uiui+12si′sf(s) ds
为了方便,我们设:
x
=
∫
u
i
u
i
+
1
f
(
s
)
d
s
,
y
=
∫
u
i
u
i
+
1
s
f
(
s
)
d
s
x=\int_{u_i}^{u_{i+1}}f(s)\ ds, y=\int_{u_i}^{u_{i+1}}sf(s)\ ds
x=∫uiui+1f(s) ds,y=∫uiui+1sf(s) ds
D = x s i ′ 2 − 2 y s i ′ D=xs_i^{'2}-2ys_i^{'} D=xsi′2−2ysi′
∂
D
∂
s
i
′
=
2
x
s
i
′
−
2
y
=
0
\frac{\partial D}{\partial s_i^{'}}=2xs_i^{'}-2y=0
∂si′∂D=2xsi′−2y=0
故:
s
i
′
=
y
x
s
i
′
=
E
{
S
}
=
∫
u
i
u
i
+
1
s
f
(
s
)
d
s
∫
u
i
u
i
+
1
f
(
s
)
d
s
\begin{aligned} & s_i^{'}=\frac{y}{x} \\ &s_i^{'}=E\{S\}=\frac{\int_{u_i}^{u_{i+1}}sf(s)\ ds}{\int_{u_i}^{u_{i+1}}f(s)\ ds} \end{aligned}
si′=xysi′=E{S}=∫uiui+1f(s) ds∫uiui+1sf(s) ds
2.2 求 u i u_i ui
我们假设levels
s
i
′
s_i^{'}
si′已知。F为MSE
D
=
E
[
f
(
s
i
n
−
s
o
u
t
)
]
=
∑
i
=
1
N
∫
u
i
u
i
+
1
F
(
s
−
s
i
′
)
f
(
x
)
d
x
\begin{aligned} D & = E[f(s_{in}-s_{out})] \\ & = \sum_{i=1}^{N} \int_{u_i}^{u_{i+1}} F(s-s_i^{'})\ f(x)\ dx \\ \end{aligned}
D=E[f(sin−sout)]=i=1∑N∫uiui+1F(s−si′) f(x) dx
∂ D ∂ u i = ∂ ∂ u i { 无 关 项 + ∫ u i − 1 u i ( s − s i − 1 ′ ) 2 f ( s ) d s + ∫ u i u i + 1 ( s − s i ′ ) 2 f ( s ) d s } = ∂ ∂ u i { 无 关 项 + ∫ u i − 1 u i s 2 f ( s ) d s − ∫ u i − 1 u i 2 s s i − 1 ′ f ( s ) d s + ∫ u i − 1 u i s i − 1 ′ 2 f ( s ) d s + ∫ u i u i + 1 s 2 f ( s ) d s − ∫ u i u i + 1 2 s s i ′ f ( s ) d s + ∫ u i u i + 1 s i ′ 2 f ( s ) d s } = ∂ ∂ u i { 无 关 项 − ∫ u i − 1 u i 2 s s i − 1 ′ f ( s ) d s + ∫ u i − 1 u i s i − 1 ′ 2 f ( s ) d s − ∫ u i u i + 1 2 s s i ′ f ( s ) d s + ∫ u i u i + 1 s i ′ 2 f ( s ) d s } = ∂ ∂ u i { 无 关 项 − ∫ u i − 1 u i + 1 2 s s i − 1 ′ f ( s ) d s + ∫ u i − 1 u i + 1 s i − 1 ′ 2 f ( s ) d s − ∫ u i u i + 1 2 s ( s i ′ − s i − 1 ′ ) f ( s ) d s + ∫ u i u i + 1 ( s i ′ 2 − s i − 1 ′ 2 ) f ( s ) d s } = ∂ ∂ u i { 无 关 项 − ( s i ′ − s i − 1 ′ ) F 1 ( s ) ∣ u i u i + 1 + ( s i ′ 2 − s i − 1 ′ 2 ) F 2 ( s ) ∣ u i u i + 1 = 2 ( s i ′ − s i − 1 ′ ) u i f ( u i ) − ( s i ′ 2 − s i − 1 ′ 2 ) f ( u i ) = ( s i ′ − s i − 1 ′ ) f ( u i ) { 2 u i − ( s i ′ + s i − 1 ′ ) } = 0 = = > u i = s i ′ + s i − 1 ′ 2 \begin{aligned} \frac{\partial D}{\partial u_i}&=\frac{\partial }{\partial u_i}\{无关项+\int_{u_i-1}^{u_{i}}(s-s_{i-1}^{'})^2f(s)\ ds+\int_{u_i}^{u_{i+1}}(s-s_i^{'})^2f(s)\ ds \} \\ = \frac{\partial }{\partial u_i}\{无关项&+\int_{u_i-1}^{u_{i}}s^2f(s)\ ds -\int_{u_i-1}^{u_{i}}2ss_{i-1}^{'}f(s)\ ds +\int_{u_i-1}^{u_{i}}s_{i-1}^{'2}f(s)\ ds \\ &+\int_{u_i}^{u_{i+1}}s^2f(s)\ ds -\int_{u_i}^{u_{i+1}}2ss_{i}^{'}f(s)\ ds +\int_{u_i}^{u_{i+1}}s_{i}^{'2}f(s)\ ds \} \\ =\frac{\partial }{\partial u_i}\{无关项&-\int_{u_i-1}^{u_{i}}2ss_{i-1}^{'}f(s)\ ds +\int_{u_i-1}^{u_{i}}s_{i-1}^{'2}f(s)\ ds \\ &-\int_{u_i}^{u_{i+1}}2ss_{i}^{'}f(s)\ ds +\int_{u_i}^{u_{i+1}}s_{i}^{'2}f(s)\ ds \} \\ =\frac{\partial }{\partial u_i}\{无关项&-\int_{u_i-1}^{u_{i+1}}2ss_{i-1}^{'}f(s)\ ds +\int_{u_i-1}^{u_{i+1}}s_{i-1}^{'2}f(s)\ ds \\ &-\int_{u_i}^{u_{i+1}}2s(s_{i}^{'}-s_{i-1}^{'})f(s)\ ds +\int_{u_i}^{u_{i+1}}(s_{i}^{'2}-s_{i-1}^{'2})f(s)\ ds \} \\ &=\frac{\partial }{\partial u_i}\{无关项-(s_{i}^{'}-s_{i-1}^{'})F_1(s)|_{u_i}^{u_{i+1}}+(s_{i}^{'2}-s_{i-1}^{'2})F_2(s)|_{u_i}^{u_{i+1}} \\ &=2(s_{i}^{'}-s_{i-1}^{'})u_if(u_i)-(s_{i}^{'2}-s_{i-1}^{'2})f(u_i) \\ &=(s_{i}^{'}-s_{i-1}^{'})f(u_i)\{2u_i-(s_{i}^{'}+s_{i-1}^{'})\}=0 \\ &==>u_i=\frac{s_{i}^{'}+s_{i-1}^{'}}{2} \end{aligned} ∂ui∂D=∂ui∂{无关项=∂ui∂{无关项=∂ui∂{无关项=∂ui∂{无关项+∫ui−1ui(s−si−1′)2f(s) ds+∫uiui+1(s−si′)2f(s) ds}+∫ui−1uis2f(s) ds−∫ui−1ui2ssi−1′f(s) ds+∫ui−1uisi−1′2f(s) ds+∫uiui+1s2f(s) ds−∫uiui+12ssi′f(s) ds+∫uiui+1si′2f(s) ds}−∫ui−1ui2ssi−1′f(s) ds+∫ui−1uisi−1′2f(s) ds−∫uiui+12ssi′f(s) ds+∫uiui+1si′2f(s) ds}−∫ui−1ui+12ssi−1′f(s) ds+∫ui−1ui+1si−1′2f(s) ds−∫uiui+12s(si′−si−1′)f(s) ds+∫uiui+1(si′2−si−1′2)f(s) ds}=∂ui∂{无关项−(si′−si−1′)F1(s)∣uiui+1+(si′2−si−1′2)F2(s)∣uiui+1=2(si′−si−1′)uif(ui)−(si′2−si−1′2)f(ui)=(si′−si−1′)f(ui){2ui−(si′+si−1′)}=0==>ui=2si′+si−1′
3.例子
作为第一个例子,我们将使用超过10000个样本的和MSE作为失真测量应用到具有单位方差的高斯pdf中。我们对重构级别使用了两种不同的初始化,具体情况请见下图。当两次迭代步骤间的相对失真减少率为小于1%时,判定其收敛性, ( D k − D k + 1 ) / D k + 1 < 0.01 (D_k-D_{k+1})/D_{k+1}<0.01 (Dk−Dk+1)/Dk+1<0.01。
对于两种初始化方式,算法在6次迭代后迅速收敛,到相同的整体失真
D
F
∗
D_F^{*}
DF∗。
相关的迭代过程图:
完结撒花,不得不说,手敲这些公式真心累~!!!!
本文与博客园同步更新!!!!
Reference: [1]Quantizing for Minimum Distortion*
[2]Source coding