本节内容摘自Steven M. Kay,《Fundamentals of Statistical Signal Processing: Estimation Theory》。
【定理10.2】多元高斯向量的条件PDF
如果
x
∈
R
k
×
1
{\bf x}\in \mathbb{R}^{k\times 1}
x∈Rk×1和
y
∈
R
l
×
1
{\bf y}\in \mathbb{R}^{l\times 1}
y∈Rl×1为联合高斯分布随机向量,均值向量为
[
E
(
x
)
E
(
x
)
]
T
[{\rm E}({\bf x})\ {\rm E}({\bf x})]^{\rm T}
[E(x) E(x)]T,分块协方差矩阵为
C
=
[
C
x
x
C
x
y
C
y
x
C
y
y
]
,
(10.23)
\tag{10.23} {\bf C}= \left[ \begin{matrix} {\bf C}_{xx} & {\bf C}_{xy} \\{\bf C}_{yx}& {\bf C}_{yy}\end{matrix} \right],
C=[CxxCyxCxyCyy],(10.23)其中
C
x
x
∈
R
k
×
k
{\bf C}_{xx}\in \mathbb{R}^{k\times k}
Cxx∈Rk×k与
C
y
y
∈
R
l
×
l
{\bf C}_{yy}\in \mathbb{R}^{l\times l}
Cyy∈Rl×l分别为向量
x
\bf x
x和
y
\bf y
y的自协方差阵,
C
x
y
∈
R
k
×
l
{\bf C}_{xy}\in \mathbb{R}^{k\times l}
Cxy∈Rk×l与
C
y
x
∈
R
l
×
k
{\bf C}_{yx}\in \mathbb{R}^{l\times k}
Cyx∈Rl×k为向量
x
\bf x
x和
y
\bf y
y的互协方差阵。显然,我们可以得到
x
\bf x
x和
y
\bf y
y的联合概率密度函数为
p
(
x
,
y
)
=
1
(
2
π
)
k
+
l
2
d
e
t
1
2
(
C
)
e
x
p
[
−
1
2
(
[
x
−
E
(
x
)
y
−
E
(
y
)
]
)
T
C
−
1
(
[
x
−
E
(
x
)
y
−
E
(
y
)
]
)
]
,
p({\bf x},{\bf y})=\frac{1}{(2\pi)^{\frac{k+l}{2}}{\rm det}^{\frac{1}{2}}(\bf C)}{\rm exp}\left[-\frac{1}{2}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)^{\rm T}{\bf C}^{-1}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)\right],
p(x,y)=(2π)2k+ldet21(C)1exp[−21([x−E(x)y−E(y)])TC−1([x−E(x)y−E(y)])],因此,条件PDF
p
(
y
∣
x
)
p(\bf y|x)
p(y∣x)也为高斯的,且
E
(
y
∣
x
)
=
E
(
y
)
+
C
y
x
C
x
x
−
1
(
x
−
E
(
x
)
)
(10.24)
\tag{10.24} {\rm E}({\bf y|x})={\rm E}({\bf y})+{\bf C}_{yx}{\bf C}^{-1}_{xx}(x-{\rm E}({\bf x}))
E(y∣x)=E(y)+CyxCxx−1(x−E(x))(10.24)
C
y
∣
x
=
C
y
y
−
C
y
x
C
x
x
−
1
C
x
y
.
(10.25)
\tag{10.25} {\bf C}_{y|x}={\bf C}_{yy}-{\bf C}_{yx}{\bf C}^{-1}_{xx}{\bf C}_{xy}.
Cy∣x=Cyy−CyxCxx−1Cxy.(10.25)注意,条件PDF的协方差矩阵并不依赖于
x
\bf x
x,尽管这个属性通常并不成立。
【附录10A】条件高斯PDF的推导
我们来推导【定理10.2】的结论。显然,我们有
p
(
y
∣
x
)
=
p
(
x
,
y
)
p
(
x
)
=
1
(
2
π
)
k
+
l
2
d
e
t
1
2
(
C
)
e
x
p
[
−
1
2
(
[
x
−
E
(
x
)
y
−
E
(
y
)
]
)
T
C
−
1
(
[
x
−
E
(
x
)
y
−
E
(
y
)
]
)
]
1
(
2
π
)
k
2
det
1
2
(
C
x
x
)
exp
[
−
1
2
(
x
−
E
(
x
)
)
T
C
x
x
−
1
(
x
−
E
(
x
)
)
]
.
\begin{aligned} p({\bf y}|{\bf x})&=\frac{p({\bf x,y})}{p({\bf x})}\\ &=\frac{\frac{1}{(2\pi)^{\frac{k+l}{2}}{\rm det}^{\frac{1}{2}}(\bf C)}{\rm exp}\left[-\frac{1}{2}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)^{\rm T}{\bf C}^{-1}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)\right]}{\frac{1}{(2\pi)^{\frac{k}{2}}\det^{\frac{1}{2}}({\bf C}_{xx})}\exp[-\frac{1}{2}({\bf x}-{\rm E}({\bf x}))^{\rm T}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x}))]}. \end{aligned}
p(y∣x)=p(x)p(x,y)=(2π)2kdet21(Cxx)1exp[−21(x−E(x))TCxx−1(x−E(x))](2π)2k+ldet21(C)1exp[−21([x−E(x)y−E(y)])TC−1([x−E(x)y−E(y)])].令
x
=
[
I
0
0
0
]
[
x
y
]
.
{\bf x}=\left[ \begin{matrix}{\bf I} & {\bf 0}\\{\bf 0}&{\bf 0}\end{matrix}\right]\left[ \begin{matrix}{\bf x}\\{\bf y}\end{matrix}\right].
x=[I000][xy].下面我们来看分块协方差阵。由于
det
(
[
A
11
A
12
A
21
A
22
]
)
=
det
(
A
11
)
det
(
A
22
−
A
21
A
11
−
1
A
12
)
,
\det\left(\left[ \begin{matrix} {\bf A}_{11} & {\bf A}_{12}\\ {\bf A}_{21} & {\bf A}_{22} \end{matrix} \right]\right)=\det({\bf A}_{11})\det({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11} ^{-1}{\bf A}_{12} ) ,
det([A11A21A12A22])=det(A11)det(A22−A21A11−1A12),可以得到
det
C
=
det
(
C
x
x
)
det
(
C
y
y
−
C
y
x
C
x
x
−
1
C
x
y
)
,
\det {\bf C} =\det({\bf C}_{xx})\det({\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx} ^{-1}{\bf C}_{xy}),
detC=det(Cxx)det(Cyy−CyxCxx−1Cxy),因此,有
det
C
det
(
C
x
x
)
det
(
C
y
y
−
C
y
x
C
x
x
−
1
C
x
y
)
.
\frac{\det {\bf C}}{\det({\bf C}_{xx})}\det({\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx} ^{-1}{\bf C}_{xy}).
det(Cxx)detCdet(Cyy−CyxCxx−1Cxy).如果令
Q
=
[
x
−
E
(
x
)
y
−
E
(
y
)
]
T
C
−
1
[
x
−
E
(
x
)
y
−
E
(
y
)
]
−
(
x
−
E
(
x
)
)
T
C
x
x
−
1
(
x
−
E
(
x
)
)
,
Q=\left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]^{\rm T}{\bf C}^{-1} \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]-({\bf x}-{\rm E}({\bf x}))^{\rm T}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x})),
Q=[x−E(x)y−E(y)]TC−1[x−E(x)y−E(y)]−(x−E(x))TCxx−1(x−E(x)),我们可以得到
p
(
y
∣
x
)
=
1
(
2
π
)
l
2
det
1
2
(
C
y
y
−
C
y
x
C
x
x
−
1
C
x
y
)
exp
(
−
1
2
Q
)
.
\begin{aligned} p({\bf y}|{\bf x})=\frac{1}{{(2\pi)^{\frac{l}{2}}\det^{\frac{1}{2}}({\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx} ^{-1}{\bf C}_{xy})}}\exp\left(-\frac{1}{2}Q\right). \end{aligned}
p(y∣x)=(2π)2ldet21(Cyy−CyxCxx−1Cxy)1exp(−21Q).下面我们来求
C
−
1
{\bf C}^{-1}
C−1,从而得到
Q
Q
Q。由于对称分块矩阵的逆矩阵有
[
A
11
A
12
A
21
A
22
]
−
1
=
[
(
A
11
−
A
12
A
22
−
1
A
21
)
−
1
−
A
11
−
1
A
12
(
A
22
−
A
21
A
11
−
1
A
12
)
−
1
−
(
A
22
−
A
21
A
11
−
1
A
12
)
−
1
A
21
A
11
−
1
(
A
22
−
A
21
A
11
−
1
A
12
)
−
1
]
.
\left[ \begin{matrix} {\bf A}_{11} & {\bf A}_{12}\\ {\bf A}_{21} & {\bf A}_{22} \end{matrix} \right]^{-1}=\left[ \begin{matrix} ({\bf A}_{11}-{\bf A}_{12}{\bf A}_{22}^{-1}{\bf A}_{21})^{-1} & -{\bf A}_{11}^{-1}{\bf A}_{12}({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1}\\ -({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1}{\bf A}_{21}{\bf A}_{11}^{-1} & ({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1} \end{matrix} \right].
[A11A21A12A22]−1=[(A11−A12A22−1A21)−1−(A22−A21A11−1A12)−1A21A11−1−A11−1A12(A22−A21A11−1A12)−1(A22−A21A11−1A12)−1].采用这种形式,非对角线元素互为转置,因此逆矩阵是对称的。这是由于
C
\bf C
C为对称的,因此
C
−
1
{\bf C}^{-1}
C−1也是对称的。根据逆矩阵性质,我们有
(
A
11
−
A
12
A
22
−
1
A
21
)
−
1
=
A
11
−
1
+
A
11
−
1
A
12
(
A
22
−
A
21
A
11
−
1
A
12
)
−
1
A
21
A
11
−
1
,
({\bf A}_{11}-{\bf A}_{12}{\bf A}_{22}^{-1}{\bf A}_{21})^{-1}={\bf A}_{11}^{-1}+{\bf A}_{11}^{-1}{\bf A}_{12}({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1}{\bf A}_{21}{\bf A}_{11}^{-1},
(A11−A12A22−1A21)−1=A11−1+A11−1A12(A22−A21A11−1A12)−1A21A11−1,因此得到
C
−
1
=
[
C
x
x
−
1
+
C
x
x
−
1
C
x
y
B
−
1
C
y
x
C
x
x
−
1
C
x
x
−
1
C
x
y
B
−
1
−
B
−
1
C
y
x
C
x
x
−
1
B
−
1
]
,
{\bf C}^{-1}=\left[ \begin{matrix} {\bf C}_{xx}^{-1}+{\bf C}_{xx}^{-1}{\bf C}_{xy}{\bf B}^{-1}{\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf C}_{xx}^{-1}{\bf C}_{xy}{\bf B}^{-1}\\ -{\bf B}^{-1} {\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf B}^{-1}\end{matrix}\right],
C−1=[Cxx−1+Cxx−1CxyB−1CyxCxx−1−B−1CyxCxx−1Cxx−1CxyB−1B−1],其中
B
=
C
y
y
−
C
y
x
C
x
x
−
1
C
x
y
.
{\bf B}={\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\bf C}_{xy}.
B=Cyy−CyxCxx−1Cxy.进一步,有
C
−
1
=
[
I
−
C
x
x
−
1
C
x
y
0
I
]
[
C
x
x
−
1
0
0
B
−
1
]
[
I
0
−
C
y
x
C
x
x
−
1
I
]
.
{\bf C}^{-1}=\left[ \begin{matrix} {\bf I}& -{\bf C}_{xx}^{-1}{\bf C}_{xy} \\ {\bf 0} & {\bf I}\end{matrix}\right] \left[ \begin{matrix} {\bf C}_{xx}^{-1}&{\bf 0} \\ {\bf 0} & {\bf B}^{-1}\end{matrix}\right] \left[ \begin{matrix} {\bf I}& {\bf 0}\\ -{\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf I}\end{matrix}\right].
C−1=[I0−Cxx−1CxyI][Cxx−100B−1][I−CyxCxx−10I].再令
x
~
=
x
−
E
(
x
)
{\tilde {\bf x}}={\bf x}-{\rm E}({\bf x})
x~=x−E(x),
y
~
=
y
−
E
(
y
)
{\tilde {\bf y}}={\bf y}-{\rm E}({\bf y})
y~=y−E(y),我们可以得到
Q
=
[
x
~
y
~
]
T
[
I
−
C
x
x
−
1
C
x
y
0
I
]
[
C
x
x
−
1
0
0
B
−
1
]
[
I
0
−
C
y
x
C
x
x
−
1
I
]
[
x
~
y
~
]
−
x
~
T
C
x
x
−
1
x
~
=
[
x
~
y
~
−
C
y
x
C
x
x
−
1
x
~
]
T
[
C
x
x
−
1
0
0
B
−
1
]
[
x
~
y
~
−
C
y
x
C
x
x
−
1
x
~
]
T
−
x
~
T
C
x
x
−
1
x
~
=
(
y
~
−
C
y
x
C
x
x
−
1
x
~
)
T
B
−
1
(
y
~
−
C
y
x
C
x
x
−
1
x
~
)
\begin{aligned} Q&=\left[ \begin{matrix} {\tilde {\bf x}}\\{\tilde {\bf y}} \end{matrix}\right] ^{\rm T} \left[ \begin{matrix} {\bf I}& -{\bf C}_{xx}^{-1}{\bf C}_{xy} \\ {\bf 0} & {\bf I}\end{matrix}\right] \left[ \begin{matrix} {\bf C}_{xx}^{-1}&{\bf 0} \\ {\bf 0} & {\bf B}^{-1}\end{matrix}\right] \left[ \begin{matrix} {\bf I}& {\bf 0}\\ -{\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf I}\end{matrix}\right]\left[ \begin{matrix} {\tilde {\bf x}}\\{\tilde {\bf y}} \end{matrix}\right] -{\tilde {\bf x}}^{\rm T}{\bf C}_{xx}^{-1}{\tilde {\bf x}}\\ &=\left[ \begin{matrix} {\tilde {\bf x}}\\ {\tilde {\bf y}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}}\end{matrix}\right]^{\rm T} \left[ \begin{matrix} {\bf C}_{xx}^{-1}&{\bf 0} \\ {\bf 0} & {\bf B}^{-1}\end{matrix}\right] \left[ \begin{matrix} {\tilde {\bf x}}\\ {\tilde {\bf y}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}}\end{matrix}\right]^{\rm T} -{\tilde {\bf x}}^{\rm T}{\bf C}_{xx}^{-1}{\tilde {\bf x}}\\ &=({\tilde {\bf y}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}})^{\rm T}{\bf B}^{-1}{(\tilde {\bf y}}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}) \end{aligned}
Q=[x~y~]T[I0−Cxx−1CxyI][Cxx−100B−1][I−CyxCxx−10I][x~y~]−x~TCxx−1x~=[x~y~−CyxCxx−1x~]T[Cxx−100B−1][x~y~−CyxCxx−1x~]T−x~TCxx−1x~=(y~−CyxCxx−1x~)TB−1(y~−CyxCxx−1x~)最终,得到
Q
=
[
y
−
(
E
(
y
)
+
C
y
x
C
x
x
−
1
(
x
−
E
(
x
)
)
)
]
T
[
C
y
y
−
C
y
x
C
x
x
−
1
C
x
y
]
−
1
[
y
−
(
E
(
y
)
+
C
y
x
C
x
x
−
1
(
x
−
E
(
x
)
)
)
]
.
Q=[{\bf y}-({\rm E}({\bf y})+{\bf C}_{yx}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x})))]^{\rm T} [{\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\bf C}_{xy}]^{-1} [{\bf y}-({\rm E}({\bf y})+{\bf C}_{yx}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x})))].
Q=[y−(E(y)+CyxCxx−1(x−E(x)))]T[Cyy−CyxCxx−1Cxy]−1[y−(E(y)+CyxCxx−1(x−E(x)))].因此均值和方差分别如(10.24)和(10.25)所示。