1.证明 ∫ − ∞ ∞ 1 2 π σ 2 e − ( x − u ) 2 2 σ 2 d x = 1 \int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{(x-u)^ 2}{2 \sigma^{2}}} d x=1 ∫−∞∞2πσ21e−2σ2(x−u)2dx=1
证:
令 x − u 2 σ = t , t ∈ ( − ∞ , ∞ ) \frac{x-u}{\sqrt{2} \sigma}=t, t \in(-\infty, \infty) 2σx−u=t,t∈(−∞,∞)
故 x = 2 σ t + u x=\sqrt{2} \sigma t+u x=2σt+u,有
∫ − ∞ ∞ 1 2 π σ 2 e − ( x − u ) 2 2 σ 2 d x = ∫ − ∞ ∞ 1 2 π σ 2 e − t 2 d ( 2 σ t + u ) = ∫ − ∞ ∞ 1 π e − t 2 d t \int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{(x-u)^{2}}{2 \sigma^{2}}} d x=\int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-t^{2}} d(\sqrt{2} \sigma t+u)=\int_{-\infty}^{\infty} \frac{1}{\sqrt{\pi}} e^{-t^{2}} d t ∫−∞∞2πσ21e−2σ2(x−u)2dx=∫−∞∞2πσ21e−t2d(2σt+u)=∫−∞∞π1e−t2dt
由 ∫ − ∞ ∞ e − t 2 d t = π \int_{-\infty}^{\infty} e^{-t^{2}} d t=\sqrt{\pi} ∫−∞∞e−t2dt=π可知
∫ − ∞ ∞ 1 π e − t 2 d t = 1 \int_{-\infty}^{\infty} \frac{1}{\sqrt{\pi}} e^{-t^{2}} d t=1 ∫−∞∞π1e−t2dt=1
故
∫ − ∞ ∞ 1 2 π σ 2 e − ( x − μ ) 2 2 σ 2 d x = 1 \int_{-\infty}^{\infty} \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{(x-\mu)^{2}}{2 \sigma^{2}}} d x=1 ∫−∞∞2πσ21e−2σ2(x−μ)2dx=1
2.(第二章第1题) μ , v \pmb \mu, \pmb v μμμ,vvv为维度相同的列向量,证明
u ⊤ v = tr ( v u ⊤ ) \pmb u^{\top} \pmb v=\operatorname{tr}\left(\pmb v \pmb u^{\top}\right) uuu⊤vvv=tr(vvvuuu⊤)
证:
令
μ
=
[
μ
1
,
⋯
,
μ
M
]
T
\pmb \mu=[ \mu_1, \cdots, \mu_M]^T
μμμ=[μ1,⋯,μM]T,
v
=
[
v
1
,
⋯
,
v
M
]
T
\pmb v=[ v_1, \cdots, v_M]^T
vvv=[v1,⋯,vM]T,则
v
u
⊤
=
[
v
1
⋮
v
m
]
[
u
1
⋯
⋅
u
m
]
=
[
v
1
u
1
⋯
⋯
⋯
v
1
u
m
⋮
⋱
⋮
⋮
v
k
u
k
⋮
⋮
⋱
⋮
v
m
u
1
⋯
⋯
⋯
v
m
u
m
]
\pmb v \pmb u^{\top}=\left[\begin{array}{c} v_{1} \\ \vdots \\ v_{m} \end{array}\right]\left[u_{1} \cdots \cdot u_{m}\right]=\left[\begin{array}{cccc} v_{1} u_{1} & \cdots & \cdots&\cdots & v_{1}u_{m} \\ \vdots & \ddots & & &\vdots& \\ \vdots & & v_{k} u_{k}&&\vdots&\\ \vdots & & &\ddots&\vdots \\ v_{m} u_{1} & \cdots &\cdots & \cdots & v_{m}u_m \end{array}\right]
vvvuuu⊤=⎣⎢⎡v1⋮vm⎦⎥⎤[u1⋯⋅um]=⎣⎢⎢⎢⎢⎢⎢⎢⎡v1u1⋮⋮⋮vmu1⋯⋱⋯⋯vkuk⋯⋯⋱⋯v1um⋮⋮⋮vmum⎦⎥⎥⎥⎥⎥⎥⎥⎤
故
tr
(
v
u
⊤
)
=
∑
i
=
1
m
v
i
u
i
=
u
⊤
v
\operatorname{tr}\left(\pmb v \pmb u^{\top}\right)=\sum_{i=1}^{m} v_{i} u_{i}=\pmb u^{\top} \pmb v
tr(vvvuuu⊤)=i=1∑mviui=uuu⊤vvv
原命题得证
3.(第二章第4题)对于高斯分布随机变量, x ∼ N ( μ , Σ ) \pmb x \sim N(\pmb \mu, \pmb \Sigma) xxx∼N(μμμ,ΣΣΣ),证明
μ = E ( x ) = ∫ − ∞ ∞ x 1 ( 2 π ) N d e t ( Σ ) exp ( − 1 2 ( x − u ) T Σ − 1 ( x − u ) ) d x \pmb\mu=E(\pmb x)=\int_{-\infty}^{\infty} \pmb x \frac{1}{\sqrt{(2 \pi)^{N } det( \Sigma)}} \exp \left(-\frac{1}{2}(\pmb x-\pmb u)^{T} \Sigma^{-1}(\pmb x-\pmb u)\right) d\pmb x μμμ=E(xxx)=∫−∞∞xxx(2π)Ndet(Σ)1exp(−21(xxx−uuu)TΣ−1(xxx−uuu))dxxx
证:
因为 Σ \Sigma Σ对称半正定且 Σ \Sigma Σ可逆
故 Σ \Sigma Σ正定且可相似对角化,即存在单位正交矩阵Q使得
Σ = Q ⊤ Λ Q \Sigma=Q^{\top} \Lambda Q Σ=Q⊤ΛQ
其中 Λ = [ λ 1 ⋱ λ N ] \Lambda=\left[\begin{array}{lll} \lambda_{1} & & \\ & \ddots & \lambda_{N} \end{array}\right] Λ=[λ1⋱λN], λ i \lambda_{i} λi为 Σ \Sigma Σ特征值且 λ i > 0 \lambda_{i}>0 λi>0
故
u = E ( x ) = ∫ − ∞ ∞ x 1 ( 2 π ) N d e t ( Σ ) exp ( − 1 2 ( x − u ) ⊤ Q ⊤ Λ Q ( x − u ) ) d x \pmb u=E( \pmb x)=\int_{-\infty}^{\infty} \pmb x \frac{1}{\sqrt{(2 \pi )^{N} d e t(\Sigma)}} \exp \left(-\frac{1}{2}(\pmb x- \pmb u)^{\top} Q^{\top} \Lambda Q(\pmb x- \pmb u)\right) d \pmb x uuu=E(xxx)=∫−∞∞xxx(2π)Ndet(Σ)1exp(−21(xxx−uuu)⊤Q⊤ΛQ(xxx−uuu))dxxx
令 t = Q ( x − u ) \pmb t=Q(\pmb x- \pmb u) ttt=Q(xxx−uuu), 则 x = Q − 1 t + μ \pmb x=Q^{-1} \pmb t+ \pmb \mu xxx=Q−1ttt+μμμ
因为Q的行列式为1,
故
u
=
E
(
x
)
=
∫
−
∞
∞
Q
−
1
t
+
u
(
2
π
)
N
d
e
t
(
Σ
)
exp
(
−
1
2
t
⊤
Λ
t
)
d
t
=
∫
−
∞
∞
Q
−
1
t
(
2
π
)
N
det
(
Σ
)
exp
(
−
1
2
t
⊤
Λ
t
)
d
t
+
∫
−
∞
∞
u
(
2
π
)
N
d
e
t
(
Σ
)
exp
(
−
1
2
t
⊤
Λ
t
)
d
t
\begin{aligned} u=E(x)&=\int_{-\infty}^{\infty} \frac{Q^{-1} \pmb t+ \pmb u}{\sqrt{(2 \pi)^{N} det(\Sigma)}} \exp \left(-\frac{1}{2} \pmb t^{\top} \Lambda \pmb t\right) d \pmb t \\ &= \int_{-\infty}^{\infty} \frac{Q^{-1} \pmb t}{\sqrt{(2 \pi)^{N} \operatorname{det}(\Sigma)}} \exp \left(-\frac{1}{2} \pmb t^{\top} \Lambda \pmb t\right) d \pmb t+\int_{-\infty}^{\infty} \frac{\pmb u}{\sqrt{(2 \pi)^{N}det(\Sigma)} } \exp \left(-\frac{1}{2} \pmb t^{\top} \Lambda \pmb t\right) d \pmb t\\ \end{aligned}
u=E(x)=∫−∞∞(2π)Ndet(Σ)Q−1ttt+uuuexp(−21ttt⊤Λttt)dttt=∫−∞∞(2π)Ndet(Σ)Q−1tttexp(−21ttt⊤Λttt)dttt+∫−∞∞(2π)Ndet(Σ)uuuexp(−21ttt⊤Λttt)dttt
因第一项中被积项为奇函数,积分后为0
有
u
=
E
(
x
)
=
∫
−
∞
∞
u
(
2
π
)
N
d
e
t
(
Σ
)
exp
(
−
1
2
t
⊤
Λ
t
)
d
t
=
∫
−
∞
∞
u
(
2
π
)
N
d
e
t
(
Σ
)
exp
(
−
1
2
∑
i
=
1
N
1
λ
i
t
i
2
)
d
t
=
u
(
2
π
)
N
∏
i
=
N
N
λ
i
∏
i
=
1
N
∫
−
∞
∞
exp
(
−
1
2
λ
i
t
i
2
)
d
t
i
\begin{aligned} \pmb u=E(\pmb x)&=\int_{-\infty}^{\infty} \frac{\pmb u}{\sqrt{(2 \pi)^{N} d e t(\Sigma)}} \exp \left(-\frac{1}{2} \pmb t^{\top} \Lambda \pmb t\right) d \pmb t\\ &=\int_{-\infty}^{\infty} \frac{\pmb u}{\sqrt{(2 \pi)^{N}det(\Sigma)} } \exp \left(-\frac{1}{2} \sum_{i=1}^{N} \frac{1}{\lambda_{i}} t_{i}^{2}\right) d \pmb t\\ &=\frac{\pmb u}{\sqrt{(2 \pi)^{N}\prod_{i=N}^{N} \lambda_{i}} } \prod_{i=1}^{N} \int_{-\infty}^{\infty} \exp \left(-\frac{1}{2} \lambda_{i} t_{i}^{2}\right) d t_{i} \end{aligned}
uuu=E(xxx)=∫−∞∞(2π)Ndet(Σ)uuuexp(−21ttt⊤Λttt)dttt=∫−∞∞(2π)Ndet(Σ)uuuexp(−21i=1∑Nλi1ti2)dttt=(2π)N∏i=NNλiuuui=1∏N∫−∞∞exp(−21λiti2)dti
令 z i = 1 2 λ i t i z_i=\frac{1}{\sqrt{2 \lambda_i}}t_i zi=2λi1ti,有
μ = μ ( 2 π ) N ∏ i = 1 N λ i ∏ i = 1 N 2 λ i ∫ − ∞ ∞ exp ( − z i 2 ) d z i = μ \begin{aligned} \pmb \mu &= \frac{\pmb \mu}{\sqrt{(2 \pi)^{N} \prod_{i=1}^{N} \lambda_{i}} } \prod_{i=1}^{N} \sqrt{2 \lambda_{i}} \int_{-\infty}^{\infty} \exp \left(-z_{i}^{2}\right) d z_{i} \\ &= \pmb \mu \end{aligned} μμμ=(2π)N∏i=1Nλiμμμi=1∏N2λi∫−∞∞exp(−zi2)dzi=μμμ
原式得证
##E 4.(第二章第5题)对于高斯分布随机变量,
x
∼
N
(
μ
,
Σ
)
\pmb x \sim N(\pmb \mu, \pmb \Sigma)
xxx∼N(μμμ,ΣΣΣ),证明
Σ
=
E
[
(
x
−
u
)
(
x
−
u
)
⊤
]
=
∫
−
∞
∞
(
x
−
u
)
(
x
−
u
)
⊤
1
(
2
π
)
N
det
(
Σ
)
exp
[
−
1
2
(
x
−
u
)
⊤
Σ
−
1
(
x
−
u
)
]
d
x
\Sigma= E\left[(\pmb x- \pmb u)(\pmb x- \pmb u)^{\top}\right]=\int_{-\infty}^{\infty}(\pmb x-\pmb u)(\pmb x-\pmb u)^{\top} \frac{1}{\sqrt{(2 \pi)^{N} \operatorname{det}(\Sigma)}} \exp \left[-\frac{1}{2}(\pmb x-\pmb u)^{\top} \Sigma^{-1}(\pmb x-\pmb u)\right] d \pmb x
Σ=E[(xxx−uuu)(xxx−uuu)⊤]=∫−∞∞(xxx−uuu)(xxx−uuu)⊤(2π)Ndet(Σ)1exp[−21(xxx−uuu)⊤Σ−1(xxx−uuu)]dxxx
证:
令 f ( x ) = Σ ( x − u ) ( x − u ) ⊤ Σ − 1 f(\pmb x)=\Sigma(\pmb x-\pmb u)(\pmb x-\pmb u)^{\top} \Sigma^{-1} f(xxx)=Σ(xxx−uuu)(xxx−uuu)⊤Σ−1,先求 f ( x ) f(\pmb x) f(xxx)期望:
E [ f ( x ) ] = ∫ − ∞ ∞ f ( x ) p ( x ) d x = ∫ − ∞ ∞ ∑ ( x − u ) ( x − u ) T Σ − 1 ( 2 π ) N det ( Σ ) exp ( − 1 2 ( x − u ) ⊤ Σ − 1 ( x − u ) ) d x = ∫ − ∞ ∞ ∑ y y T ∑ − 1 ( 2 π ) N d e t ( Σ ) exp ( − 1 2 y T Σ − 1 y ) d y = ∫ − ∞ ∞ Σ y ( 2 π ) N det ( Σ ) ( exp ( − 1 2 y ⊤ Σ − 1 y ) Σ − 1 y ) ⊤ d y = ∫ − ∞ ∞ Σ y ( 2 π ) N det ( Σ ) d exp ⊤ ( − 1 2 y ⊤ Σ − 1 y ) = − Σ ∫ − ∞ ∞ y ( 2 π ) N det ( Σ ) d exp ( − 1 2 y ⊤ Σ − 1 y ) = − Σ [ y ( 2 π ) N d e t ( Σ ) exp ( − 1 2 y ⊤ Σ − 1 y ) ∣ − ∞ ∞ − ∫ − ∞ ∞ 1 ( 2 π ) N d e t Σ exp ( − 1 2 y ⊤ Σ − 1 y ) d y ] = − Σ [ 0 − I ] = Σ \begin{aligned} E[f(\pmb x)] &=\int_{-\infty}^{\infty} f(\pmb x) p(\pmb x) d \pmb x \\ &=\int_{-\infty}^{\infty} \frac{\sum(\pmb x-\pmb u)(\pmb x-\pmb u)^{T} \Sigma^{-1}}{\sqrt{(2 \pi)^{N}\operatorname{det}(\Sigma)} } \exp \left(-\frac{1}{2}(\pmb x-\pmb u)^{\top} \Sigma^{-1}(\pmb x-\pmb u)\right) d \pmb x \\ &=\int_{-\infty}^{\infty} \frac{\sum \pmb y \pmb y^{T} \sum^{-1}}{\sqrt{(2 \pi)^{N}d e t(\Sigma)} } \exp \left(-\frac{1}{2} \pmb y^{T} \Sigma^{-1} \pmb y\right) d \pmb y \\ &=\int_{-\infty}^{\infty} \frac{\Sigma \pmb y}{\sqrt{(2 \pi)^{N} \operatorname{det}(\Sigma)}}\left(\exp \left(-\frac{1}{2} \pmb y^{\top} \Sigma^{-1} \pmb y\right) \Sigma^{-1} \pmb y\right)^{\top} d \pmb y \\ &=\int_{-\infty}^{\infty} \frac{\Sigma \pmb y}{\sqrt{(2 \pi)^{N}\operatorname{det}(\Sigma)} } d \exp ^{\top}\left(-\frac{1}{2} \pmb y^{\top} \Sigma^{-1} \pmb y\right) \\ &=-\Sigma \int_{-\infty}^{\infty} \frac{\pmb y}{\sqrt{(2 \pi)^{N} \operatorname{det}(\Sigma)}} d \exp \left(-\frac{1}{2} \pmb y^{\top} \Sigma^{-1} \pmb y\right) \\ &=-\Sigma\left[\left.\frac{\pmb y}{\sqrt{(2 \pi)^{N} d e t(\Sigma)}} \exp \left(-\frac{1}{2} \pmb y^{\top} \Sigma^{-1} \pmb y\right)\right|_{-\infty} ^{\infty}-\int_{-\infty}^{\infty} \frac{1}{\sqrt{(2 \pi)^{N} d e t{\Sigma}}} \exp \left(-\frac{1}{2} \pmb y^{\top} \Sigma^{-1} \pmb y\right) d\pmb y\right]\\ &=-\Sigma[0-I]\\ &=\Sigma \end{aligned} E[f(xxx)]=∫−∞∞f(xxx)p(xxx)dxxx=∫−∞∞(2π)Ndet(Σ)∑(xxx−uuu)(xxx−uuu)TΣ−1exp(−21(xxx−uuu)⊤Σ−1(xxx−uuu))dxxx=∫−∞∞(2π)Ndet(Σ)∑yyyyyyT∑−1exp(−21yyyTΣ−1yyy)dyyy=∫−∞∞(2π)Ndet(Σ)Σyyy(exp(−21yyy⊤Σ−1yyy)Σ−1yyy)⊤dyyy=∫−∞∞(2π)Ndet(Σ)Σyyydexp⊤(−21yyy⊤Σ−1yyy)=−Σ∫−∞∞(2π)Ndet(Σ)yyydexp(−21yyy⊤Σ−1yyy)=−Σ[(2π)Ndet(Σ)yyyexp(−21yyy⊤Σ−1yyy)∣∣∣∣∣−∞∞−∫−∞∞(2π)NdetΣ1exp(−21yyy⊤Σ−1yyy)dyyy]=−Σ[0−I]=Σ
上面推导过程用到了如下性质:
1.若 Σ \Sigma Σ对称正定
d y ⊤ Σ − 1 y d y = ( Σ − 1 + Σ − T ) y = 2 Σ − 1 y \frac{d \pmb y^{\top} \Sigma^{-1} \pmb y}{d \pmb y}=\left(\Sigma^{-1}+\Sigma^{-T}\right) \pmb y=2 \Sigma^{-1} \pmb y dyyydyyy⊤Σ−1yyy=(Σ−1+Σ−T)yyy=2Σ−1yyy
2. exp ( − 1 2 y ⊤ Σ − 1 y ) \exp \left(-\frac{1}{2} \pmb y^{\top} \Sigma^{-1} \pmb y\right) exp(−21yyy⊤Σ−1yyy)对称(可从泰勒展开得出)
因
E
[
f
(
x
)
]
=
E
[
Σ
(
x
−
u
)
(
x
−
u
)
⊤
Σ
−
1
]
=
Σ
E
[
(
x
−
u
)
(
x
−
u
)
⊤
]
Σ
−
1
=
Σ
\begin{aligned} E[f(\pmb x)] &=E\left[\Sigma(\pmb x-\pmb u)(\pmb x-\pmb u)^{\top} \Sigma^{-1}\right] \\ &=\Sigma E\left[(\pmb x-\pmb u)(\pmb x-\pmb u)^{\top}\right] \Sigma^{-1} \\ &=\Sigma \end{aligned}
E[f(xxx)]=E[Σ(xxx−uuu)(xxx−uuu)⊤Σ−1]=ΣE[(xxx−uuu)(xxx−uuu)⊤]Σ−1=Σ
故 E [ ( x − u ) ( x − u ) ⊤ ] = Σ E\left[(\pmb x-\pmb u)(\pmb x-\pmb u)^{\top}\right]=\Sigma E[(xxx−uuu)(xxx−uuu)⊤]=Σ
原命题得证
5.(第二章第6题)对于K个相互独立的高斯变量, x k ∼ N ( μ k , Σ k ) x_k \sim N(\mu_k, \Sigma_k) xk∼N(μk,Σk),请证明它们的归一化积仍是高斯分布
exp ( − 1 2 ( x − u ) ⊤ Σ − 1 ( x − u ) ) = η ∏ k = 1 K exp ( − 1 2 ( x − u k ) ⊤ Σ k − 1 ( x − u k ) ) \exp \left(-\frac{1}{2}(\pmb x-\pmb u)^{\top} \Sigma^{-1}(\pmb x-\pmb u)\right)=\eta \prod_{k=1}^{K} \exp \left(-\frac{1}{2}\left( \pmb x-\pmb u_{k}\right)^{\top} \Sigma_{k}^{-1}\left( \pmb x- \pmb u_{k}\right)\right) exp(−21(xxx−uuu)⊤Σ−1(xxx−uuu))=ηk=1∏Kexp(−21(xxx−uuuk)⊤Σk−1(xxx−uuuk))
其中
Σ
−
1
=
∑
k
=
1
K
Σ
k
−
1
Σ
−
1
u
=
∑
k
=
1
K
Σ
k
−
1
u
k
\Sigma^{-1}=\sum_{k=1}^{K} \Sigma_{k}^{-1} \quad \Sigma^{-1} \pmb u=\sum_{k=1}^{K} \Sigma_{k}^{-1} u_{k}
Σ−1=k=1∑KΣk−1Σ−1uuu=k=1∑KΣk−1uk
证明:
η
∏
k
=
1
K
exp
(
−
1
2
(
x
−
u
k
)
T
Σ
k
−
1
(
x
−
u
k
)
)
=
η
exp
[
∑
k
=
1
K
−
1
2
(
x
−
u
k
)
⊤
Σ
k
−
1
(
x
−
u
k
)
]
=
η
exp
[
−
1
2
x
⊤
∑
k
=
1
k
Σ
k
−
1
x
−
x
⊤
∑
k
=
1
k
Σ
k
−
1
u
k
−
∑
k
=
1
k
u
k
⊤
Σ
k
−
1
x
+
∑
k
=
1
k
u
k
⊤
Σ
k
−
1
u
k
)
]
\begin{aligned} \eta \prod_{k=1}^{K} \exp \left(-\frac{1}{2}\left(\pmb x- \pmb u_{k }\right)^{T} \Sigma_{k}^{-1}\left(\pmb x-\pmb u_{k}\right)\right) &= \eta \exp \left[\sum_{k=1}^{K}-\frac{1}{2}\left(\pmb x-\pmb u_{k}\right)^{\top} \Sigma_{k}^{-1}\left(\pmb x-\pmb u_{k}\right)\right] \\ &= \eta \exp \left[-\frac{1}{2} \pmb x^{\top} \sum_{k=1}^{k} \Sigma_{k}^{-1} \pmb x-\pmb x^{\top} \sum_{k=1}^{k} \Sigma_{k}^{-1} \pmb u_{k}-\sum_{k=1}^{k} \pmb u_{k}^{\top} \Sigma_{k}^{-1} \pmb x+\sum_{k=1}^{k} \pmb u_{k}^{\top} \Sigma_{k}^{-1} \pmb u_{k}) \right] \\ \end{aligned}
ηk=1∏Kexp(−21(xxx−uuuk)TΣk−1(xxx−uuuk))=ηexp[k=1∑K−21(xxx−uuuk)⊤Σk−1(xxx−uuuk)]=ηexp[−21xxx⊤k=1∑kΣk−1xxx−xxx⊤k=1∑kΣk−1uuuk−k=1∑kuuuk⊤Σk−1xxx+k=1∑kuuuk⊤Σk−1uuuk)]
记 M = ( ∑ k = 1 K Σ k − 1 U k ) ⊤ ( ∑ k = 1 k ∑ k − 1 ) − 1 ( ∑ k = 1 k Σ k − 1 U k ) M=\left(\sum_{k=1}^{K} \Sigma_{k}^{-1} U_{k}\right)^{\top}\left(\sum_{k=1}^{k} \sum_{k}^{-1}\right)^{-1}\left(\sum_{k=1}^{k} \Sigma_{k}^{-1} U_{k}\right) M=(∑k=1KΣk−1Uk)⊤(∑k=1k∑k−1)−1(∑k=1kΣk−1Uk)
η ∏ k = 1 K exp ( − 1 2 ( x − u k ) T Σ k − 1 ( x − u k ) ) = η exp [ − 1 2 ( x T ∑ k = 1 K Σ k − 1 x − x ⊤ ∑ k = 1 k Σ k − 1 u k − ∑ k = 1 k u k ⊤ Σ k − 1 x + M ) + 1 2 M + 1 2 ∑ k = 1 K u k ⊤ Σ k − 1 u k ] = η exp [ 1 2 M + 1 2 ∑ k = 1 K u k T Σ k − 1 u k ] exp [ − 1 2 x ⊤ ∑ k = 1 K Σ k = 1 − 1 x − x T ∑ k = 1 K Σ k − 1 u k − ∑ k = 1 T u k T Σ k − 1 x + M ] = η ′ exp [ − 1 2 ( x − ( ∑ k = 1 k Σ k − 1 ) − 1 ∑ k = 1 k Σ k − 1 u k ) T ∑ k = 1 K Σ k − 1 ( x − ( ∑ k = 1 K Σ k − 1 ) − 1 ∑ k = 1 k Σ k − 1 u k ) ] = η ′ exp [ − 1 2 ( x − u ) ⊤ Σ − 1 ( x − u ) ] ( 记 Σ − 1 = ∑ k = 1 K Σ k − 1 u = Σ − 1 ∑ k = 1 K Σ k − 1 u k ) \begin{aligned} \eta \prod_{k=1}^{K} \exp \left(-\frac{1}{2}\left(\pmb x- \pmb u_{k }\right)^{T} \Sigma_{k}^{-1}\left(\pmb x-\pmb u_{k}\right)\right) &= \eta \exp \left[-\frac{1}{2}\left(\pmb x^T \sum_{k=1}^{K} \Sigma_{k}^{-1} \pmb x- \pmb x^{\top} \sum_{k=1}^{k} \Sigma_{k}^{-1} \pmb u_{k}-\sum_{k=1}^{k} \pmb u_{k}^{\top} \Sigma_{k}^{-1} \pmb x+M\right)+\frac{1}{2} M+\frac{1}{2} \sum_{k=1}^{K} \pmb u_{k}^{\top} \Sigma_{k}^{-1} \pmb u_{k}\right] \\ &= \eta \exp \left[\frac{1}{2} M+\frac{1}{2} \sum_{k=1}^{K} \pmb u_{k}^{T} \Sigma_{k}^{-1} \pmb u_{k}\right] \exp \left[-\frac{1}{2} \pmb x^{\top} \sum_{k=1}^{K} \Sigma_{k=1}^{-1} \pmb x-\pmb x^{T} \sum_{k=1}^{K}\Sigma_{k}^{-1} \pmb u_{k}-\sum_{k=1}^{T} \pmb u_{k}^{T} \Sigma_{k}^{-1} \pmb x+M \right] \\ &= \eta^{\prime} \exp \left[-\frac{1}{2}\left(\pmb x-\left(\sum_{k=1}^{k} \Sigma_{k}^{-1}\right)^{-1} \sum_{k=1}^{k} \Sigma_{k}^{-1} \pmb u_{k}\right)^{T} \sum_{k=1}^{K} \Sigma_{k}^{-1}\left(\pmb x-\left(\sum_{k=1}^{K} \Sigma_{k}^{-1}\right)^{-1} \sum_{k=1}^{k} \Sigma_{k}^{-1} \pmb u_{k}\right)\right] \\ &= \eta^{\prime} \exp \left[-\frac{1}{2}(\pmb x-\pmb u)^{\top} \Sigma^{-1}(\pmb x-\pmb u)\right]\left(记 \Sigma^{-1}=\sum_{k=1}^{K} \Sigma_{k}^{-1} \quad \pmb u=\Sigma^{-1} \sum_{k=1}^{K} \Sigma_{k}^{-1} \pmb u_{k}\right) \end{aligned} ηk=1∏Kexp(−21(xxx−uuuk)TΣk−1(xxx−uuuk))=ηexp[−21(xxxTk=1∑KΣk−1xxx−xxx⊤k=1∑kΣk−1uuuk−k=1∑kuuuk⊤Σk−1xxx+M)+21M+21k=1∑Kuuuk⊤Σk−1uuuk]=ηexp[21M+21k=1∑KuuukTΣk−1uuuk]exp[−21xxx⊤k=1∑KΣk=1−1xxx−xxxTk=1∑KΣk−1uuuk−k=1∑TuuukTΣk−1xxx+M]=η′exp⎣⎢⎡−21⎝⎛xxx−(k=1∑kΣk−1)−1k=1∑kΣk−1uuuk⎠⎞Tk=1∑KΣk−1⎝⎛xxx−(k=1∑KΣk−1)−1k=1∑kΣk−1uuuk⎠⎞⎦⎥⎤=η′exp[−21(xxx−uuu)⊤Σ−1(xxx−uuu)](记Σ−1=k=1∑KΣk−1uuu=Σ−1k=1∑KΣk−1uuuk)
上式中
{
Σ
−
1
=
∑
k
=
1
K
Σ
k
−
1
u
=
Σ
∑
k
=
1
K
Σ
k
−
1
u
k
⇒
Σ
−
1
u
=
∑
k
=
1
k
Σ
k
−
1
u
k
η
′
=
η
exp
[
1
2
u
⊤
Σ
−
1
u
+
1
2
∑
k
=
1
k
u
k
⊤
Σ
k
−
1
u
k
]
\left\{\begin{array}{l} \Sigma^{-1}=\sum_{k=1}^{K} \Sigma_{k}^{-1} \\ \pmb u=\Sigma \sum_{k=1}^{K} \Sigma_{k}^{-1} \pmb u_{k} \Rightarrow \Sigma^{-1} \pmb u=\sum_{k=1}^{k} \Sigma_{k}^{-1} \pmb u_{k} \\ \eta^{\prime}=\eta \exp \left[\frac{1}{2} \pmb u^{\top} \Sigma^{-1} \pmb u+\frac{1}{2} \sum_{k=1}^{k} \pmb u_{k}^{\top} \Sigma_{k}^{-1} \pmb u_{k}\right] \end{array}\right.
⎩⎪⎨⎪⎧Σ−1=∑k=1KΣk−1uuu=Σ∑k=1KΣk−1uuuk⇒Σ−1uuu=∑k=1kΣk−1uuukη′=ηexp[21uuu⊤Σ−1uuu+21∑k=1kuuuk⊤Σk−1uuuk]
原命题得证