混合高斯模型的EM算法
Pre-requisites
"混合模型简介"笔记
Overview
在"混合模型简介"笔记中, 我们介绍了混合模型. 回顾一下, 假如观测值 X i X_{i} Xi来源于含有 K K K个组件的混合模型, 那么 X i X_{i} Xi的边缘概率分布的形式为: P ( X i = x ) = ∑ k = 1 K π k P ( X i = x ∣ Z i = k ) P\left(X_{i}=x\right)=\sum_{k=1}^{K} \pi_{k} P\left(X_{i}=x | Z_{i}=k\right) P(Xi=x)=k=1∑KπkP(Xi=x∣Zi=k)其中 Z i ∈ { 1 , … , K } Z_{i} \in\{1, \ldots, K\} Zi∈{
1,…,K}为隐变量, 代表 X i X_{i} Xi对应的模型组件, P ( X i ∣ Z i ) P\left(X_{i} | Z_{i}\right) P(Xi∣Zi)代表混合组件的分布, π k \pi_{k} πk代表 X i X_{i} Xi来源于第k个组件的概率.
在这篇笔记中, 我们将介绍混合高斯模型中的 expectation-maximization (EM) \textbf{expectation-maximization (EM)} expectation-maximization (EM)算法. 我们有条件分布 X i ∣ Z i = k ∼ N ( μ k , σ k 2 ) X_{i} | Z_{i}=k \sim N\left(\mu_{k}, \sigma_{k}^{2}\right) Xi∣Zi=k∼N(μk,σk2), 因此 X i X_{i} Xi的边缘概率分布为: P ( X i = x ) = ∑ k = 1 K P ( Z i = k ) P ( X i = x ∣ Z i = k ) = ∑ k = 1 K π k N ( x ; μ k , σ k 2 ) P\left(X_{i}=x\right)=\sum_{k=1}^{K} P\left(Z_{i}=k\right) P\left(X_{i}=x | Z_{i}=k\right)=\sum_{k=1}^{K} \pi_{k} N\left(x ; \mu_{k}, \sigma_{k}^{2}\right) P(Xi=x)=k=1∑KP(Zi=k)P(Xi=x∣Zi=k)=k=1∑KπkN(x;μk,σk2)类似地, X 1 , … , X n X_{1}, \ldots, X_{n} X1,…,Xn的联合概率分布为 P ( X 1 = x 1 , … , X n = x n ) = ∏ i = 1 n ∑ k = 1 K π k N ( x i ; μ k , σ k 2 ) P\left(X_{1}=x_{1}, \ldots, X_{n}=x_{n}\right)=\prod_{i=1}^{n} \sum_{k=1}^{K} \pi_{k} N\left(x_{i} ; \mu_{k}, \sigma_{k}^{2}\right) P(X1=x1,…,Xn=xn)=i=1∏nk=1∑Kπ