EM
极大似然估计:要求样本来自同一个分布,然后估计参数。
EM:为个体指定一个参数–>估计参数–>调节分布–>估计参数。
E-step : repeatedly construct a low-bound on
l
.
M-step : optimize that low-bound.
details are as follows:
EM : repeat until convergence{
(E-step) for each i ,set
(M-step) set
θ:=argmaxθ∑i∑z(i)Qi(z(i))logp(x(i),z(i);θ)Qi(z(i))
compensatory acknowlege:
1) Jensen’s inequality
f is a convex function ,
E[f(x)]≥f[E(x)]
2) 多元正态分布:若
x∼Np(μ,Σ)
x
服从
f(x1,x2……xp)=12πp2|Σ|12exp−12(x−μ)TΣ−1(x−μ)
如果,
y=cx+b ,c∈RM×1 ,
则
Y∼Nm(cμ+b,cΣcT)
FA
A joint distribution on (x,z) as follows :
z∼N(0,I),x|z∼N(μ+Λz,ψ)
z∈Rk,μ∈Rn,Λ∈Rn×k,ψ∈Rn×n
,
ψ
is a diagonal matrix and
k
is usually chosen to be smaller than n . Imagining that
x=μ+Λz+ϵ,[z x]T∼N(μzx,Σ)
z∼N(0,I),ϵ∼N(0,ψ),μzx=[0μ],Σ=[IΛΛTΛΛT+ψ]
编辑公式会有眼花缭乱的感觉~