【西瓜书笔记】10. 高斯混合模型

定义

定义：
$$P(\mathbf{x})=\sum _{i=1}^k{\alpha }_i\cdot \varphi \left(\mathbf{x}\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)$$
该模型共由k个混合成分组成，每个混合成分对应一个高斯分布，其中， $\mathbf{x}\in {\mathbb{R}}^n$，${\alpha }_i$为混合系数，且${\alpha }_i\ge 0,{\sum }_{i=1}^k{\alpha }_i=1$, $\varphi \left(\mathbf{x}\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)$为多元高斯分布（当$\mathbf{x}$为标量时，相应地替换为一元高斯分布）的概率密度函数：
$$\varphi \left(\mathbf{x}\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)=\frac{1}{(2\pi {)}^{\frac{n}{2}}{\left|{\mathbf{\Sigma }}_i\right|}^{\frac{1}{2}}}\exp \left(-\frac{1}{2}{\left(\mathbf{x}-{\mathbf{\mu }}_i\right)}^{\mathrm{T}}{\mathbf{\Sigma }}_i^{-1}\left(\mathbf{x}-{\mathbf{\mu }}_i\right)\right)$$
其生成数据的方式为：首先，依概率${\alpha }_i$选择第i个高斯混合成分，接着依据该混合成分的概率分布$\varphi \left(\mathbf{x}\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)$生成样本。

参数估计

EM算法

已知数据集 D = { x 1 , x 2 , … , x m } D=\left\{\boldsymbol{x}_{1}, \boldsymbol{x}_{2}, \ldots, \boldsymbol{x}_{m}\right\} D={x1​,x2​,…,xm​}中的样本均由某个高斯混合模型生成，而每个样本${\mathbf{x}}_j$是由哪个高斯混合成分生成是未知的，属于一个隐变量，我们令其为 z j , z j ∈ { 1 , 2 , … , k } z_{j}, z_{j} \in\{1,2, \ldots, k\} zj​,zj​∈{1,2,…,k}表示生成样本${\mathbf{x}}_j$的高斯混合成分，结合高斯混合模型生成数据的方式易知$z_j$的分布律为$P\left(z_j=i\right)={\alpha }_i$。接下来应用EM算法。

E步：确定Q函数，并把样本序列和隐变量序列代入其中

$$\begin{array}{rl}Q\left(\theta \mid {\theta }^{(i)}\right)& =\sum _{Z}P\left(Z\mid X,{\theta }^{(i)}\right)\ln P(X,Z\mid \theta )\\ & =\sum _{z_1,z_2,\dots ,z_m}\left\{\prod _{j=1}^{m}P\left(z_j\mid x_j,{\theta }^{(i)}\right)\ln \left[\prod _{j=1}^{m}P\left(x_j,z_j\mid \theta \right)\right]\right\}\\ & =\sum _{j=1}^m\left[\sum _{z_j}P\left(z_j\mid x_j,{\theta }^{(i)}\right)\ln P\left(x_j,z_j\mid \theta \right)\right]\\ & =\sum _{j=1}^m\left[\sum _{i=1}^{k}P\left(z_j=i\mid x_j,{\theta }^{(i)}\right)\ln P\left(x_j,z_j=i\mid \theta \right)\right]\end{array}$$

其中，第2个等式到第3等式是根据EM算法笔记中的结果得到。

对于$P\left(z_j=i\mid {\mathbf{x}}_j,{\theta }^{(i)}\right)$，如果我们先不考虑${\theta }^{(i)}$，有
$$\begin{array}{rl}P\left(z_j=i\mid {\mathbf{x}}_j\right)& =\frac{P\left(z_j=i\right)\cdot P\left({\mathbf{x}}_j\mid z_j=i\right)}{P\left({\mathbf{x}}_j\right)}\\ & =\frac{{\alpha }_i\cdot \varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)}{{\sum }_{l=1}^k{\alpha }_l\cdot \varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_l,{\mathbf{\Sigma }}_l\right)}\end{array}$$
这就是西瓜书中的式9.30。如果考虑${\theta }^{(i)}$， 那么
$$P\left(z_j=i\mid {\mathbf{x}}_j,{\theta }^{(i)}\right)=\frac{{\alpha }_i^{(i)}\cdot \varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_i^{(i)},{\mathbf{\Sigma }}_i^{(i)}\right)}{{\sum }_{l=1}^k{\alpha }_l^{(i)}\cdot \varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_l^{(i)},{\mathbf{\Sigma }}_l^{(i)}\right)}$$
这里的i表示这是第i次迭代，同时也表示这个参数是已知的，也就是说式中都是已知量，那这个概率值也是已知量，所以我们可以将其简记为${\gamma }_{ji}$。

对于$P\left(x_j,z_j=i\mid \theta \right)$,利用$P(A,B)=P(A)\cdot P(A|B)$，可以有:
$$\begin{array}{rl}P\left({\mathbf{x}}_j,z_j=i\mid \theta \right)& =P\left({\mathbf{x}}_j\mid z_j=i,\theta \right)\cdot P\left(z_j=i\mid \theta \right)\\ & =\varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)\cdot {\alpha }_i\end{array}$$

这时这个式子中的三个参数都是未知的。将上面两个式子代回到Q函数，可得
$$\begin{array}{rl}Q\left(\theta \mid {\theta }^{(i)}\right)& =\sum _{j=1}^m\left[\sum _{i=1}^{k}P\left(z_j=i\mid {\mathbf{x}}_j,{\theta }^{(i)}\right)\ln P\left({\mathbf{x}}_j,z_j=i\mid \theta \right)\right]\\ & =\sum _{j=1}^m\sum _{i=1}^k{\gamma }_{ji}\ln \left[{\alpha }_i\cdot \varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)\right]\\ & =\sum _{j=1}^m\sum _{i=1}^k{\gamma }_{ji}\left[\ln {\alpha }_i+\ln \varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)\right]\\ & =\sum _{j=1}^m\sum _{i=1}^k\left[{\gamma }_{ji}\ln {\alpha }_i+{\gamma }_{ji}\ln \varphi \left({\mathbf{x}}_j\mid {\mathbf{\mu }}_i,{\mathbf{\Sigma }}_i\right)\right]\\ & =\sum _{j=1}^m\sum _{i=1}^k\left\{{\gamma }_{ji}\ln {\alpha }_i+{\gamma }_{ji}\ln \left[\frac{1}{(2\pi {)}^{\frac{n}{2}}{\left|{\mathbf{\Sigma }}_i\right|}^{\frac{1}{2}}}\exp \left(-\frac{1}{2}{\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^{\mathrm{T}}{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)\right)\right]\right\}\\ & =\sum _{j=1}^m\sum _{i=1}^k\left\{{\gamma }_{ji}\ln {\alpha }_i+{\gamma }_{ji}\left[\ln \frac{1}{(2\pi {)}^{\frac{n}{2}}}-\frac{1}{2}\ln \left|{\mathbf{\Sigma }}_i\right|-\frac{1}{2}{\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)\right]\right\}\\ & =\sum _{j=1}^m\sum _{i=1}^k\left\{{\gamma }_{ji}\ln {\alpha }_i+{\gamma }_{ji}\ln \frac{1}{(2\pi {)}^{\frac{n}{2}}}-\frac{1}{2}{\gamma }_{ji}\ln \left|{\mathbf{\Sigma }}_i\right|-\frac{1}{2}{\gamma }_{ji}{\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)\right\}\end{array}$$
接下来就对Q函数进行极大化操作。对于m个多元正态分布生成的样本的似然函数${\sum }_{i=1}^m\ln \varphi ({\mathbf{x}}_j\mid {\mathbf{\mu }}_j,{\mathbf{\Sigma }}_j)$，对比上面第4个等式的式子，差别仅仅是多了$\gamma ,\alpha$这些常数项。而且注意，虽然有${\sum }_{i=1}^k$这个求和符号，但是我们想要极大化的是特定的$({\mu }_i,{\Sigma }_i)$，对于下标不等于i的参数集合$(\mu ,\Sigma )$,我们都可以看成是常数，所以${\sum }_{i=1}^k$这个求和符号其实可以在极大化操作中忽略。而$\ln {\alpha }_i$是个凹函数，且有线性等式约束${\alpha }_i\ge 0,{\sum }_{i=1}^k{\alpha }_i=1$,我们可以用拉格朗日乘子法求出来的点一定是目标函数的最大值点，所以我们用拉格朗日乘子法求${\alpha }_i$。

M步：求使得Q函数达到极大的${\theta }^{(i+1)}$。

求${\mathbf{\mu }}_i^{(i+1)}$,也就是对于Q函数关于${\mathbf{\mu }}_i$求偏导
$$\begin{array}{rl}\frac{\partial Q\left(\theta ,{\theta }^{(i)}\right)}{\partial {\mathbf{\mu }}_i}& =\sum _{j=1}^m\left\{0+0-0-\frac{1}{2}{\gamma }_{ji}\frac{\partial \left({\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)\right)}{\partial {\mathbf{\mu }}_i}\right\}\\ & =-\sum _{j=1}^m\frac{1}{2}{\gamma }_{ji}\frac{\partial \left({\mathbf{x}}_j^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j-{\mathbf{x}}_j^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i-{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j+{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i\right)}{\partial {\mathbf{\mu }}_i}\\ & =-\sum _{j=1}^m\frac{1}{2}{\gamma }_{ji}\frac{\partial \left(-{\mathbf{x}}_j^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i-{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j+{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i\right)}{\partial {\mathbf{\mu }}_i}\end{array}$$
由于${\mathbf{x}}_j^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i$和${\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j$均为标量且${\Sigma }_i$为对称矩阵，标量转置还是它本身，所以
$${\mathbf{x}}_j^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i={\left({\mathbf{x}}_j^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i\right)}^T={\mathbf{\mu }}_i^T{\left({\mathbf{\Sigma }}_i^{-1}\right)}^T{\mathbf{x}}_j={\mathbf{\mu }}_i^T{\left({\mathbf{\Sigma }}_i^T\right)}^{-1}{\mathbf{x}}_j={\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j$$
代入上式可得
$$\begin{array}{rl}\frac{\partial Q\left(\theta ,{\theta }^{(i)}\right)}{\partial {\mathbf{\mu }}_{\mathbf{i}}}& =-\sum _{j=1}^m\frac{1}{2}{\gamma }_{ji}\frac{\partial \left(-x_j^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i-{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j+{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i\right)}{\partial {\mathbf{\mu }}_i}\\ & =-\sum _{j=1}^m\frac{1}{2}{\gamma }_{ji}\frac{\partial \left(-2{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j+{\mathbf{\mu }}_i^T{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i\right)}{\partial {\mathbf{\mu }}_i}\end{array}$$
又由矩阵微分公式$\frac{\partial {\mathbf{x}}^T\mathbf{a}}{\partial \mathbf{x}}=\mathbf{a},\frac{\partial {\mathbf{x}}^T\mathbf{B}\mathbf{x}}{\partial \mathbf{x}}=\left(\mathbf{B}+{\mathbf{B}}^T\right)\mathbf{x}$可得
$$\frac{\partial Q\left(\theta ,{\theta }^{(i)}\right)}{\partial {\mathbf{\mu }}_{\mathbf{i}}}=\sum _{j=1}^m\frac{1}{2}{\gamma }_{ji}\left(2{\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_{\mathbf{j}}-2{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_{\mathbf{i}}\right)=\sum _{j=1}^m{\gamma }_{ji}\left({\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_{\mathbf{j}}-{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_{\mathbf{i}}\right)$$

令上式等于0可得
$$\begin{array}{c}\sum _{j=1}^m{\gamma }_{ji}\left({\mathbf{\Sigma }}_i^{-1}{\mathbf{x}}_j-{\mathbf{\Sigma }}_i^{-1}{\mathbf{\mu }}_i\right)=0\\ {\mathbf{\Sigma }}_i^{-1}\cdot \sum _{j=1}^m{\gamma }_{ji}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)=0\\ \sum _{j=1}^m{\gamma }_{ji}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)=0\end{array}$$

$${\mathbf{\mu }}_i=\frac{{\sum }_{j=1}^m{\gamma }_{ji}{\mathbf{x}}_j}{{\sum }_{j=1}^m{\gamma }_{ji}}\Rightarrow {\mathbf{\mu }}_i^{(i+1)}=\frac{{\sum }_{j=1}^m{\gamma }_{ji}{\mathbf{x}}_j}{{\sum }_{j=1}^m{\gamma }_{ji}}$$

此即为西瓜书式9.34

求${\Sigma }_i^{(i+1)}$,对Q函数关于${\Sigma }_i$求偏导
$$\begin{array}{rl}\frac{\partial Q\left(\theta ,{\theta }^{(i)}\right)}{\partial {\mathbf{\Sigma }}_i}& =\sum _{j=1}^m\left\{0+0-\frac{\partial }{\partial {\mathbf{\Sigma }}_i}\left(\frac{1}{2}{\gamma }_{ji}\ln \left|{\mathbf{\Sigma }}_i\right|\right)-\frac{\partial }{\partial {\mathbf{\Sigma }}_i}\left[\frac{1}{2}{\gamma }_{ji}{\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)\right]\right\}\\ & =\sum _{j=1}^m\left\{-\frac{1}{2}{\gamma }_{ji}\frac{\partial \left(\ln \left|{\mathbf{\Sigma }}_i\right|\right)}{\partial {\mathbf{\Sigma }}_i}-\frac{1}{2}{\gamma }_{ji}\frac{\partial \left[{\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)\right]}{\partial {\mathbf{\Sigma }}_i}\right\}\end{array}$$
由矩阵微分公式$\frac{\partial |\mathbf{X}|}{\partial \mathbf{X}}=|\mathbf{X}|\cdot {\left({\mathbf{X}}^{-1}\right)}^T,\frac{\partial {\mathbf{a}}^T{\mathbf{X}}^{-1}\mathbf{b}}{\partial \mathbf{X}}=-{\mathbf{X}}^{-T}\mathbf{a}{\mathbf{b}}^T{\mathbf{X}}^{-T}$,且${\Sigma }_i$是对称矩阵（先求逆再求转置相当于只求逆），可得
$$\begin{array}{rl}\frac{\partial Q\left(\theta ,{\theta }^{(i)}\right)}{\partial {\mathbf{\Sigma }}_i}& =\sum _{j=1}^m\left\{-\frac{1}{2}{\gamma }_{ji}\cdot \frac{1}{\left|{\mathbf{\Sigma }}_i\right|}\cdot \left|{\mathbf{\Sigma }}_i\right|\cdot {\left({\mathbf{\Sigma }}_i^{-1}\right)}^T-\frac{1}{2}{\gamma }_{ji}\cdot {\left(-{\mathbf{\Sigma }}_i\right)}^{-T}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-T}\right\}\\ & =\sum _{j=1}^m\left\{-\frac{1}{2}{\gamma }_{ji}{\mathbf{\Sigma }}_i^{-1}+\frac{1}{2}{\gamma }_{ji}{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\right\}\end{array}$$
令上式等于0可得
$$\begin{array}{rl}& \sum _{j=1}^m\left\{-\frac{1}{2}{\gamma }_{ji}{\mathbf{\Sigma }}_i^{-1}+\frac{1}{2}{\gamma }_{ji}{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\right\}=0\\ & \sum _{j=1}^m\left\{-\frac{1}{2}{\gamma }_{ji}+\frac{1}{2}{\gamma }_{ji}{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T\right\}=0\\ & \frac{1}{2}\sum _{j=1}^m{\gamma }_{ji}{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T=\frac{1}{2}\sum _{j=1}^m{\gamma }_{ji}\\ & {\Sigma }_i^{-1}\sum _{j=1}^m{\gamma }_{ji}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T=\sum _{j=1}^m{\gamma }_{ji}\end{array}$$

$$\begin{array}{c}{\mathbf{\Sigma }}_i^{-1}=\frac{{\sum }_{j=1}^m{\gamma }_{ji}}{{\sum }_{j=1}^m{\gamma }_{ji}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T}\\ {\mathbf{\Sigma }}_i=\frac{{\sum }_{j=1}^m{\gamma }_{ji}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T}{{\sum }_{j=1}^m{\gamma }_{ji}}\Rightarrow {\mathbf{\Sigma }}_i^{(i+1)}=\frac{{\sum }_{j=1}^m{\gamma }_{ji}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right){\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T}{{\sum }_{j=1}^m{\gamma }_{ji}}\end{array}$$

此即为西瓜书式9.35

求${\alpha }_i^{(i+1)}$,由于${\alpha }_i$存在约束${\sum }_{i=1}^k{\alpha }_i=1$，所以考虑使用拉格朗日乘子法，其拉格朗日函数为
$$\begin{array}{rl}L(\mathbf{\alpha },\lambda )& =Q\left(\theta ,{\theta }^{(i)}\right)+\lambda \left(\sum _{i=1}^k{\alpha }_i-1\right)\\ & =\sum _{j=1}^m\sum _{i=1}^k\left\{{\gamma }_{ji}\ln {\alpha }_i+{\gamma }_{ji}\ln \frac{1}{(2\pi {)}^{\frac{n}{2}}}-\frac{1}{2}{\gamma }_{ji}\ln \left|{\mathbf{\Sigma }}_i\right|-\frac{1}{2}{\gamma }_{ji}{\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)}^T{\mathbf{\Sigma }}_i^{-1}\left({\mathbf{x}}_j-{\mathbf{\mu }}_i\right)\right\}+\lambda \left(\sum _{i=1}^k{\alpha }_i-1\right)\end{array}$$

对拉格朗日函数关于${\alpha }_i$求偏导：
$$\frac{\partial L(\mathbf{\alpha },\lambda )}{\partial {\alpha }_i}=\sum _{j=1}^m\left\{\frac{\partial \left({\gamma }_{ji}\ln {\alpha }_i\right)}{\partial {\alpha }_i}+0-0-0\right\}+\lambda \frac{\partial \left({\sum }_{i=1}^k{\alpha }_i-1\right)}{\partial {\alpha }_i}=\sum _{j=1}^m\frac{{\gamma }_{ji}}{{\alpha }_i}+\lambda$$

令上式等于0可得
$$\begin{array}{rl}& \sum _{j=1}^m\frac{{\gamma }_{ji}}{{\alpha }_i}+\lambda =0\\ & \frac{1}{{\alpha }_i}\sum _{j=1}^m{\gamma }_{ji}=-\lambda \\ & {\alpha }_i=-\frac{1}{\lambda }\sum _{j=1}^m{\gamma }_{ji}\end{array}$$
由于${\sum }_{i=1}^k{\alpha }_i=1$,则上式两边关于i求和可得
$$\begin{array}{c}\sum _{i=1}^k{\alpha }_i=-\frac{1}{\lambda }\sum _{i=1}^k\sum _{j=1}^m{\gamma }_{ji}\\ 1=-\frac{1}{\lambda }\sum _{i=1}^k\sum _{j=1}^m{\gamma }_{ji}\\ \lambda =-\sum _{i=1}^k\sum _{j=1}^m{\gamma }_{ji}\end{array}$$
这时我们要求出$\lambda$，又因为
$$\sum _{i=1}^k\sum _{j=1}^m{\gamma }_{ji}=\sum _{j=1}^m\sum _{i=1}^k{\gamma }_{ji}=\sum _{j=1}^m\sum _{i=1}^{k}P\left(z_j=i\mid {\mathbf{x}}_j,{\theta }^{(i)}\right)=\sum _{j=1}^{m}1=m$$
所以
$$\begin{array}{c}\lambda =-\sum _{i=1}^k\sum _{j=1}^m{\gamma }_{ji}=-m\\ {\alpha }_i=-\frac{1}{\lambda }\sum _{j=1}^m{\gamma }_{ji}=\frac{1}{m}\sum _{j=1}^m{\gamma }_{ji}\end{array}$$
由于$0\le {\gamma }_{ji}=P\left(z_j=i\mid {\mathbf{x}}_j,{\theta }^{(i)}\right)\le 1$，所以
$$0\le \sum _{j=1}^m{\gamma }_{ji}\le m\Rightarrow 0\le \frac{1}{m}\sum _{j=1}^m{\gamma }_{ji}\le 1$$
那么此时解得的${\alpha }_i$是有效解，可以作为下一次迭代的初始参数，也即
$${\alpha }_i=\frac{1}{m}\sum _{j=1}^m{\gamma }_{ji}\Rightarrow {\alpha }_i^{(i+1)}=\frac{1}{m}\sum _{j=1}^m{\gamma }_{ji}$$
此即为西瓜书式9.38