Machine-Learning：第3章 EM算法

最新推荐文章于 2021-02-23 11:16:52 发布

Axiiiz

最新推荐文章于 2021-02-23 11:16:52 发布

阅读量417

点赞数

分类专栏： sklearn

本文链接：https://blog.csdn.net/sinat_38059712/article/details/105766422

版权

sklearn 专栏收录该内容

9 篇文章 0 订阅

订阅专栏

第3章 EM算法

本章思维导图
本章内容概要

本章思维导图

（待更新）

本章内容概要

前言

EM算法是机器学习十大算法之一，它很简单，但是也同样很有深度，简单是因为它就分两步求解问题，

E步：求期望（expectation）
M步：求极大（maximization)

深度在于它的数学推理涉及到比较繁杂的概率公式等，所以本文会介绍很多概率方面的知识，不懂的同学可以先去了解一些知识，当然本文也会尽可能的讲解清楚这些知识，讲的不好的地方麻烦大家评论指出，后续不断改进完善。

EM算法引入

概率模型有时候既含有观测变量，又含有隐变量或潜在变量，如果概率模型的变量都是观测变量，那么给定数据，可以直接用极大似然估计法，或贝叶斯估计方法估计模型参数，但是当模型含有隐变量时，就不能简单的使用这些方法，EM算法就是含有隐变量的概率模型参数的极大似然估计法，或极大后验概率估计法，我们讨论极大似然估计，极大后验概率估计与其类似。参考统计学习方法书中的一个例子来引入EM算法，假设有3枚硬币，分别记做A、B、C，这些硬币正面出现的概率分别是 $\pi$ 、 $p$ 、 $q$ ，进行如下实验：

先掷硬币A，根据结果选出硬币B和硬币C，正面选硬币B，反面选硬币C
通过选择出的硬币，掷硬币的结果出现正面为1，反面为0 如此独立地重复n次实验，我们当前规定n=10，则10次的结果如下所示： $1, 1, 0, 1, 0, 0, 1, 0, 1, 1$ 假设只通过观测到掷硬币的结果，不能观测掷硬币的过程，问如何估计三个硬币出现正面的概率？我们来构建这样一个三硬币模型： $\begin{aligned} P(y|\theta) &=\sum_{z}P(y,z|\theta)=\sum_{z}P(z|\theta)P(y|z,\theta) \\ &=\pi p^{y}(1-p)^{1-y}+(1-\pi)q^{y}(1-q)^{1-y} \end{aligned}$

若 $y = 1$ ，表示这此看到的是正面，这个正面有可能是B的正面，也可能是C的正面，则 $P(1|\theta)=\pi p+(1-\pi)q$

若 $y = 0$ ，则 $P(0|\theta)=\pi (1-p)+(1-\pi)(1-q)$
y是观测变量，表示一次观测结果是1或0，z是隐藏变量，表示掷硬币A的结果，这个是观测不到结果的， $\theta=(\pi,p,q)$ 表示模型参数，将观测数据表示为 $Y=(Y_1,Y_2,...,Y_n)^{T}$ ，未观测的数据表示为 $Z=(Z_1,Z_2,...,Z_n)^{T}$ ，则观测函数的似然函数是： $\begin{aligned} P(Y|\theta)&=\sum_{Z}P(Z|\theta)P(Y|Z,\theta)\\ &=\prod_{i=0} ( \pi p^{y_i}(1-p)^{1-y_{i}}+(1-\pi)q^{y_{i}}(1-q)^{1-y_{i}}) \end{aligned}$ 考虑求模型参数 $\theta=(\pi,p,q)$ 的极大似然估计，即： $\hat{\theta}=arg\max_{\theta}logP(Y|\theta)$ 这个问题没有解析解，只有通过迭代方法来求解，EM算法就是可以用于求解这个问题的一种迭代算法，下面给出EM算法的迭代过程：

首先选取初始值，记做 $\theta^{0}=(\pi^{0},p^{0},q^{0})$ ，第i次的迭代参数的估计值为 $\theta^{i}=(\pi^{i},p^{i},q^{i})$
E步：计算在模型参数 $\pi^{i}，p^{i}，q^{i}$ 下观测变量 $y_i$ 来源于硬币B的概率： $\mu^{i+1}=\frac{\pi^{i}(p^{i})^{y_i}(1-p^i)^{1-y_i}}{\pi^{i}(p^{i})^{y_i}(1-p^i)^{1-y_i}+(1-\pi^{i})(q^{i})^{y_i}(1-p^i)^{1-y_i}}$ 备注一下：这个公式的分母是 $P(Y|\theta)$ ，分子表示是来源与B硬币的概率。

M步：计算模型参数的新估计值： $\pi^{i+1}=\frac{1}{n}\sum_{j=1}^{n}\mu_{j}^{i+1}$ 因为B硬币A硬币出现正面的结果，所以A硬币概率就是 $\mu_{j}$ 的平均值。 $p^{i+1}=\frac{\sum_{j=1}^{n}\mu_{j}^{i+1}y_j}{\sum_{j=1}^{n}\mu_{j}^{i+1}}$ 分子乘以 $y_{i}$ ，所以其实是计算B硬币出现正面的概率。 $q^{i+1}=\frac{\sum_{j=1}^{n}(1-\mu_{j}^{i+1})y_j}{\sum_{j=1}^{n}(1-\mu_{j}^{i+1})}$ $(1-\mu_{j}^{i+1})$ 表示出现C硬币的概率。

闭环形成，从 $P(Y|\theta)$ 到 $\pi、p、q$ 一个闭环流程，接下来可以通过迭代法来做完成。针对上述例子，我们假设初始值为 $\pi^{0}=0.5，p^{0}=0.5，q^{0}=0.5$ ，因为对 $y_i=1$ 和 $y_i=0$ 均有 $\mu_j^{1}=0.5$ ，利用迭代公式计算得到 $\pi^{1}=0.5，p^{1}=0.6，q^{1}=0.6$ ，继续迭代得到最终的参数： $\widehat{\pi^{0}}=0.5，\widehat{p^{0}}=0.6，\widehat{q^{0}}=0.6$ 如果一开始初始值选择为： $\pi^{0}=0.4，p^{0}=0.6，q^{0}=0.7$ ，那么得到的模型参数的极大似然估计是 $\widehat{\pi}=0.4064，\widehat{p}=0.5368，\widehat{q}=0.6432$ ，这说明EM算法与初值的选择有关，选择不同的初值可能得到不同的参数估计值。

这个例子中你只观察到了硬币抛完的结果，并不了解A硬币抛完之后，是选择了B硬币抛还是C硬币抛，这时候概率模型就存在着隐含变量！

EM算法

输入：观测变量数据Y，隐变量数据Z，联合分布 $P(Y,Z|\theta)$ ，条件分布 $P(Z|Y,\theta)$ ；输出：模型参数 $\theta$

(1)选择参数的初值 $\theta^0$ ，开始迭代
(2) E步：记 $\theta^i$ 为第i次迭代参数 $\theta$ 的估计值，在第i+1次迭代的E步，计算 $\begin{aligned} Q(\theta,\theta^i)&=E_{Z}[logP(Y,Z|\theta)|Y,\theta^i]\\ &=\sum_{Z}logP(Y,Z|\theta)P(Z|Y,\theta^i) \end{aligned}$ 这里， $P(Z|Y,\theta^i)$ 是在给定观测数据Y和当前的参数估计 $\theta^i$ 下隐变量数据Z的条件概率分布；

(3) M步：求使 $Q(\theta,\theta^i)$ 极大化的 $\theta$ ，确定第i+1次迭代的参数的估计值 $\theta^{i+1}$ ， $\theta^{i+1}=arg \max \limits_{\theta}Q(\theta,\theta^{i})$ $Q(\theta,\theta^{i})$ 是EM算法的核心，称为Q函数(Q function)，这个是需要自己构造的。

(4) 重复第(2)步和第(3)步，直到收敛，收敛条件：

$||\theta^{i+1}-\theta^{i} || < \varepsilon_1$

或者：

$||Q(\theta^{i+1},\theta^{i})-Q(\theta^{i},\theta^{i})|| < \varepsilon_2$

收敛迭代就结束了。我们来拆解一下这个M步骤。

推导逼近

主要讲解Jensen不等式，这个公式在推导和收敛都用到，主要是如下的结论：

$f (x)$ 是凸函数 $\le E(f(x))$
$f (x)$ 是凹函数 $\ge E(f(x))$
推导出Em算法可以近似实现对观测数据的极大似然估计的办法是找到E步骤的下界，让下届最大，通过逼近的方式实现对观测数据的最大似然估计。统计学习基础中采用的是相减方式，我们来看下具体的步骤。

增加隐藏变量 $L(\theta)=\sum_{Z}logP(Y|Z,\theta)P(Z,\theta)$

则 $L(\theta)-L(\theta^{i})$ 为： $\begin{aligned} L(\theta)-L(\theta^{i})=log(\sum_{Z} P(Y|Z,\theta^i)\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Y|Z,\theta^i)})-L(\theta^{i})\\ \ge \sum_{Z} P(Y|Z,\theta^i)log(\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Y|Z,\theta^i)})-L(\theta^{i}) \end{aligned}$ $\ge$ 这一个步骤就是采用了凹函数的Jensen不等式做转换。因为 $Z$ 是隐藏变量，所以有 $\sum_{Z} P(Y|Z,\theta^i)==1，P(Y|Z,\theta^i)>0$ ，于是继续变：
$\begin{aligned} L(\theta)-L(\theta^{i})&=log(\sum_{Z} P(Y|Z,\theta^i)\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Y|Z,\theta^i)})-L(\theta^{i})\\ &\ge \sum_{Z} P(Z|Y,\theta^i)log(\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Z|Y,\theta^i)})-L(\theta^{i})\\ &=\sum_{Z} P(Z|Y,\theta^i)log(\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Z|Y,\theta^i)})-\sum_{Z} P(Z|Y,\theta^i)L(\theta^{i})\\ &= \sum_{Z} P(Z|Y,\theta^i)log(\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Z|Y,\theta^i) (P(Y|\theta^{i})}) \\ & \ge0 \end{aligned}$
也就是： $L(\theta)\ge L(\theta^{i})+ \sum_{Z} P(Z|Y,\theta^i)log(\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Y|Z,\theta^i) L(\theta^{i})})$ ，有下界，最大化下界，来得到近似值。这里有一个细节： $P(Y|Z,\theta^i)$ 变为 $P(Z|Y,\theta^i)$ ？如果要满足Jensen不等式的等号，则有： $\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Y|Z,\theta^i)} = c$ c为一个常数，而 $\sum_{Z}P(Y|Z,\theta^i)=1$ 则： $\begin{aligned} \sum_{Z}P(Y|Z,\theta)P(Z,\theta)= c\sum_{Z}P(Y|Z,\theta^i)&=c\\ &=\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Y|Z,\theta^i)}\\ P(Y|Z,\theta)=\frac{P(Y|Z,\theta)P(Z,\theta)}{\sum_{Z}P(Y|Z,\theta)P(Z,\theta)}=\frac{P(Y,Z,\theta)}{P(Y,\theta)}=P(Z|Y,\theta) \end{aligned}$

大家是不是很奇怪 $P(Y|Z,\theta)P(Z,\theta)$ 加上 $\sum$ 之后等于什么，其实有的博客这里使用 $P(Z,\theta) = P(Y^i,Z^i,\theta^i)$ 来替代 $P(Y|Z,\theta)$ 参与计算，这样 $\sum_{Z}P(Y^i,Z^i,\theta^i)$ ，这样就方便理解来了。

于是最大化如下：

$\begin{aligned} \theta^{i+1}&=arg \max_{\theta}\sum_{Z} P(Z|Y,\theta^i)log(\frac{P(Y|Z,\theta)P(Z,\theta)}{P(Z|Y,\theta^i)})\\ &=arg \max_{\theta}\sum_{Z} P(Z|Y,\theta^i)log(P(Y|Z,\theta)P(Z,\theta))\\ & =arg \max_{\theta}\sum_{Z} P(Z|Y,\theta^i)log(P(Y,Z|\theta))\\ &=arg \max_{\theta}Q(\theta,\theta^i) \end{aligned}$
其中 $l o g$ 分母提出来是关于 $Z$ 的 $\sum_{Z} P(Z|Y,\theta^i)logP(Z|Y,\theta^i)$ ，可以去掉。当然也有博客写的形式是： $\max_{\theta}\sum_{i=1}^{M}\sum_{Z^{i}} P(Z^{i}|Y^{i},\theta^i)log(P(Y^{i},Z^{i};\theta))\\$ 形式其实一样，表示的不一样而已。

证明收敛

我们知道已知观测数据的似然函数是 $P(Y,\theta)$ ，对数似然函数为： $\begin{aligned} L()=\sum_{i=1}^{M}logP(y^{i},\theta) =\sum_{i=1}^{M}log(\frac{P(y^i,Z|\theta)}{P(Z|y^i,\theta)})\\ =\sum_{i=1}^{M}logP(y^i,Z|\theta) - \sum_{i=1}^{M}logP(Z|y^i,\theta) \end{aligned}$ 要证明收敛，就证明单调递增， $\sum_{i=1}^{M}logP(y^{i},\theta^{j+1})>\sum_{i=1}^{M}logP(y^{i},\theta^{j})$ 由上文知道： $\begin{aligned} Q(\theta,\theta^i)=\sum_{Z}logP(Y,Z|\theta)P(Z|Y,\theta^i)\\ =\sum_{i=1}^{M}\sum_{Z^j}logP(y^i,Z^j|\theta)P(Z^j|y^i,\theta^i) \end{aligned}$ 我们构造一个函数 $H$ ，让他等于： $H(\theta,\theta^{i})=\sum_{i=1}^{M}\sum_{Z^j}log(P(Z|y^i,\theta)P(Z|y^i,\theta^i))$ 让 $Q(\theta,\theta^i)-H(\theta,\theta^{i})$ ： $\begin{aligned} Q(\theta,\theta^i)-H(\theta,\theta^{i})=\sum_{i=1}^{M}\sum_{Z^j}logP(y^i,Z^j|\theta)P(Z^j|y^i,\theta^i) - \sum_{i=1}^{M}\sum_{Z^j}log(P(Z^j|y^i,\theta)P(Z^j|y^i,\theta^i)) \\ =\sum_{i=1}^{M}\sum_{Z^j}log\bigg(P(y^i,Z^j|\theta)-P(Z^j|y^i,\theta)\bigg) \\ =\sum_{i=1}^{M}logP(y^{i},\theta) \end{aligned}$
所以：
$\sum_{i=1}^{M}logP(y^{i},\theta^{j+1})-\sum_{i=1}^{M}logP(y^{i},\theta^{j}) \\ = Q(\theta^{i+1},\theta^i)-H(\theta^{i+1},\theta^{i}) - (Q(\theta^{i},\theta^{i})-H(\theta^{i},\theta^{i}))\\ = Q(\theta^{i+1},\theta^i)- Q(\theta^{i},\theta^{i}) -( H(\theta^{i+1},\theta^{i}) - H(\theta^{i},\theta^{i}))$ 该公式左边已经被证明是大于0，证明右边： $H(\theta^{i+1},\theta^{i}) - H(\theta^{i},\theta^{i})<0$ ： $\begin{aligned} H(\theta^{i+1},\theta^{i}) - H(\theta^{i},\theta^{i}) =\sum_{Z^j}\bigg(log(\frac{P(Z^j|Y,\theta^{i+1})}{P(Z^j|Y,\theta^i)}) \bigg)P(Z^j|Y,\theta^i) \\ =log\bigg(\sum_{Z^j}\frac{P(Z^j|Y,\theta^{i+1})}{P(Z^j|Y,\theta^i)}P(Z^j|Y,\theta^i) \bigg)\\ =logP(Z|Y,\theta^{i+1})=log1=0 \end{aligned}$ 其中不等式是由于Jensen不等式，由此证明了 $\sum_{i=1}^{M}logP(y^{i},\theta^{j+1})>\sum_{i=1}^{M}logP(y^{i},\theta^{j})$ ，证明了EM算法的收敛性。但不能保证是全局最优，只能保证局部最优。

高斯混合分布

EM算法的一个重要应用场景就是高斯混合模型的参数估计。高斯混合模型就是由多个高斯模型组合在一起的混合模型（可以理解为多个高斯分布函数的线性组合，理论上高斯混合模型是可以拟合任意类型的分布），例如对于下图中的数据集如果用一个高斯模型来描述的话显然是不合理的。两个高斯模型可以拟合数据集，如图所示：

在这里插入图片描述
如果有多个高斯模型，公式表示为： $P(y|\theta)=\sum_{k=1}^{K}a_k\phi(y|\theta_{k}) \\ \phi(y|\theta_{k})=\frac{1}{\sqrt{2\pi}\delta_{k}}exp(-\frac{(y-\mu_{k})^2}{2 \delta_{k}^{2}}) \\ a_k>0,\sum a_k =1$ $\phi(y|\theta_{k})$ 表示为第k个高斯分布密度模型，定义如上，其中 $a_k$ 表示被选中的概率。在本次模型 $P(y|\theta)$ 中，观测数据是已知的，而观测数据具体来自哪个模型是未知的，有点像之前提过的三硬币模型，我们来对比一下，A硬币就像是概率 $a_k$ ，用来表明具体的模型，而B、C硬币就是具体的模型，只不过这里有很多个模型，不仅仅是B、C这两个模型。我们用 $\gamma_{jk}$ 来表示，则： $\gamma_{jk} = \begin{cases} 1 \text{第j个观测数据来源于第k个模型}\\ 0 \text{否则} \end{cases}$ 所以一个观测数据 $y_j$ 的隐藏数据 $(\gamma_{j1},\gamma_{j2},...,\gamma_{jk})$ ，那么完全似然函数就是：

$P(y,\gamma|\theta)= \prod_{k=1}^{K}\prod_{j=1}^{N}[a_{k}\phi(y|\theta_{k})]^{\gamma_{jk}}$
取对数之后等于：

$\begin{aligned} log(P(y,\gamma|\theta))&=log( \prod_{k=1}^{K}\prod_{j=1}^{N}[a_{k}\phi(y|\theta_{k})]^{\gamma_{jk}})\\ &=\sum_{K}^{k=1}\bigg(\sum_{j=1}^{N}(\gamma_{jk}) log(a_k)+\sum_{j=1}^{N}( \gamma_{jk})\bigg[log(\frac{1}{\sqrt{2\pi}})-log(\delta_{k})-\frac{(y_i-\mu_{k})^2}{2 \delta_{k}^{2}}\bigg]\bigg) \end{aligned}$
E 步： $\begin{aligned} Q(\theta.\theta^i) &= E[log(P(y,\gamma|\theta))]\\ &=\sum_{K}^{k=1}\bigg(\sum_{j=1}^{N}(E\gamma_{jk}) log(a_k)+\sum_{j=1}^{N}(E\gamma_{jk})\bigg[log(\frac{1}{\sqrt{2\pi}})-log(\delta_{k})-\frac{(y_i-\mu_{k})^2}{2 \delta_{k}^{2}}\bigg]\bigg) \end{aligned}$ 其中我们定义 $\hat{\gamma_{jk}}$ ： $\hat{\gamma_{jk}} = E(\gamma_{jk}|y,\theta)=\frac{a_k\phi(y_i|\theta_{k})}{\sum_{k=1}^{K}a_k\phi(y_i|\theta_{k}) }\\ j=1,2,..,N；k=1,2,...,K\\ n_k=\sum_{j=i}^{N}E\gamma_{jk}$ 于是化简得到： $\begin{aligned} Q(\theta.\theta^i) &= \sum_{K}^{k=1}\bigg(n_k log(a_k)+\sum_{j=1}^{N}(E\gamma_{jk})\bigg[log(\frac{1}{\sqrt{2\pi}})-log(\delta_{k})-\frac{(y_i-\mu_{k})^2}{2 \delta_{k}^{2}}\bigg]\bigg) \end{aligned}$
E 步在代码设计上只有 $\hat{\gamma_{jk}}$ 有用，用于M步的计算。

M步， $\theta^{i+1}=arg \max_{\theta}Q(\theta,\theta^i)$ 对 $Q(\theta,\theta^i)$ 求导，得到每个未知量的偏导，使其偏导等于0，求解得到： $\hat{\mu_k}=\frac{\sum_{j=1}^{N}\hat{\gamma_{jk}}y_i}{\sum_{j=1}^{N}\hat{\gamma_{jk}}} \\ \\ \hat{\delta_k}=\frac{\sum_{j=1}^{N}\hat{\gamma_{jk}}(y_i-\mu_k)^2}{\sum_{j=1}^{N}\hat{\gamma_{jk}}} \\ \\ \\ \hat{a_k}=\frac{\sum_{j=1}^{N}\hat{\gamma_{jk}} }{N}$ 给一个初始值，来回迭代就可以求得值内容。这一块主要用到了 $Q(\theta.\theta^i)$ 的导数，并且用到了E步的 $\hat{\gamma_{jk}}$ 。

混合高斯分布模型

EM算法更多是一种思想，用概率来解决问题的一种方法，具体的代码看自己选用模型，所以并没有通用的模型，本此代码主要是讲解混合高斯分布模型。这其中的M步完全按照了公式来计算。

import numpy as np
import random
import math
import time

'''
数据集：伪造数据集（两个高斯分布混合）
数据集长度：1000
------------------------------
运行结果：
----------------------------
the Parameters set is:
alpha0:0.3, mu0:0.7, sigmod0:-2.0, alpha1:0.5, mu1:0.5, sigmod1:1.0
----------------------------
the Parameters predict is:
alpha0:0.4, mu0:0.6, sigmod0:-1.7, alpha1:0.7, mu1:0.7, sigmod1:0.9
----------------------------
'''

def loadData(mu0, sigma0, mu1, sigma1, alpha0, alpha1):
    '''
    初始化数据集
    这里通过服从高斯分布的随机函数来伪造数据集
    :param mu0: 高斯0的均值
    :param sigma0: 高斯0的方差
    :param mu1: 高斯1的均值
    :param sigma1: 高斯1的方差
    :param alpha0: 高斯0的系数
    :param alpha1: 高斯1的系数
    :return: 混合了两个高斯分布的数据
    '''
    # 定义数据集长度为1000
    length = 1000

    # 初始化第一个高斯分布，生成数据，数据长度为length * alpha系数，以此来
    # 满足alpha的作用
    data0 = np.random.normal(mu0, sigma0, int(length * alpha0))
    # 第二个高斯分布的数据
    data1 = np.random.normal(mu1, sigma1, int(length * alpha1))

    # 初始化总数据集
    # 两个高斯分布的数据混合后会放在该数据集中返回
    dataSet = []
    # 将第一个数据集的内容添加进去
    dataSet.extend(data0)
    # 添加第二个数据集的数据
    dataSet.extend(data1)
    # 对总的数据集进行打乱（其实不打乱也没事，只不过打乱一下直观上让人感觉已经混合了
    # 读者可以将下面这句话屏蔽以后看看效果是否有差别）
    random.shuffle(dataSet)

    #返回伪造好的数据集
    return dataSet

# 高斯分布公式，没有什么特殊的
def calcGauss(dataSetArr, mu, sigmod):
    '''
    根据高斯密度函数计算值
    依据：“9.3.1 高斯混合模型” 式9.25
    注：在公式中y是一个实数，但是在EM算法中(见算法9.2的E步)，需要对每个j
    都求一次yjk，在本实例中有1000个可观测数据，因此需要计算1000次。考虑到
    在E步时进行1000次高斯计算，程序上比较不简洁，因此这里的y是向量，在numpy
    的exp中如果exp内部值为向量，则对向量中每个值进行exp，输出仍是向量的形式。
    所以使用向量的形式1次计算即可将所有计算结果得出，程序上较为简洁
    
    :param dataSetArr: 可观测数据集
    :param mu: 均值
    :param sigmod: 方差
    :return: 整个可观测数据集的高斯分布密度（向量形式）
    '''
    # 计算过程就是依据式9.25写的，没有别的花样
    result = (1 / (math.sqrt(2*math.pi)*sigmod**2)) * np.exp(-1 * (dataSetArr-mu) * (dataSetArr-mu) / (2*sigmod**2))
    # 返回结果
    return result


def E_step(dataSetArr, alpha0, mu0, sigmod0, alpha1, mu1, sigmod1):
    '''
    EM算法中的E步
    依据当前模型参数，计算分模型k对观数据y的响应度
    :param dataSetArr: 可观测数据y
    :param alpha0: 高斯模型0的系数
    :param mu0: 高斯模型0的均值
    :param sigmod0: 高斯模型0的方差
    :param alpha1: 高斯模型1的系数
    :param mu1: 高斯模型1的均值
    :param sigmod1: 高斯模型1的方差
    :return: 两个模型各自的响应度
    '''
    # 计算y0的响应度
    # 先计算模型0的响应度的分子
    gamma0 = alpha0 * calcGauss(dataSetArr, mu0, sigmod0)
    #print("gamma0=",gamma0.shape) # 1000, 维向量
    # 模型1响应度的分子
    gamma1 = alpha1 * calcGauss(dataSetArr, mu1, sigmod1)

    # 两者相加为E步中的分布
    sum = gamma0 + gamma1
    # 各自相除，得到两个模型的响应度
    gamma0 = gamma0 / sum
    gamma1 = gamma1 / sum

    # 返回两个模型响应度
    return gamma0, gamma1

def M_step(muo, mu1, gamma0, gamma1, dataSetArr):
    # 依据算法9.2计算各个值
    # 这里没什么花样，对照书本公式看看这里就好了
    
    # np.dot 点积：[1,2] [2,3] = [2,6]
    mu0_new = np.dot(gamma0, dataSetArr) / np.sum(gamma0)
    mu1_new = np.dot(gamma1, dataSetArr) / np.sum(gamma1)

    # math.sqrt  平方根 
    sigmod0_new = math.sqrt(np.dot(gamma0, (dataSetArr - muo)**2) / np.sum(gamma0))
    sigmod1_new = math.sqrt(np.dot(gamma1, (dataSetArr - mu1)**2) / np.sum(gamma1))

    alpha0_new = np.sum(gamma0) / len(gamma0)
    alpha1_new = np.sum(gamma1) / len(gamma1)

    # 将更新的值返回
    return mu0_new, mu1_new, sigmod0_new, sigmod1_new, alpha0_new, alpha1_new


## 训练主函数
def EM_Train(dataSetList, iter=500):
    '''
    根据EM算法进行参数估计
    算法依据“9.3.2 高斯混合模型参数估计的EM算法” 算法9.2
    :param dataSetList:数据集（可观测数据）
    :param iter: 迭代次数
    :return: 估计的参数
    '''
    # 将可观测数据y转换为数组形式，主要是为了方便后续运算
    dataSetArr = np.array(dataSetList)

    # 步骤1：对参数取初值，开始迭代
    alpha0 = 0.5
    mu0 = 0
    sigmod0 = 1
    alpha1 = 0.5
    mu1 = 1
    sigmod1 = 1

    # 开始迭代
    step = 0
    while (step < iter):
        # 每次进入一次迭代后迭代次数加1
        step += 1
        # 步骤2：E步：依据当前模型参数，计算分模型k对观测数据y的响应度
        gamma0, gamma1 = E_step(dataSetArr, alpha0, mu0, sigmod0, alpha1, mu1, sigmod1)
        # 步骤3：M步
        mu0, mu1, sigmod0, sigmod1, alpha0, alpha1 = M_step(mu0, mu1, gamma0, gamma1, dataSetArr)

    # 迭代结束后将更新后的各参数返回
    return alpha0, mu0, sigmod0, alpha1, mu1, sigmod1

if __name__ == '__main__':
    start = time.time()

    # 设置两个高斯模型进行混合，这里是初始化两个模型各自的参数
    # 见“9.3 EM算法在高斯混合模型学习中的应用”
    # alpha是“9.3.1 高斯混合模型” 定义9.2中的系数α
    # mu0是均值μ
    # sigmod是方差σ
    # 在设置上两个alpha的和必须为1，其他没有什么具体要求，符合高斯定义就可以
    
    alpha0 = 0.3  # 系数α
    mu0 = -2  # 均值μ
    sigmod0 = 0.5  # 方差σ

    alpha1 = 0.7  # 系数α
    mu1 = 0.5  # 均值μ
    sigmod1 = 1  # 方差σ

    # 初始化数据集
    dataSetList = loadData(mu0, sigmod0, mu1, sigmod1, alpha0, alpha1)

    #打印设置的参数
    print('---------------------------')
    print('the Parameters set is:')
    print('alpha0:%.1f, mu0:%.1f, sigmod0:%.1f, alpha1:%.1f, mu1:%.1f, sigmod1:%.1f' % (
        alpha0, alpha1, mu0, mu1, sigmod0, sigmod1
    ))

    # 开始EM算法，进行参数估计
    alpha0, mu0, sigmod0, alpha1, mu1, sigmod1 = EM_Train(dataSetList)

    # 打印参数预测结果
    print('----------------------------')
    print('the Parameters predict is:')
    print('alpha0:%.1f, mu0:%.1f, sigmod0:%.1f, alpha1:%.1f, mu1:%.1f, sigmod1:%.1f' % (
        alpha0, alpha1, mu0, mu1, sigmod0, sigmod1
    ))

    # 打印时间
    print('----------------------------')
    print('time span:', time.time() - start)

the Parameters set is:
alpha0:0.3, mu0:0.7, sigmod0:-2.0, alpha1:0.5, mu1:0.5, sigmod1:1.0
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
gamma0= (1000,)
/----------------------------
the Parameters predict is:
alpha0:0.4, mu0:0.6, sigmod0:-1.9, alpha1:0.7, mu1:0.6, sigmod1:0.8
/----------------------------
time span: 0.19019794464111328

E步主要计算内容

其中我们定义 $\hat{\gamma_{jk}}$ ： $\hat{\gamma_{jk}} = E(\gamma_{jk}|y,\theta)=\frac{a_k\phi(y_i|\theta_{k})}{\sum_{k=1}^{K}a_k\phi(y_i|\theta_{k}) }\\ j=1,2,..,N；k=1,2,...,K\\ n_k=\sum_{j=i}^{N}E\gamma_{jk}$

M步主要计算内容

这一步骤主要是对Q函数求导后的数据进行计算，利用了 E 步的 $\hat{\gamma_{jk}}$

import math
import copy
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
 
#生成随机数据，4个高斯模型
def generate_data(sigma,N,mu1,mu2,mu3,mu4,alpha):
    global X                  #可观测数据集
    X = np.zeros((N, 2))       # 初始化X，2行N列。2维数据，N个样本
    X=np.matrix(X)
    global mu                 #随机初始化mu1，mu2，mu3，mu4
    mu = np.random.random((4,2))
    mu=np.matrix(mu)
    global excep              #期望第i个样本属于第j个模型的概率的期望
    excep=np.zeros((N,4))
    global alpha_             #初始化混合项系数
    alpha_=[0.25,0.25,0.25,0.25]
    for i in range(N):
        if np.random.random(1) < 0.1:  # 生成0-1之间随机数
            X[i,:]  = np.random.multivariate_normal(mu1, sigma, 1)     #用第一个高斯模型生成2维数据
        elif 0.1 <= np.random.random(1) < 0.3:
            X[i,:] = np.random.multivariate_normal(mu2, sigma, 1)      #用第二个高斯模型生成2维数据
        elif 0.3 <= np.random.random(1) < 0.6:
            X[i,:] = np.random.multivariate_normal(mu3, sigma, 1)      #用第三个高斯模型生成2维数据
        else:
            X[i,:] = np.random.multivariate_normal(mu4, sigma, 1)      #用第四个高斯模型生成2维数据
 
    print("可观测数据：\n",X)       #输出可观测样本
    print("初始化的mu1，mu2，mu3，mu4：",mu)      #输出初始化的mu


# E 期望
#  \hat{\gamma_{jk}}
def e_step(sigma,k,N):
    global X
    global mu
    global excep
    global alpha_
    for i in range(N):
        denom=0
        for j in range(0,k):
            #  sigma.I 表示矩阵的逆矩阵
            # np.transpose ：矩阵转置   np.linalg.det():矩阵求行列式
            denom += alpha_[j]*  math.exp(-(X[i,:]-mu[j,:])*sigma.I*np.transpose(X[i,:]-mu[j,:]))  /np.sqrt(np.linalg.det(sigma))       #分母
        for j in range(0,k):
            numer = math.exp(-(X[i,:]-mu[j,:])*sigma.I*np.transpose(X[i,:]-mu[j,:]))/np.sqrt(np.linalg.det(sigma))        #分子
            excep[i,j]=alpha_[j]*numer/denom      #求期望
    print("隐藏变量：\n",excep)

    
def m_step(k,N):
    global excep
    global X
    global alpha_
    for j in range(0,k):
        denom=0   #分母
        numer=0   #分子
        for i in range(N):
            numer += excep[i,j]*X[i,:]
            denom += excep[i,j]
        mu[j,:] = numer/denom    #求均值
        alpha_[j]=denom/N        #求混合项系数

        #     #可视化结果
def plotShow():
    # 画生成的原始数据
    plt.subplot(221)
    plt.scatter(X[:,0].tolist(), X[:,1].tolist(),c='b',s=25,alpha=0.4,marker='o')    #T散点颜色，s散点大小，alpha透明度，marker散点形状
    plt.title('random generated data')
    #画分类好的数据
    plt.subplot(222)
    plt.title('classified data through EM')
    order=np.zeros(N)
    color=['b','r','k','y']
    for i in range(N):
        for j in range(k):
            if excep[i,j]==max(excep[i,:]):
                order[i]=j     #选出X[i,:]属于第几个高斯模型
            probility[i] += alpha_[int(order[i])]*math.exp(-(X[i,:]-mu[j,:])*sigma.I*np.transpose(X[i,:]-mu[j,:]))/(np.sqrt(np.linalg.det(sigma))*2*np.pi)    #计算混合高斯分布
        plt.scatter(X[i, 0], X[i, 1], c=color[int(order[i])], s=25, alpha=0.4, marker='o')      #绘制分类后的散点图
    #绘制三维图像
    ax = plt.subplot(223, projection='3d')
    plt.title('3d view')
    for i in range(N):
        ax.scatter(X[i, 0], X[i, 1], probility[i], c=color[int(order[i])])
    plt.show()

if __name__ == '__main__':
    iter_num=1000  #迭代次数
    N=500         #样本数目
    k=4            #高斯模型数
    probility = np.zeros(N)    #混合高斯分布
    u1=[5,35]
    u2=[30,40]
    u3=[20,20]
    u4=[45,15]
    sigma=np.matrix([[30, 0], [0, 30]])               #协方差矩阵
    alpha=[0.1,0.2,0.3,0.4]         #混合项系数
    generate_data(sigma,N,u1,u2,u3,u4,alpha)     #生成数据
    #迭代计算
    for i in range(iter_num):
        err=0     #均值误差
        err_alpha=0    #混合项系数误差
        Old_mu = copy.deepcopy(mu)
        Old_alpha = copy.deepcopy(alpha_)
        
        e_step(sigma,k,N)     # E步
        m_step(k,N)           # M步
        
        print("迭代次数:",i+1)
        print("估计的均值:",mu)
        print("估计的混合项系数:",alpha_)
        for z in range(k):
            err += (abs(Old_mu[z,0]-mu[z,0])+abs(Old_mu[z,1]-mu[z,1]))      #计算误差
            err_alpha += abs(Old_alpha[z]-alpha_[z])
        if (err<=0.001) and (err_alpha<0.001):     #达到精度退出迭代
            print(err,err_alpha)
            break

[[34.10078398 41.99939702]
[17.97196632 19.52471006]
[32.34127531 41.41768696]
[43.55623437 15.6460001 ]
[26.75126564 35.50622855]
[13.2335205 32.86586681]
[12.70538568 11.48725734]
[45.06316267 8.41118073]
[47.57630029 18.86976153]
[46.24701093 8.09261051]
[31.6856349 38.3816145 ]
[41.64126306 3.1628792 ]
[46.14857455 7.21170413]
[21.93536542 27.07271141]
[14.02413964 29.70759751]
[28.00761242 41.35510063]
[-1.48775397 40.97093856]
[48.65773485 16.15378227]
[22.48474941 27.32601426]
[50.67364683 13.88159845]
[37.79824479 11.1916526 ]
[37.72662908 36.09471673]
[27.49916272 40.96482288]
[28.19440799 16.28953534]
[39.80916916 8.45412261]
[48.62197899 26.63845404]
[41.0543496 7.4001902 ]
[28.9050116 44.98388131]
[24.17507743 16.95299992]
[16.94171114 34.58447536]
[34.608337 16.77946838]
[41.51172038 16.48020012]
[42.77898465 12.35588339]
[45.12250132 17.42877677]
[42.67598701 6.58295637]
[19.34219222 23.05409255]
[30.31108537 25.63345344]
[30.4130082 40.52392205]
[41.46644655 16.94538105]
[43.8056078 10.78415563]
[ 6.84141737 15.4165794 ]
[49.18263591 10.5222882 ]
[44.26373762 21.7874918 ]
[34.97029744 10.60403938]
[41.59429269 24.13482791]
[ 4.63391475 31.2073789 ]
[33.40104196 42.23381551]
[22.64400842 18.27278289]
[41.13874878 19.34870817]
[48.9296952 17.82749922]
[ 1.35837365 37.43428297]
[22.85980648 35.7114185 ]
[43.23194757 1.80047766]
[43.97224829 16.33780178]
[19.81974073 9.21257053]
[52.87644002 17.03347397]
[38.11937795 17.52775566]
[48.70514656 22.21187488]
[28.42904897 40.05690694]
[37.78525025 16.60381937]
[54.44856232 16.97833772]
[20.00792055 12.55996805]
[29.24710253 20.00953972]
[ 4.36216746 29.47967659]
[ 5.64290401 36.84855284]
[49.44880872 17.70366146]
[38.97919295 14.17682754]
[44.35047857 16.96203349]
[20.16578155 28.27155509]
[17.84314586 26.07604556]
[48.5369958 14.78976177]
[37.56664902 9.64744929]
[41.90612082 27.98063948]
[19.39379041 23.38263542]
[40.78311853 22.7069765 ]
[31.31100649 4.74654749]
[27.81850002 23.88740218]
[24.40281112 19.21007619]
[52.79444648 11.10367651]
[48.69136118 18.48706472]
[27.36179794 38.96839341]
[14.3955137 23.73250706]
[12.83638793 22.67206662]
[41.21627903 15.91466713]
[21.99014159 17.60699749]
[22.43105158 18.31396831]
[39.65773965 9.57539553]
[44.87559084 19.75504824]
[24.05663756 24.77172765]
[ 3.87301084 29.96607249]
[25.53648145 5.98228589]
[38.85213406 23.97987467]
[47.4766735 16.58538884]
[37.75225065 13.17503997]
[14.76537384 23.60775947]
[46.88185733 18.04404044]
[ 1.34308284 38.55982558]
[51.5799769 7.27271334]
[48.76146614 12.94900173]
[50.67032735 14.53052206]
[43.33943081 20.38314748]
[50.29498112 20.16756997]
[18.97445258 20.88689473]
[45.72879937 15.14860253]
[44.00399859 15.5586213 ]
[-4.55563736 27.92724986]
[38.93040102 15.69105206]
[49.34375964 12.73842381]
[44.71506179 16.2445931 ]
[41.99812117 16.20963758]
[45.80468869 6.82945586]
[21.62171107 13.85136938]
[37.90647348 16.27884234]
[18.30524585 24.91855222]
[46.37665861 7.71412768]
[48.60522007 21.0196075 ]
[19.74846478 27.93769448]
[29.4137209 37.82860776]
[14.74758561 24.42050311]
[11.17655821 17.80424563]
[38.65076781 3.01710606]
[43.72088098 18.15703023]
[19.7418687 19.31755897]
[53.27530711 17.87916791]
[25.43497442 28.87639188]
[48.88182112 16.57530126]
[ 9.11179244 34.60472006]
[26.9252331 17.07561473]
[36.18748449 17.53461272]
[44.43474823 14.12185961]
[19.06719824 19.67077106]
[45.44463698 18.19513106]
[44.24167965 14.24678861]
[37.04531475 15.51170619]
[46.56082908 16.95933005]
[47.62363791 14.43358361]
[ 6.77763595 28.0281931 ]
[14.31363065 33.04536861]
[30.91101949 44.57098876]
[21.51916492 26.69631943]
[20.82663718 26.60433864]
[40.26843255 10.57322165]
[50.5349425 9.64935112]
[50.3974125 30.13343422]
[18.90279678 18.37681572]
[20.31276064 26.8056216 ]
[43.58181405 15.3611326 ]
[49.37506208 8.14037624]
[ 2.79605425 30.40621777]
[40.00938791 8.38742237]
[47.06648275 19.08476278]
[ 3.39154166 36.30283386]
[26.56273594 20.95872678]
[37.85733711 44.63804562]
[50.46537897 30.58691539]
[25.88870663 45.27153535]
[38.35374961 11.15656457]
[42.79144313 16.74488319]
[11.35150002 35.95308486]
[ 8.01775495 20.17951082]
[40.05358933 14.59760428]
[42.28819614 29.31538571]
[ 5.71898699 48.49639906]
[55.43513184 0.98679793]
[24.36822266 32.23987359]
[30.83996661 34.40822302]
[26.75868065 21.9874932 ]
[49.76002442 15.45076927]
[44.74063527 27.85118268]
[ 8.38057535 36.68918615]
[12.82896812 18.08989977]
[39.98610321 24.20054852]
[42.86457138 5.27643928]
[46.09962776 17.91291264]
[47.19816647 6.52083557]
[17.33488067 29.76100635]
[38.45174576 12.54518776]
[48.70756491 12.71376596]
[46.36148576 18.05755857]
[37.47166178 18.92720756]
[48.87596147 22.12670257]
[49.57591528 15.8568224 ]
[41.6613632 43.66814863]
[16.84380237 19.89880443]
[13.69768659 21.43724681]
[48.8103184 19.02706345]
[25.19055568 19.21318208]
[52.45524162 21.14514719]
[48.92908397 12.14609127]
[41.45363577 20.60158731]
[46.54136475 12.15508256]
[16.32041319 26.76891283]
[19.17556278 39.88809292]
[27.7398478 35.26056078]
[39.53534295 24.1502645 ]
[23.6101855 24.64636073]
[41.07545864 14.45243467]
[45.5412231 12.03696757]
[15.74960785 9.18036224]
[42.66313033 18.15568087]
[51.82521638 10.03603077]
[-0.35168172 32.66732811]
[47.56228723 16.19234061]
[25.01004746 36.024799 ]
[39.01422316 11.64893375]
[49.57822452 6.50447041]
[50.22347651 8.6984849 ]
[49.40633658 10.5228405 ]
[52.98362025 14.95409286]
[29.55097051 45.21370681]
[38.70075406 25.64772518]
[27.04561708 37.20857784]
[12.33882511 23.57754988]
[26.66212044 48.28846805]
[52.90164768 7.1837463 ]
[46.74388686 3.66289806]
[25.06656585 16.56052017]
[25.43402908 46.22534413]
[45.96332029 11.3173738 ]
[29.85637153 51.65605238]
[27.07764357 37.39973973]
[46.47346365 12.4928957 ]
[13.96086543 18.11571644]
[ 2.00023769 34.75667918]
[29.46834876 29.7450956 ]
[50.40919421 12.3811701 ]
[16.3574766 29.32949119]
[42.57880543 17.92591292]
[29.29295357 21.90832228]
[ 9.12679085 34.57291242]
[47.98138458 19.63154583]
[14.15676853 33.0779678 ]
[44.31645946 9.1684328 ]
[13.05676502 21.49522895]
[14.93788973 29.12945902]
[15.70135648 12.07938282]
[ 3.91738347 39.61418558]
[46.15150294 9.26044012]
[18.62293325 25.83742768]
[22.28719741 19.12155356]
[35.46021241 15.66913954]
[22.91646716 16.309418 ]
[40.9333429 26.38609539]
[20.61107052 18.47615901]
[41.99606722 7.94909265]
[24.55782096 31.00108147]
[40.59990055 15.8564243 ]
[39.44811928 19.05314726]
[30.52537538 35.1964212 ]
[43.84205217 14.66079242]
[52.76015224 22.27392048]
[ 0.53635851 33.2155064 ]
[44.88162761 18.73865156]
[44.44014864 21.48709943]
[47.82606872 10.38403453]
[36.26157604 37.82367763]
[43.62612613 14.47178666]
[31.02987893 37.47201458]
[ 8.6870975 28.72764772]
[47.03787343 10.6021404 ]
[-2.2273671 31.534357 ]
[-5.5527884 35.78317105]
[33.97410704 18.27894531]
[35.08805903 36.54684565]
[47.11719623 12.60702487]
[18.59791274 14.80784179]
[16.88512604 36.87556821]
[-2.46728743 31.69879922]
[10.2195111 28.01834712]
[17.14716289 26.71018422]
[15.87912908 15.55351361]
[28.07369566 32.33877451]
[37.92413287 15.7082284 ]
[47.21946769 11.55866576]
[22.04470036 29.11036652]
[47.56019889 7.58746674]
[46.80805009 15.61906449]
[20.87034473 16.54594974]
[10.83385054 23.77147518]
[47.22374574 15.79694751]
[ 7.10622286 38.56567066]
[39.1541188 23.47717576]
[18.90596043 14.20053477]
[47.18290947 14.80314809]
[16.27962629 21.10193262]
[19.48056864 29.96877045]
[41.07311764 14.65460825]
[51.49046834 12.43188518]
[36.13700081 7.18562116]
[26.31326858 39.45031334]
[46.29175628 9.84302885]
[ 8.18468255 32.50658552]
[47.6801346 15.10827806]
[38.3127991 22.32224051]
[19.87856263 16.29540339]
[35.93030051 46.78499113]
[50.54193438 16.49593679]
[26.98983193 39.50752382]
[41.10864415 8.75050624]
[-0.59134994 36.04513784]
[32.76561906 39.77393837]
[17.47044798 16.49877386]
[22.50429257 41.66412521]
[37.70944163 49.69855197]
[43.86133668 12.2038014 ]
[22.17456317 14.30107001]
[16.30903718 17.13935 ]
[24.6073783 16.97211229]
[40.30644112 4.22253284]
[-2.84953878 43.48700219]
[32.91453906 44.90119634]
[46.44308873 18.02404918]
[37.3836467 14.75452789]
[47.33363318 20.36012798]
[41.74642063 27.03559179]
[29.31598311 41.93510181]
[48.22347798 21.62569022]
[ 3.17735918 29.25572957]
[31.6461558 20.78888781]
[33.50294755 14.46878239]
[18.9812725 22.14299312]
[43.8738558 12.67516662]
[57.97405644 23.14127176]
[33.65001616 44.37996747]
[40.08608404 34.53885576]
[48.11079596 7.78926236]
[33.82549885 42.27612509]
[20.86273026 30.79132298]
[18.70056825 20.62206572]
[33.15880447 44.88629903]
[37.4362346 37.41992638]
[ 3.81016154 39.48962836]
[47.79564803 21.70275157]
[39.0008174 21.08286979]
[39.67007978 15.6638161 ]
[50.3540132 6.97142327]
[51.22993184 20.86038852]
[25.34780084 36.04993353]
[25.8099056 16.31838414]
[17.87456001 17.24086795]
[42.18432075 19.49102872]
[ 0.77389513 35.73945298]
[28.84158093 25.9747731 ]
[44.56458945 16.8096007 ]
[16.48325211 12.82712674]
[55.54813167 10.13983485]
[44.84227321 20.04591581]
[39.9303047 14.88429312]
[46.73229805 18.87442115]
[44.06042308 17.75104303]
[48.43173896 15.30141463]
[ 9.28491339 40.59064676]
[23.91953267 21.67275322]
[59.15488587 15.28271498]
[28.2029577 28.71580638]
[39.84986713 34.12515617]
[31.4900022 35.49798299]
[52.43744267 7.0471627 ]
[18.09728084 58.96694133]
[53.03410082 9.34412334]
[21.76734261 3.69027898]
[ 1.8458268 36.28669624]
[22.7322565 12.94515186]
[12.95886653 21.36141223]
[55.62274463 18.8514422 ]
[33.72616719 26.7407725 ]
[52.61860902 14.89879178]
[48.22746409 15.64536993]
[17.88394301 12.51091537]
[50.26283254 22.46648565]
[44.72663488 19.76503597]
[24.40311185 14.73062202]
[27.19901717 43.47928673]
[43.05060849 13.06655809]
[44.77230721 18.24725475]
[47.75525156 15.90833203]
[31.79166009 48.4449465 ]
[37.22065533 37.27995287]
[ 5.8327125 42.18174967]
[40.84405302 37.99263624]
[50.31672717 5.0201574 ]
[19.56157411 9.62681703]
[17.65180009 26.36157925]
[50.47905327 12.56459176]
[43.72288064 17.30236971]
[32.58711875 41.67314512]
[28.0813277 43.36109863]
[49.24188116 15.69614044]
[44.53679433 23.40222034]
[24.55634024 44.41905486]
[24.19807813 32.17738141]
[51.44755427 19.31466872]
[48.37459525 15.61966358]
[37.19591127 18.58052694]
[35.03470148 37.96630752]
[55.94714181 17.28201297]
[13.11359375 10.95151261]
[20.62450402 16.97277941]
[29.51375629 26.29389605]
[ 5.81029283 40.88280446]
[43.18504329 15.81468197]
[12.85033004 39.56089251]
[40.95846471 9.10179159]
[24.629248 20.91898324]
[36.43034188 43.2707104 ]
[24.58811257 48.16267479]
[27.1293558 45.31207983]
[37.14752856 16.18811707]
[15.2655828 23.78727062]
[17.96962961 23.15933344]
[41.67960922 18.31257833]
[23.25010001 21.61212361]
[21.06446702 47.65133533]
[46.58973596 10.44750494]
[ 3.07615683 36.82828907]
[50.9135076 3.37674452]
[40.34920095 15.44876942]
[35.89000167 13.66760966]
[42.04094777 24.76482129]
[17.52525829 17.42764559]
[12.31690045 9.24433637]
[26.58200859 8.23591132]
[41.63966598 8.21667593]
[20.17877755 11.99634565]
[61.67708583 19.70748798]
[39.39037903 11.41986853]
[22.41528605 19.21555276]
[42.81675312 15.6921447 ]
[40.99616294 19.58223833]
[ 4.84589475 42.2913667 ]
[47.36320583 22.13312622]
[52.32460463 17.48055118]
[ 0.26061262 31.53116015]
[37.73788355 33.97179794]
[58.36581659 16.59819707]
[37.90636038 38.94282108]
[-2.08168895 26.54818442]
[26.74718411 22.59862955]
[ 9.43625799 34.59995523]
[50.25594578 21.10792157]
[43.74002138 0.81621241]
[49.24369694 9.0857212 ]
[38.46565583 12.40154322]
[42.66611695 14.7283639 ]
[32.86484795 41.9641505 ]
[10.03732253 27.4766204 ]
[44.17206352 18.83111666]
[50.19790216 28.45569455]
[10.62002631 46.26260766]
[31.48815931 33.47984284]
[46.75121958 17.81070944]
[34.39886357 4.23348705]
[28.85252696 35.99002672]
[18.61944798 18.18903209]
[22.88495677 26.06605579]
[40.56233268 15.8406341 ]
[39.45243605 15.0994373 ]
[42.29944358 25.88846118]
[42.47641326 11.39220724]
[ 6.6793156 35.23439658]
[46.50270477 16.2841948 ]
[41.46867499 17.08656174]
[11.15994409 27.95130881]
[18.2132395 14.15072105]
[15.5741201 38.92002488]
[ 2.80455719 36.14087444]
[26.16868621 24.99967331]
[43.25652286 18.74812046]
[32.49424606 42.57789034]
[46.69719091 17.97622407]
[42.59649352 6.88890803]
[29.69650488 50.13363846]
[38.32543452 19.95682787]
[37.84187112 16.8230419 ]
[33.55352019 3.02575625]
[43.71207664 10.60423708]
[18.90667597 24.6764515 ]
[41.53240143 13.88091882]
[-0.61351581 39.39421656]
[40.45059704 7.36019341]
[31.43807884 18.12233643]
[23.86321692 18.36065462]
[13.82747253 25.04157229]
[42.23238901 18.73489884]
[42.14302727 18.98447353]
[34.92707606 35.73705217]
[59.57320284 22.57352783]
[28.0682902 42.716359 ]
[48.18341538 10.49951822]
[35.93596481 13.21017045]
[37.44002969 45.99082288]
[34.38998009 38.6183042 ]
[47.86510206 13.87264923]
[26.725038 31.32641901]
[49.08552279 22.01980689]
[42.26343175 17.49672525]
[19.17560368 16.79761814]
[10.10778035 41.93231823]
[19.84391871 16.86139614]
[39.46129257 9.06815109]]
初始化的mu1，mu2，mu3，mu4： [[0.28947187 0.22554279]
[0.01447452 0.5124134 ]
[0.10294728 0.15686306]
[0.69678914 0.89667212]]
隐藏变量：
[[0.05365836 0.06384243 0.02906718 0.85343203]
[0.15315695 0.15935179 0.11238167 0.5751096 ]
[0.05700052 0.06926734 0.03164486 0.84208727]
…
[0.08445446 0.15580072 0.06167289 0.69807193]
[0.16159985 0.15439595 0.11726983 0.56673436]
[0.15675736 0.09005354 0.09236928 0.66081983]]
迭代次数: 1
估计的均值: [[31.65316945 19.89603321]
[25.63672599 23.62681403]
[29.12939399 20.41782732]
[33.98478206 22.99299238]]
估计的混合项系数: [0.11594808400275008, 0.10335377348283055, 0.0726856257795488, 0.7080125167348705]
隐藏变量：
[[1.86457199e-03 2.85967847e-02 1.33872895e-03 9.68199914e-01]
[2.30863410e-02 8.53065626e-01 1.14413670e-01 9.43436345e-03]
[2.63762082e-03 7.08943099e-02 2.49520575e-03 9.23972863e-01]
…
[4.38501025e-06 9.99763774e-01 1.77933738e-04 5.39068213e-05]
[7.35055781e-02 6.60969408e-01 2.42338324e-01 2.31866902e-02]
[4.12377547e-01 2.04232730e-04 3.82123353e-02 5.49205885e-01]]
迭代次数: 2
估计的均值: [[37.93594076 11.89499695]
[14.94926956 29.41774966]
[23.87998539 15.98752609]
[42.25358839 20.87200857]]
估计的混合项系数: [0.08962512910647233, 0.3122664181489847, 0.0458092402134442, 0.5522992125310985]
隐藏变量：
[[1.45579396e-07 2.73216520e-01 7.88678857e-06 7.26775448e-01]
[1.19872635e-06 4.83243646e-01 5.16755073e-01 8.30533287e-08]
[6.16268676e-08 8.69089952e-01 1.48147901e-05 1.30895172e-01]
…
[6.20162624e-23 1.00000000e+00 1.91685195e-11 2.97877121e-19]
[2.69593702e-05 2.74863433e-02 9.72486047e-01 6.50327626e-07]
[9.39414030e-01 9.36434253e-15 4.19840895e-05 6.05439856e-02]]
迭代次数: 3
估计的均值: [[42.37627335 9.36451582]
[16.89398296 33.74959707]
[21.50625389 16.41879915]
[44.36491724 20.05770285]]
估计的混合项系数: [0.13721527301326916, 0.3192554988084501, 0.12350162509023202, 0.4200276030880484]
隐藏变量：
[[3.12549924e-12 9.99213728e-01 1.22839180e-07 7.86149425e-04]
[1.76764670e-10 6.08404926e-03 9.93915950e-01 5.77003877e-10]
[4.05315118e-13 9.99946530e-01 1.40146325e-07 5.33302850e-05]
…
[6.93222853e-30 1.00000000e+00 8.30495647e-11 6.90625379e-23]
[8.41518850e-09 1.58636581e-04 9.99841350e-01 5.27312644e-09]
[9.68388979e-01 1.93080615e-16 4.12504064e-06 3.16068958e-02]]
迭代次数: 4
估计的均值: [[43.96279316 9.55068753]
[18.58478664 36.00242554]
[20.55945572 17.68515427]
[44.81944502 19.37490103]]
估计的混合项系数: [0.17185168724460861, 0.3074291907473439, 0.16934105280452583, 0.351378069203522]
隐藏变量：
[[1.26665428e-13 9.99990181e-01 3.42010021e-08 9.78512085e-06]
[8.56718548e-12 2.94268706e-04 9.99705731e-01 1.06661317e-10]
[1.80026578e-14 9.99999086e-01 5.52014979e-08 8.59295700e-07]
…
[3.34757769e-31 9.99999998e-01 1.57716450e-09 6.28738008e-24]
[6.73977093e-10 8.89987050e-06 9.99991098e-01 1.63188631e-09]
[9.56871950e-01 5.23428791e-17 1.05708319e-06 4.31269930e-02]]
迭代次数: 5
估计的均值: [[44.3557747 9.86047413]
[19.31929982 37.09815726]
[20.11945741 18.66625064]
[44.88231194 19.04166866]]
估计的混合项系数: [0.1844119832193132, 0.2900644571819827, 0.19672913292266966, 0.32879442667603415]
隐藏变量：
[[6.89800882e-14 9.99998169e-01 4.27354628e-08 1.78847723e-06]
[4.16967660e-12 5.61096046e-05 9.99943890e-01 6.51173638e-11]
[1.05204618e-14 9.99999741e-01 7.95172047e-08 1.79613630e-07]
…
[3.15915229e-31 9.99999987e-01 1.29045243e-08 3.38633772e-24]
[4.09940187e-10 1.92483548e-06 9.99998073e-01 1.33931556e-09]
[9.47725564e-01 1.91542764e-17 4.08579022e-07 5.22740273e-02]]
迭代次数: 6
估计的均值: [[44.46539955 10.04135093]
[19.69535999 37.70332355]
[19.82497396 19.28491523]
[44.88414162 18.91438842]]
估计的混合项系数: [0.18995262821351014, 0.27699894857210305, 0.21264147641363979, 0.3204069468007472]
隐藏变量：
[[5.85113374e-14 9.99999082e-01 5.45832562e-08 8.63774619e-07]
[3.45561681e-12 2.17989925e-05 9.99978201e-01 5.47493727e-11]
[9.35986763e-15 9.99999797e-01 1.09815570e-07 9.32516225e-08]
…
[4.11144706e-31 9.99999952e-01 4.81717031e-08 2.98562705e-24]
[3.86179802e-10 8.15007830e-07 9.99999183e-01 1.33392377e-09]
[9.43893505e-01 9.74540406e-18 2.02833939e-07 5.61062925e-02]]
迭代次数: 7
估计的均值: [[44.50082855 10.14475691]
[20.04384496 38.12278518]
[19.46089664 19.72459962]
[44.87762301 18.89435699]]
估计的混合项系数: [0.19320256483830528, 0.26639964137438127, 0.22388405490169302, 0.3165137388856205]
隐藏变量：
[[4.82182890e-14 9.99999394e-01 5.21341944e-08 5.53937350e-07]
[3.21142567e-12 1.09341973e-05 9.99989066e-01 4.98368138e-11]
[8.10051153e-15 9.99999823e-01 1.13942694e-07 6.34166756e-08]
…
[5.59822405e-31 9.99999856e-01 1.43817067e-07 3.38500246e-24]
[4.00442972e-10 4.48422050e-07 9.99999550e-01 1.37760851e-09]
[9.43616112e-01 6.60605412e-18 9.73807060e-08 5.63837906e-02]]
迭代次数: 8
估计的均值: [[44.51294892 10.21402556]
[20.42147939 38.46538952]
[19.08715875 20.08667662]
[44.8699599 18.89649596]]
估计的混合项系数: [0.19580352468051282, 0.2574765384508134, 0.232962652015997, 0.3137572848526768]
隐藏变量：
[[3.76353825e-14 9.99999586e-01 4.28522928e-08 3.71445831e-07]
[3.12305135e-12 6.10569437e-06 9.99993894e-01 4.70329401e-11]
[6.66992385e-15 9.99999853e-01 1.02208497e-07 4.50817376e-08]
…
[7.85012587e-31 9.99999607e-01 3.93168371e-07 4.21026049e-24]
[4.30438953e-10 2.75967508e-07 9.99999722e-01 1.45067960e-09]
[9.44227384e-01 5.29607071e-18 4.70061383e-08 5.57725691e-02]]
迭代次数: 9
估计的均值: [[44.51659086 10.26681568]
[20.86320703 38.77253123]
[18.69400516 20.42946739]
[44.86354283 18.89699992]]
估计的混合项系数: [0.1979459404889354, 0.24924043990967576, 0.24128415618953222, 0.3115294634118566]
隐藏变量：
[[2.75232341e-14 9.99999728e-01 3.17535046e-08 2.40476083e-07]
[3.10654268e-12 3.54197403e-06 9.99996458e-01 4.52787040e-11]
[5.18591272e-15 9.99999885e-01 8.34133368e-08 3.11294393e-08]
…
[1.15823784e-30 9.99998893e-01 1.10736899e-06 5.58975265e-24]
[4.73670829e-10 1.78799168e-07 9.99999819e-01 1.55745993e-09]
[9.44786821e-01 4.84826810e-18 2.18795988e-08 5.52131572e-02]]
迭代次数: 10
估计的均值: [[44.51565341 10.30842795]
[21.43276391 39.07327891]
[18.23994635 20.78992309]
[44.85816428 18.88697641]]
估计的混合项系数: [0.19965596054308057, 0.24089341257986294, 0.24975523521632426, 0.30969539166073223]
隐藏变量：
[[1.80439987e-14 9.99999839e-01 2.03651273e-08 1.40567770e-07]
[3.14552243e-12 2.00908389e-06 9.99997991e-01 4.43775932e-11]
[3.67185567e-15 9.99999920e-01 6.01826568e-08 1.97019015e-08]
…
[1.89337958e-30 9.99996350e-01 3.65043303e-06 8.20726494e-24]
[5.37076656e-10 1.16466278e-07 9.99999881e-01 1.72413708e-09]
[9.44939924e-01 5.13442016e-18 9.04154556e-09 5.50600669e-02]]
迭代次数: 11
估计的均值: [[44.51044378 10.33983496]
[22.06860091 39.34021697]
[17.79698777 21.15088757]
[44.85355819 18.85827476]]
估计的混合项系数: [0.20090931105285778, 0.2331030840659099, 0.25778076668923877, 0.3082068381919933]
隐藏变量：
[[1.13438434e-14 9.99999909e-01 1.26321625e-08 7.82249456e-08]
[3.26255045e-12 1.16901176e-06 9.99998831e-01 4.45545091e-11]
[2.51152046e-15 9.99999946e-01 4.22119458e-08 1.19556436e-08]
…
[3.32225956e-30 9.99987290e-01 1.27102415e-05 1.27386639e-23]
[6.23789697e-10 7.88388525e-08 9.99999919e-01 1.95931797e-09]
[9.44405981e-01 6.11986530e-18 3.69829549e-09 5.55940152e-02]]
迭代次数: 12
估计的均值: [[44.50038062 10.35961563]
[22.76929317 39.55082345]
[17.3679637 21.54275188]
[44.85193968 18.81389727]]
估计的混合项系数: [0.2016000793485042, 0.2255791330762731, 0.2656444273257188, 0.3071763602495035]
隐藏变量：
[[6.90389153e-15 9.99999950e-01 8.04169460e-09 4.21035382e-08]
[3.47131685e-12 7.15061911e-07 9.99999285e-01 4.57375716e-11]
[1.67375693e-15 9.99999962e-01 3.04651682e-08 7.05697508e-09]
…
[6.30756141e-30 9.99950747e-01 4.92531749e-05 2.11081306e-23]
[7.47091497e-10 5.73417347e-08 9.99999940e-01 2.29239744e-09]
[9.43292809e-01 8.51653238e-18 1.46972428e-09 5.67071894e-02]]
迭代次数: 13
估计的均值: [[44.48151997 10.36648065]
[23.73843833 39.77616388]
[16.85545872 22.07914985]
[44.85412021 18.75909663]]
估计的混合项系数: [0.20173564954496506, 0.2155247715339381, 0.27605908657628914, 0.3066804923448076]
隐藏变量：
[[3.56081176e-15 9.99999976e-01 4.82714071e-09 1.93334246e-08]
[3.87845045e-12 3.85933627e-07 9.99999614e-01 4.87834910e-11]
[9.77595119e-16 9.99999975e-01 2.14974401e-08 3.66933233e-09]
…
[1.58110864e-29 9.99684272e-01 3.15727615e-04 4.55604755e-23]
[9.75270740e-10 3.93485647e-08 9.99999957e-01 2.89582686e-09]
[9.41847479e-01 1.45270003e-17 4.47667275e-10 5.81525204e-02]]
迭代次数: 14
估计的均值: [[44.43908486 10.35888692]
[25.3551494 40.08155987]
[16.17698522 22.89592521]
[44.861286 18.68629281]]
估计的混合项系数: [0.2015024059465105, 0.19976276060095488, 0.292177097022309, 0.30655773643022516]
隐藏变量：
[[1.33132745e-15 9.99999991e-01 2.46484690e-09 6.34314391e-09]
[4.92836796e-12 1.37839339e-07 9.99999862e-01 5.68653803e-11]
[4.49457067e-16 9.99999984e-01 1.40863196e-08 1.47552851e-09]
…
[8.36106857e-29 9.93716355e-01 6.28364485e-03 1.95352368e-22]
[1.55313832e-09 2.04935868e-08 9.99999974e-01 4.30630145e-09]
[9.40168904e-01 3.52977402e-17 7.95846430e-11 5.98310963e-02]]
迭代次数: 15
估计的均值: [[44.32409353 10.32999173]
[27.66608038 40.27920082]
[15.44236836 23.9050458 ]
[44.88203118 18.57037908]]
估计的混合项系数: [0.20129542511928245, 0.18166595193465496, 0.3105827951405498, 0.3064558278055118]
隐藏变量：
[[4.51409792e-16 9.99999997e-01 1.24644519e-09 1.73877405e-09]
[8.03898336e-12 3.47797028e-08 9.99999965e-01 7.38305901e-11]
[2.04349425e-16 9.99999990e-01 9.86649662e-09 5.35594454e-10]
…
[9.77080058e-28 7.06305656e-01 2.93694344e-01 1.48599261e-21]
[3.29104016e-09 8.73391250e-09 9.99999980e-01 7.50213128e-09]
[9.38637029e-01 1.50407536e-16 9.72404747e-12 6.13629712e-02]]
迭代次数: 16
估计的均值: [[44.11021577 10.27957514]
[29.29752543 40.26127808]
[15.06648473 24.65560201]
[44.91815172 18.45347115]]
估计的混合项系数: [0.2007998901723772, 0.1698846742772788, 0.32170723450813465, 0.3076082010422088]
隐藏变量：
[[2.71946365e-16 9.99999998e-01 1.13633387e-09 8.23533469e-10]
[1.48193003e-11 1.39371068e-08 9.99999986e-01 9.03465887e-11]
[1.53052080e-16 9.99999989e-01 1.10487251e-08 3.07051987e-10]
…
[1.53845053e-27 9.64456987e-02 9.03554301e-01 1.23355645e-21]
[7.13680630e-09 5.15356777e-09 9.99999976e-01 1.12740769e-08]
[9.38938337e-01 4.49690948e-16 2.39370099e-12 6.10616634e-02]]
迭代次数: 17
估计的均值: [[43.90799179 10.23217418]
[29.99645232 40.24409958]
[14.97380216 25.02371628]
[44.97768692 18.38637751]]
估计的混合项系数: [0.198899008582842, 0.16362705463830654, 0.3271201171767318, 0.3103538196021196]
隐藏变量：
[[2.35419007e-16 9.99999998e-01 1.32275683e-09 6.05451409e-10]
[2.31070585e-11 9.10434020e-09 9.99999991e-01 9.29923341e-11]
[1.47439497e-16 9.99999986e-01 1.39034953e-08 2.43794082e-10]
…
[1.47337407e-27 2.52402799e-02 9.74759720e-01 6.60131784e-22]
[1.18174727e-08 3.97922215e-09 9.99999971e-01 1.27769033e-08]
[9.40446503e-01 6.71889803e-16 1.34650460e-12 5.95534967e-02]]
迭代次数: 18
估计的均值: [[43.73712019 10.18955783]
[30.29694755 40.23075774]
[14.95075648 25.18962827]
[45.04836594 18.33624019]]
估计的混合项系数: [0.19669177551084918, 0.16074450952333744, 0.32961186348714566, 0.31295185147866755]
隐藏变量：
[[2.23803200e-16 9.99999998e-01 1.46943505e-09 5.04723829e-10]
[3.16826574e-11 7.57066367e-09 9.99999992e-01 8.72105829e-11]
[1.48301141e-16 9.99999984e-01 1.59315299e-08 2.09085982e-10]
…
[1.61758933e-27 1.37127341e-02 9.86287266e-01 4.31869585e-22]
[1.65070469e-08 3.55665326e-09 9.99999967e-01 1.25975803e-08]
[9.42024396e-01 7.85681846e-16 1.05123940e-12 5.79756036e-02]]
迭代次数: 19
估计的均值: [[43.58723142 10.15440641]
[30.40662594 40.23023582]
[14.94813378 25.25689745]
[45.11717428 18.29185387]]
估计的混合项系数: [0.1946357821395416, 0.15962401271301882, 0.3305547576731, 0.3151854474743397]
隐藏变量：
[[2.21597201e-16 9.99999998e-01 1.55156613e-09 4.41555515e-10]
[4.05803293e-11 7.03370734e-09 9.99999993e-01 7.91005250e-11]
[1.51582420e-16 9.99999983e-01 1.70002997e-08 1.84109941e-10]
…
[1.92614657e-27 1.08859597e-02 9.89114040e-01 3.20260312e-22]
[2.11387584e-08 3.39160958e-09 9.99999964e-01 1.17592228e-08]
[9.43394647e-01 8.08067697e-16 9.45055278e-13 5.66053532e-02]]
迭代次数: 20
估计的均值: [[43.45473785 10.12788427]
[30.44574987 40.23124592]
[14.94909103 25.28562747]
[45.18028862 18.25146157]]
估计的混合项系数: [0.19279393655512383, 0.15921316638064156, 0.33088217127266967, 0.3171107257915648]
隐藏变量：
[[2.23878788e-16 9.99999998e-01 1.59422986e-09 3.95000563e-10]
[5.00328374e-11 6.84885976e-09 9.99999993e-01 7.14436569e-11]
[1.56424195e-16 9.99999982e-01 1.75272865e-08 1.64499584e-10]
…
[2.34729364e-27 9.97758354e-03 9.90022416e-01 2.52748855e-22]
[2.58849145e-08 3.33461667e-09 9.99999960e-01 1.08358212e-08]
[9.44521690e-01 8.01497696e-16 8.94003186e-13 5.54783101e-02]]
迭代次数: 21
估计的均值: [[43.33632476 10.10904766]
[30.46084276 40.23034995]
[14.94947544 25.30041093]
[45.23672955 18.21455063]]
估计的混合项系数: [0.1911895249313257, 0.15907476157654166, 0.33097108537334624, 0.3187646281187865]
隐藏变量：
[[2.28924606e-16 9.99999998e-01 1.61756303e-09 3.58208928e-10]
[6.02527648e-11 6.80208761e-09 9.99999993e-01 6.49610430e-11]
[1.62587372e-16 9.99999982e-01 1.78036963e-08 1.48663268e-10]
…
[2.86212379e-27 9.60925187e-03 9.90390748e-01 2.06651639e-22]
[3.08974329e-08 3.32717846e-09 9.99999956e-01 1.00132607e-08]
[9.45451762e-01 7.91161298e-16 8.60556983e-13 5.45482383e-02]]
迭代次数: 22
估计的均值: [[43.22921326 10.0965657 ]
[30.46768182 40.22780556]
[14.9488227 25.31008279]
[45.28707373 18.18090294]]
估计的混合项系数: [0.18981460809415082, 0.1590409262643288, 0.33096357124902875, 0.32018089439249126]
隐藏变量：
[[2.36052592e-16 9.99999998e-01 1.63180021e-09 3.28177067e-10]
[7.13866486e-11 6.81219654e-09 9.99999993e-01 5.95927004e-11]
[1.69971247e-16 9.99999982e-01 1.79676738e-08 1.35669763e-10]
…
[3.46961268e-27 9.41431420e-03 9.90585686e-01 1.73040816e-22]
[3.62660951e-08 3.34248211e-09 9.99999951e-01 9.31579594e-09]
[9.46245027e-01 7.82570267e-16 8.33745116e-13 5.37549731e-02]]
迭代次数: 23
估计的均值: [[43.13125256 10.08930782]
[30.4715465 40.2243953 ]
[14.94741595 25.3176842 ]
[45.33240999 18.15026971]]
估计的混合项系数: [0.18864671832823302, 0.15904731429667726, 0.3309157275910514, 0.32139023978403847]
隐藏变量：
[[2.44977733e-16 9.99999998e-01 1.64152560e-09 3.03141221e-10]
[8.35433159e-11 6.84557991e-09 9.99999993e-01 5.51062492e-11]
[1.78544054e-16 9.99999982e-01 1.80782177e-08 1.24843415e-10]
…
[4.17809360e-27 9.28165996e-03 9.90718340e-01 1.47491213e-22]
[4.20444073e-08 3.36765030e-09 9.99999946e-01 8.72387553e-09]
[9.46949643e-01 7.76003719e-16 8.10283776e-13 5.30503565e-02]]
迭代次数: 24
估计的均值: [[43.0407747 10.08642401]
[30.47423868 40.22065945]
[14.94556125 25.32426646]
[45.3738127 18.12234054]]
估计的混合项系数: [0.1876606110527325, 0.15906894376118258, 0.3308510136613136, 0.32241943152477065]
隐藏变量：
[[2.55607808e-16 9.99999998e-01 1.64878118e-09 2.81902476e-10]
[9.68252217e-11 6.88788193e-09 9.99999993e-01 5.12856739e-11]
[1.88332342e-16 9.99999982e-01 1.81605428e-08 1.15679120e-10]
…
[5.00194671e-27 9.17526049e-03 9.90824740e-01 1.27458495e-22]
[4.82738251e-08 3.39660080e-09 9.99999940e-01 8.21316159e-09]
[9.47598918e-01 7.70879668e-16 7.89099404e-13 5.24010823e-02]]
迭代次数: 25
估计的均值: [[42.95645749 10.08728947]
[30.47639717 40.2168795 ]
[14.94346252 25.33021277]
[45.41218244 18.09678529]]
估计的混合项系数: [0.186833065987643, 0.15909603348929255, 0.33077961998129995, 0.3232912805417645]
隐藏变量：
[[2.67960279e-16 9.99999998e-01 1.65451211e-09 2.63599880e-10]
[1.11346910e-10 6.93297597e-09 9.99999993e-01 4.79662699e-11]
[1.99410629e-16 9.99999982e-01 1.82258931e-08 1.07801134e-10]
…
[5.96064808e-27 9.08267100e-03 9.90917329e-01 1.11351933e-22]
[5.49967520e-08 3.42656710e-09 9.99999934e-01 7.76377051e-09]
[9.48214897e-01 7.66695588e-16 7.69734088e-13 5.17851032e-02]]
迭代次数: 26
估计的均值: [[42.8772295 10.09143987]
[30.47826297 40.21317812]
[14.94123556 25.33568316]
[45.44823624 18.07328666]]
估计的混合项系数: [0.18614414235881865, 0.15912478341423905, 0.33070601461798116, 0.32402505960896094]
隐藏变量：
[[2.82124371e-16 9.99999998e-01 1.65918467e-09 2.47599742e-10]
[1.27244852e-10 6.97833547e-09 9.99999993e-01 4.50287927e-11]
[2.11893388e-16 9.99999982e-01 1.82796391e-08 1.00929959e-10]
…
[7.07894055e-27 8.99912163e-03 9.91000878e-01 9.81247128e-23]
[6.22626402e-08 3.45637179e-09 9.99999927e-01 7.36097988e-09]
[9.48812104e-01 7.63130002e-16 7.51901518e-13 5.11878964e-02]]
迭代次数: 27
估计的均值: [[42.80220589 10.09852384]
[30.47993676 40.20959588]
[14.93893976 25.34076246]
[45.48253979 18.05155433]]
估计的混合项系数: [0.18557709053751037, 0.15915378197659005, 0.3306320856964241, 0.3246370417894752]
隐藏变量：
[[2.98244451e-16 9.99999998e-01 1.66304888e-09 2.33427348e-10]
[1.44683654e-10 7.02302127e-09 9.99999993e-01 4.23874282e-11]
[2.25931469e-16 9.99999982e-01 1.83245893e-08 9.48560592e-11]
…
[8.38766722e-27 8.92242111e-03 9.91077579e-01 8.70624570e-23]
[7.01310676e-08 3.48558464e-09 9.99999919e-01 6.99412663e-09]
[9.49400163e-01 7.59992405e-16 7.35372662e-13 5.05998372e-02]]
迭代次数: 28
估计的均值: [[42.73064355 10.10827167]
[30.48146758 40.20613493]
[14.93660178 25.34550892]
[45.51554323 18.03132768]]
估计的混合项系数: [0.18511787588441114, 0.15918257769742047, 0.3305585181503539, 0.32514102826781466]
隐藏变量：
[[3.16514448e-16 9.99999998e-01 1.66625196e-09 2.20721210e-10]
[1.63861425e-10 7.06679220e-09 9.99999993e-01 3.99797742e-11]
[2.41712445e-16 9.99999982e-01 1.83623713e-08 8.94203722e-11]
…
[9.92512848e-27 8.85129127e-03 9.91148709e-01 7.76634519e-23]
[7.86738105e-08 3.51411954e-09 9.99999911e-01 6.65545744e-09]
[9.49985467e-01 7.57171053e-16 7.19949484e-13 5.00145330e-02]]
迭代次数: 29
估计的均值: [[42.66190873 10.12047473]
[30.48288418 40.20278101]
[14.93422995 25.34996998]
[45.54761048 18.01237338]]
估计的混合项系数: [0.1847546615022768, 0.15921110234578367, 0.33048541999163134, 0.32554881616030806]
隐藏变量：
[[3.37178434e-16 9.99999998e-01 1.66888849e-09 2.09201684e-10]
[1.85015683e-10 7.10970922e-09 9.99999993e-01 3.77597505e-11]
[2.59463917e-16 9.99999982e-01 1.83940255e-08 8.45004244e-11]
…
[1.17389734e-26 8.78481429e-03 9.91215186e-01 6.95667961e-23]
[8.79768132e-08 3.54204433e-09 9.99999902e-01 6.33926363e-09]
[9.50572211e-01 7.54599478e-16 7.05459271e-13 4.94277887e-02]]
迭代次数: 30
估计的均值: [[42.59545288 10.13497148]
[30.48420648 40.19951388]
[14.93182232 25.35418727]
[45.57904126 17.99448121]]
估计的混合项系数: [0.18447735258954256, 0.15923943610555813, 0.33041261550303164, 0.3258705958018675]
隐藏变量：
[[3.60535105e-16 9.99999998e-01 1.67102334e-09 1.98649204e-10]
[2.08430483e-10 7.15195932e-09 9.99999993e-01 3.56928011e-11]
[2.79459179e-16 9.99999981e-01 1.84202678e-08 8.00005632e-11]
…
[1.38887268e-26 8.72224492e-03 9.91277755e-01 6.25075300e-23]
[9.81424583e-08 3.56949248e-09 9.99999892e-01 6.04128322e-09]
[9.51163040e-01 7.52237603e-16 6.91752452e-13 4.88369604e-02]]
迭代次数: 31
估计的均值: [[42.5307944 10.15163761]
[30.4854499 40.19631189]
[14.92937107 25.35819824]
[45.61008692 17.97745956]]
估计的混合项系数: [0.18427721837847763, 0.15926771224672237, 0.33033978548351434, 0.32611528389128575]
隐藏变量：
[[3.86945214e-16 9.99999998e-01 1.67270263e-09 1.88889021e-10]
[2.34445263e-10 7.19377784e-09 9.99999993e-01 3.37526688e-11]
[3.02025016e-16 9.99999981e-01 1.84416128e-08 7.58450896e-11]
…
[1.64491304e-26 8.66294323e-03 9.91337057e-01 5.62877094e-23]
[1.09292349e-07 3.59662282e-09 9.99999881e-01 5.75829626e-09]
[9.51759454e-01 7.50061545e-16 6.78700421e-13 4.82405462e-02]]
迭代次数: 32
估计的均值: [[42.46750452 10.17037868]
[30.48662716 40.19315394]
[14.92686478 25.36203684]
[45.6409619 17.96113119]]
估计的混合项系数: [0.18414658119284794, 0.15929607920851013, 0.3302665351304893, 0.3262908044681525]
隐藏变量：
[[4.16841710e-16 9.99999998e-01 1.67395928e-09 1.79780375e-10]
[2.63465875e-10 7.23541506e-09 9.99999992e-01 3.19192050e-11]
[3.27551752e-16 9.99999981e-01 1.84584344e-08 7.19733908e-11]
…
[1.95145610e-26 8.60634704e-03 9.91393653e-01 5.07572440e-23]
[1.21570762e-07 3.62360183e-09 9.99999869e-01 5.48785031e-09]
[9.52362076e-01 7.48058133e-16 6.66193082e-13 4.76379241e-02]]
迭代次数: 33
估计的均值: [[42.40519623 10.19112446]
[30.48774908 40.1900201 ]
[14.9242897 25.36573383]
[45.67185185 17.94532943]]
估计的混合项系数: [0.18407855680900317, 0.15932468629465307, 0.33019242667662335, 0.3264043302197197]
隐藏变量：
[[4.50742747e-16 9.99999998e-01 1.67481604e-09 1.71208718e-10]
[2.95978369e-10 7.27712271e-09 9.99999992e-01 3.01768599e-11]
[3.56505836e-16 9.99999981e-01 1.84709985e-08 6.83364541e-11]
…
[2.32048967e-26 8.55195754e-03 9.91448042e-01 4.58008976e-23]
[1.35148937e-07 3.65059636e-09 9.99999856e-01 5.22807193e-09]
[9.52970812e-01 7.46221926e-16 6.54136384e-13 4.70291880e-02]]
迭代次数: 34
估计的均值: [[42.34351542 10.2138244 ]
[30.488825 40.18689177]
[14.92163035 25.36931723]
[45.70291943 17.92989474]]
估计的混合项系数: [0.1840668314634411, 0.15935367892495772, 0.3301169943319753, 0.32646249527962556]
隐藏变量：
[[4.89267935e-16 9.99999998e-01 1.67528730e-09 1.63080060e-10]
[3.32566243e-10 7.31914939e-09 9.99999992e-01 2.85136222e-11]
[3.89445473e-16 9.99999981e-01 1.84794825e-08 6.48943416e-11]
…
[2.76733505e-26 8.49932951e-03 9.91500670e-01 4.13292711e-23]
[1.50230409e-07 3.67777128e-09 9.99999841e-01 4.97753524e-09]
[9.53584954e-01 7.44553567e-16 6.42450043e-13 4.64150460e-02]]
迭代次数: 35
估计的均值: [[42.28213364 10.23844383]
[30.48986304 40.18375158]
[14.91886981 25.37281257]
[45.73430827 17.91467157]]
估计的混合项系数: [0.1841054634718351, 0.15938319757839967, 0.3300397500414968, 0.3264715889082682]
隐藏变量：
[[5.33158468e-16 9.99999998e-01 1.67538022e-09 1.55316822e-10]
[3.73932124e-10 7.36173985e-09 9.99999992e-01 2.69202599e-11]
[4.27039980e-16 9.99999981e-01 1.84839867e-08 6.16143410e-11]
…
[3.31169992e-26 8.44806335e-03 9.91551937e-01 3.72724202e-23]
[1.67057651e-07 3.70528922e-09 9.99999825e-01 4.73516854e-09]
[9.54203234e-01 7.43058858e-16 6.31065556e-13 4.57967659e-02]]
迭代次数: 36
估计的均值: [[42.220742 10.26496075]
[30.4908703 40.18058328]
[14.91598978 25.37624327]
[45.76614573 17.8995055 ]]
估计的混合项系数: [0.184188699952516, 0.1594133779937191, 0.3299601842810219, 0.3264377377727428]
隐藏变量：
[[5.83301945e-16 9.99999998e-01 1.67509549e-09 1.47854762e-10]
[4.20925154e-10 7.40513591e-09 9.99999992e-01 2.53897659e-11]
[4.70093743e-16 9.99999981e-01 1.84845431e-08 5.84695989e-11]
…
[3.97910376e-26 8.39779813e-03 9.91602202e-01 3.35752469e-23]
[1.85920390e-07 3.73331137e-09 9.99999806e-01 4.50018671e-09]
[9.54823847e-01 7.41748261e-16 6.19924503e-13 4.51761525e-02]]
迭代次数: 37
估计的均值: [[42.15904587 10.29336303]
[30.49185298 40.17737161]
[14.91297061 25.37963101]
[45.79854449 17.88424044]]
估计的混合项系数: [0.18431080122740917, 0.15944435163709425, 0.32987776401960217, 0.32636708311589374]
隐藏变量：
[[6.40762976e-16 9.99999998e-01 1.67442789e-09 1.40640685e-10]
[4.74575811e-10 7.44957806e-09 9.99999992e-01 2.39169418e-11]
[5.19575964e-16 9.99999981e-01 1.84811213e-08 5.54381011e-11]
…
[4.80282259e-26 8.34820537e-03 9.91651795e-01 3.01941080e-23]
[2.07166072e-07 3.76199857e-09 9.99999785e-01 4.27204096e-09]
[9.55444450e-01 7.40636651e-16 6.08977138e-13 4.45555497e-02]]
迭代次数: 38
估计的均值: [[42.09675996 10.32364591]
[30.4928165 40.17410219]
[14.90979114 25.38299606]
[45.83160348 17.86871599]]
估计的混合项系数: [0.18446586712714413, 0.15947624605474087, 0.32979192890764314, 0.326265957910472]
隐藏变量：
[[7.06821017e-16 9.99999998e-01 1.67336668e-09 1.33630729e-10]
[5.36140559e-10 7.49530723e-09 9.99999992e-01 2.24980763e-11]
[5.76657729e-16 9.99999981e-01 1.84736326e-08 5.25019066e-11]
…
[5.82656497e-26 8.29898334e-03 9.91701017e-01 2.70942684e-23]
[2.31213157e-07 3.79151269e-09 9.99999761e-01 4.05038022e-09]
[9.56062138e-01 7.39743234e-16 5.98181209e-13 4.39378617e-02]]
迭代次数: 39
估计的均值: [[42.03360349 10.35580987]
[30.49376561 40.1707614 ]
[14.90642861 25.3863576 ]
[45.86540802 17.85276465]]
估计的混合项系数: [0.18464766076531805, 0.15950918498654104, 0.32970208616473073, 0.3261410680834102]
隐藏变量：
[[7.83017351e-16 9.99999998e-01 1.67189587e-09 1.26789068e-10]
[6.07159643e-10 7.54256660e-09 9.99999992e-01 2.11306878e-11]
[6.42758457e-16 9.99999981e-01 1.84619332e-08 4.96465625e-11]
…
[7.10818906e-26 8.24985201e-03 9.91750148e-01 2.42479497e-23]
[2.58568158e-07 3.82201796e-09 9.99999734e-01 3.83502027e-09]
[9.56673409e-01 7.39091593e-16 5.87500961e-13 4.33265909e-02]]
迭代次数: 40
估计的均值: [[41.96929513 10.38985885]
[30.49470445 40.16733639]
[14.90285842 25.38973405]
[45.90002961 17.83620911]]
估计的混合项系数: [0.1848494264738752, 0.15954328823184044, 0.32960760427381663, 0.3259996810204678]
隐藏变量：
[[8.71213844e-16 9.99999998e-01 1.66999443e-09 1.20086922e-10]
[6.89532805e-10 7.59160351e-09 9.99999992e-01 1.98133076e-11]
[7.19604588e-16 9.99999982e-01 1.84458267e-08 4.68606526e-11]
…
[8.72492041e-26 8.20054814e-03 9.91799452e-01 2.16328029e-23]
[2.89847766e-07 3.85368243e-09 9.99999703e-01 3.62591747e-09]
[9.57274112e-01 7.38709820e-16 5.76906285e-13 4.27258877e-02]]
迭代次数: 41
估计的均值: [[41.90354726 10.42579905]
[30.4956366 40.16381501]
[14.89905387 25.39314349]
[45.93552533 17.81885915]]
估计的混合项系数: [0.18506369957026028, 0.15957867131938236, 0.3295078053004223, 0.3258498238099352]
隐藏变量：
[[9.73667200e-16 9.99999998e-01 1.66763640e-09 1.13501777e-10]
[7.85619805e-10 7.64267145e-09 9.99999992e-01 1.85452883e-11]
[8.09304499e-16 9.99999982e-01 1.84250654e-08 4.41354354e-11]
…
[1.07807651e-25 8.15082064e-03 9.91849179e-01 1.92306887e-23]
[3.25807981e-07 3.88667962e-09 9.99999667e-01 3.42314502e-09]
[9.57859385e-01 7.38630759e-16 5.66371956e-13 4.21406150e-02]]
迭代次数: 42
估计的均值: [[41.83605909 10.4636384 ]
[30.49656521 40.16018577]
[14.89498588 25.39660396]
[45.97193706 17.80050844]]
估计的混合项系数: [0.18528210646135795, 0.15961544507854117, 0.3294019553617604, 0.3257004930983405]
隐藏变量：
[[1.09312412e-15 9.99999998e-01 1.66479097e-09 1.07016760e-10]
[8.98376002e-10 7.69603239e-09 9.99999991e-01 1.73266259e-11]
[9.14445491e-16 9.99999982e-01 1.83993517e-08 4.14645423e-11]
…
[1.34171869e-25 8.10042572e-03 9.91899574e-01 1.70266832e-23]
[3.67383082e-07 3.92119035e-09 9.99999625e-01 3.22687029e-09]
[9.58423575e-01 7.38892350e-16 5.55876919e-13 4.15764249e-02]]
迭代次数: 43
估计的均值: [[41.76650813 10.50338719]
[30.49749298 40.15643784]
[14.8906225 25.40013399]
[46.00929088 17.78093079]]
估计的混合项系数: [0.18549515420559307, 0.15965371525500055, 0.3292892524056833, 0.32556187813372317]
隐藏变量：
[[1.23294552e-15 9.99999998e-01 1.66142241e-09 1.00620080e-10]
[1.03153849e-09 7.75195985e-09 9.99999991e-01 1.61577880e-11]
[1.03822157e-15 9.99999982e-01 1.83683372e-08 3.88437092e-11]
…
[1.68287328e-25 8.04912170e-03 9.91950878e-01 1.50082578e-23]
[4.15738738e-07 3.95740514e-09 9.99999577e-01 3.03733197e-09]
[9.58960147e-01 7.39538137e-16 5.45403582e-13 4.10398532e-02]]
迭代次数: 44
估计的均值: [[41.6945391 10.54506023]
[30.49842229 40.15256101]
[14.88592839 25.40375306]
[46.04759663 17.7598757 ]]
估计的混合项系数: [0.18569200888347653, 0.1596935823665739, 0.3291688099593771, 0.32544559879057294]
隐藏变量：
[[1.39727242e-15 9.99999998e-01 1.65748986e-09 9.43045193e-11]
[1.18988671e-09 7.81074288e-09 9.99999991e-01 1.50395442e-11]
[1.18460551e-15 9.99999982e-01 1.83316216e-08 3.62705209e-11]
…
[2.12863224e-25 7.99666282e-03 9.92003337e-01 1.31645961e-23]
[4.72345811e-07 3.99552740e-09 9.99999521e-01 2.85481640e-09]
[9.59461564e-01 7.40618014e-16 5.34937076e-13 4.05384361e-02]]
迭代次数: 45
估计的均值: [[41.61974956 10.58868123]
[30.49935526 40.14854553]
[14.88086395 25.40748236]
[46.08684824 17.73706315]]
估计的混合项系数: [0.18586026168975883, 0.15973514205326683, 0.3290396348000742, 0.32536496145689997]
隐藏变量：
[[1.59125352e-15 9.99999998e-01 1.65294690e-09 8.80669072e-11]
[1.37961537e-09 7.87269189e-09 9.99999991e-01 1.39727963e-11]
[1.35858640e-15 9.99999982e-01 1.82887488e-08 3.37441545e-11]
…
[2.71727035e-25 7.94279174e-03 9.92057208e-01 1.14860294e-23]
[5.39085148e-07 4.03577787e-09 9.99999454e-01 2.67963312e-09]
[9.59919139e-01 7.42189347e-16 5.24464441e-13 4.00808606e-02]]
迭代次数: 46
估计的均值: [[41.54167092 10.63429018]
[30.50029385 40.14438201]
[14.8753842 25.41134559]
[46.12702483 17.71217691]]
估计的混合项系数: [0.1859856801836716, 0.1597784862332621, 0.3289005954955445, 0.3253352380875214]
隐藏变量：
[[1.82136671e-15 9.99999998e-01 1.64774074e-09 8.19075803e-11]
[1.60888043e-09 7.93814712e-09 9.99999990e-01 1.29584117e-11]
[1.56650713e-15 9.99999982e-01 1.82391986e-08 3.12651141e-11]
…
[3.50377573e-25 7.88722977e-03 9.92112770e-01 9.96358033e-24]
[6.18399707e-07 4.07840099e-09 9.99999375e-01 2.51208976e-09]
[9.60322833e-01 7.44318669e-16 5.13973714e-13 3.96771668e-02]]
迭代次数: 47
估计的均值: [[41.45974338 10.68195499]
[30.50123998 40.14006105]
[14.86943705 25.41537022]
[46.16809319 17.68485613]]
估计的混合项系数: [0.18605193922833357, 0.1598237054191537, 0.3287503773465937, 0.3253739780059189]
隐藏变量：
[[2.09588769e-15 9.99999998e-01 1.64181080e-09 7.58298135e-11]
[1.88861835e-09 8.00749092e-09 9.99999990e-01 1.19970647e-11]
[1.81655799e-15 9.99999982e-01 1.81823740e-08 2.88349586e-11]
…
[4.56871770e-25 7.82966393e-03 9.92170336e-01 8.58861170e-24]
[7.13520587e-07 4.12367410e-09 9.99999280e-01 2.35246724e-09]
[9.60660969e-01 7.47084305e-16 5.03452871e-13 3.93390311e-02]]
迭代次数: 48
估计的均值: [[41.37328278 10.73178848]
[30.50219576 40.13557282]
[14.86296085 25.419589 ]
[46.21001159 17.65468443]]
估计的混合项系数: [0.1860403216280052, 0.15987089255161996, 0.32858741728655927, 0.3255013685338153]
隐藏变量：
[[2.42559398e-15 9.99999998e-01 1.63508647e-09 6.98392601e-11]
[2.23380717e-09 8.08116572e-09 9.99999990e-01 1.10890936e-11]
[2.11952097e-15 9.99999982e-01 1.81175790e-08 2.64560365e-11]
…
[6.03291361e-25 7.76972942e-03 9.92230271e-01 7.35258082e-24]
[8.28811000e-07 4.17192076e-09 9.99999165e-01 2.20099664e-09]
[9.60919825e-01 7.50580384e-16 4.92888636e-13 3.90801749e-02]]
迭代次数: 49
估计的均值: [[41.28143709 10.78397272]
[30.5031636 40.13090643]
[14.85588077 25.4240423 ]
[46.25273514 17.62117575]]
估计的混合项系数: [0.18592937283626168, 0.15992014862910228, 0.3284098095488631, 0.3257406689857727]
隐藏变量：
[[2.82485357e-15 9.99999998e-01 1.62748356e-09 6.39434687e-11]
[2.66546068e-09 8.15969992e-09 9.99999989e-01 1.02343854e-11]
[2.48992496e-15 9.99999982e-01 1.80439843e-08 2.41312573e-11]
…
[8.08253927e-25 7.70698553e-03 9.92293014e-01 6.24690145e-24]
[9.70302828e-07 4.22352993e-09 9.99999023e-01 2.05783998e-09]
[9.61083030e-01 7.54923025e-16 4.82265139e-13 3.89169695e-02]]
迭代次数: 50
估计的均值: [[41.18312957 10.83879254]
[30.50414658 40.12604912]
[14.84810357 25.42878115]
[46.29622212 17.58375627]]
估计的混合项系数: [0.18569448958535423, 0.15997159018086787, 0.3282151691288174, 0.3261187511049607]
隐藏变量：
[[3.31335414e-15 9.99999998e-01 1.61889906e-09 5.81515994e-11]
[3.21387144e-09 8.24374476e-09 9.99999989e-01 9.43230166e-12]
[2.94788974e-15 9.99999982e-01 1.79605755e-08 2.18639468e-11]
…
[1.10137906e-24 7.64088260e-03 9.92359117e-01 5.26291260e-24]
[1.14655539e-06 4.27898346e-09 9.99998847e-01 1.92307734e-09]
[9.61130658e-01 7.60259782e-16 4.71562468e-13 3.88693424e-02]]
迭代次数: 51
估计的均值: [[41.07698556 10.8966806 ]
[30.50514873 40.12098516]
[14.83951008 25.43387177]
[46.34044026 17.54174206]]
估计的混合项系数: [0.18530742494927321, 0.160025358120896, 0.328000434919629, 0.3266667820102022]
隐藏变量：
[[3.91891900e-15 9.99999998e-01 1.60920346e-09 5.24745268e-11]
[3.92404373e-09 8.33412619e-09 9.99999988e-01 8.68166581e-12]
[3.52214929e-15 9.99999982e-01 1.78660786e-08 1.96578584e-11]
…
[1.53157233e-24 7.57071641e-03 9.92429284e-01 4.39195061e-24]
[1.37006739e-06 4.33889502e-09 9.99998624e-01 1.79670445e-09]
[9.61037857e-01 7.66784043e-16 4.60755211e-13 3.89621427e-02]]
迭代次数: 52
估计的均值: [[40.96124084 10.95827601]
[30.50617541 40.11569488]
[14.82994447 25.43940187]
[46.38537026 17.49431361]]
估计的混合项系数: [0.184735716419081, 0.1600816264967176, 0.3277615877129701, 0.327421069371231]
隐藏变量：
[[4.68220052e-15 9.99999998e-01 1.59823003e-09 4.69255983e-11]
[4.86508296e-09 8.43191674e-09 9.99999987e-01 7.98083648e-12]
[4.25513812e-15 9.99999982e-01 1.77588539e-08 1.75174394e-11]
…
[2.18311289e-24 7.49556571e-03 9.92504434e-01 3.62551537e-24]
[1.65966416e-06 4.40406491e-09 9.99998334e-01 1.67864603e-09]
[9.60772799e-01 7.74756856e-16 4.49811146e-13 3.92272010e-02]]
迭代次数: 53
估计的均值: [[40.8336336 11.02449641]
[30.50723375 40.11015359]
[14.81919921 25.44548984]
[46.43100144 17.44049186]]
估计的混合项系数: [0.18394211959930148, 0.1601406076683685, 0.32749325065983137, 0.32842402207249816]
隐藏变量：
[[5.66467222e-15 9.99999998e-01 1.58576041e-09 4.15223995e-11]
[6.14695322e-09 8.53853319e-09 9.99999985e-01 7.32790844e-12]
[5.21176015e-15 9.99999982e-01 1.76367544e-08 1.54484789e-11]
…
[3.20849856e-24 7.41420725e-03 9.92585793e-01 2.95551455e-24]
[2.04465055e-06 4.47555591e-09 9.99997949e-01 1.56879355e-09]
[9.60293591e-01 7.84539685e-16 4.38690306e-13 3.97064094e-02]]
迭代次数: 54
估计的均值: [[40.69129382 11.09661731]
[30.50833313 40.10433146]
[14.80699457 25.45229772]
[46.47730953 17.37912843]]
估计的混合项系数: [0.1828843178614694, 0.16020254671699616, 0.3271881347376156, 0.32972500068391886]
隐藏变量：
[[6.96248675e-15 9.99999998e-01 1.57150638e-09 3.62899205e-11]
[7.95135785e-09 8.65586519e-09 9.99999983e-01 6.72110377e-12]
[6.49484069e-15 9.99999982e-01 1.74969468e-08 1.34592751e-11]
…
[4.89920772e-24 7.32500325e-03 9.92674997e-01 2.37455683e-24]
[2.57223578e-06 4.55479611e-09 9.99997422e-01 1.46708132e-09]
[9.59543753e-01 7.96642659e-16 4.27344639e-13 4.04562474e-02]]
迭代次数: 55
估计的均值: [[40.53067202 11.17633652]
[30.50948549 40.09819498]
[14.79295201 25.46004941]
[46.52419917 17.30893878]]
估计的混合项系数: [0.1815155746104749, 0.160267691122327, 0.32683630047037193, 0.3313804337968262]
隐藏变量：
[[8.73078129e-15 9.99999998e-01 1.55508948e-09 3.12653373e-11]
[1.05903432e-08 8.78643527e-09 9.99999981e-01 6.15944578e-12]
[8.27279932e-15 9.99999983e-01 1.73357117e-08 1.15623890e-11]
…
[7.84698791e-24 7.22576013e-03 9.92774240e-01 1.87623630e-24]
[3.32112579e-06 4.64371243e-09 9.99996673e-01 1.37362269e-09]
[9.58445817e-01 8.11792629e-16 4.15718151e-13 4.15541833e-02]]
迭代次数: 56
估计的均值: [[40.34760412 11.26576874]
[30.51070545 40.09171229]
[14.77656328 25.46905472]
[46.57138437 17.22863236]]
估计的混合项系数: [0.17978768556597496, 0.16033621063248138, 0.32642425845205764, 0.3334518453494864]
隐藏变量：
[[1.12262556e-14 9.99999998e-01 1.53602454e-09 2.65038221e-11]
[1.46200862e-08 8.93357464e-09 9.99999976e-01 5.64381307e-12]
[1.08294665e-14 9.99999983e-01 1.71482826e-08 9.77675464e-12]
…
[1.33328100e-23 7.11357183e-03 9.92886428e-01 1.45529111e-24]
[4.42679271e-06 4.74488897e-09 9.99995567e-01 1.28893341e-09]
[9.56892928e-01 8.31020608e-16 4.03746137e-13 4.31070715e-02]]
迭代次数: 57
估计的均值: [[40.13768544 11.36726759]
[30.5120098 40.08486537]
[14.75716361 25.47973794]
[46.61818128 17.13721784]]
估计的混合项系数: [0.17765839298491456, 0.16040802773653048, 0.32593408289085357, 0.3359994963877011]
隐藏变量：
[[1.48801015e-14 9.99999998e-01 1.51372243e-09 2.20827065e-11]
[2.10642639e-08 9.10156104e-09 9.99999970e-01 5.17835632e-12]
[1.46525623e-14 9.99999983e-01 1.69288791e-08 8.12915285e-12]
…
[2.43148803e-23 6.98469760e-03 9.93015302e-01 1.10742811e-24]
[6.12852660e-06 4.86171869e-09 9.99993865e-01 1.21425432e-09]
[9.54739388e-01 8.55751661e-16 3.91349583e-13 4.52606122e-02]]
迭代次数: 58
估计的均值: [[39.89715097 11.48294739]
[30.51341605 40.07767315]
[14.73393004 25.4926633 ]
[46.66323003 17.03453417]]
估计的混合项系数: [0.17510526517615915, 0.16048251125708687, 0.32534303080515775, 0.3390691927615959]
隐藏变量：
[[2.04163116e-14 9.99999998e-01 1.48754288e-09 1.80979821e-11]
[3.18497291e-08 9.29559280e-09 9.99999959e-01 4.77186409e-12]
[2.05845185e-14 9.99999983e-01 1.66712380e-08 6.65285646e-12]
…
[4.80439240e-23 6.83459155e-03 9.93165408e-01 8.28565284e-25]
[8.85550786e-06 4.99846410e-09 9.99991138e-01 1.15191418e-09]
[9.51793217e-01 8.87845054e-16 3.78418302e-13 4.82067826e-02]]
迭代次数: 59
估计的均值: [[39.62424089 11.61388578]
[30.51494023 40.07022749]
[14.70594105 25.50854127]
[46.70428496 16.92189458]]
估计的混合项系数: [0.17214460805033135, 0.16055801287460342, 0.324624565887793, 0.3426728131872722]
隐藏变量：
[[2.90260185e-14 9.99999999e-01 1.45693804e-09 1.46459234e-11]
[5.06344440e-08 9.52139123e-09 9.99999940e-01 4.43801392e-12]
[3.00608117e-14 9.99999984e-01 1.63700539e-08 5.38087178e-12]
…
[1.03150109e-22 6.65827151e-03 9.93341728e-01 6.13487613e-25]
[1.33784266e-05 5.16007503e-09 9.99986615e-01 1.10553105e-09]
[9.47816227e-01 9.29489959e-16 3.64781366e-13 5.21837729e-02]]
迭代次数: 60
估计的均值: [[39.320443 11.75930774]
[30.51659579 40.06273657]
[14.67232629 25.52819558]
[46.73835341 16.80244355]]
估计的混合项系数: [0.16884637410757108, 0.16063132610391867, 0.3237516024972363, 0.34677069729127347]
隐藏变量：
[[4.25903537e-14 9.99999999e-01 1.42169670e-09 1.17908145e-11]
[8.43208838e-08 9.78425446e-09 9.99999906e-01 4.19334302e-12]
[4.54474794e-14 9.99999984e-01 1.60234331e-08 4.33425182e-12]
…
[2.38870006e-22 6.45115836e-03 9.93548842e-01 4.54579950e-25]
[2.10644068e-05 5.35161442e-09 9.99978929e-01 1.07977735e-09]
[9.42536061e-01 9.82873984e-16 3.50183582e-13 5.74639391e-02]]
迭代次数: 61
估计的均值: [[38.99048155 11.91637126]
[30.51839728 40.0555578 ]
[14.63246332 25.55249296]
[46.76236677 16.6807344 ]]
估计的混合项系数: [0.1653307785575671, 0.1606973011126511, 0.32270115186046233, 0.3512707684693197]
隐藏变量：
[[6.39232340e-14 9.99999999e-01 1.38218391e-09 9.53635143e-12]
[1.45747457e-07 1.00877495e-08 9.99999844e-01 4.05339454e-12]
[7.04680225e-14 9.99999984e-01 1.56353221e-08 3.51196345e-12]
…
[5.86300176e-22 6.21015299e-03 9.93789847e-01 3.41747899e-25]
[3.42964143e-05 5.57738922e-09 9.99965697e-01 1.07971136e-09]
[9.35665405e-01 1.04969085e-15 3.34303283e-13 6.43345945e-02]]
迭代次数: 62
估计的均值: [[38.64033301 12.08104155]
[30.52037102 40.04919636]
[14.586055 25.58229189]
[46.77410884 16.56148606]]
估计的混合项系数: [0.16174075772368837, 0.16074892097158994, 0.3214564938537436, 0.3560538274509778]
隐藏变量：
[[9.70441021e-14 9.99999999e-01 1.33933442e-09 7.82245257e-12]
[2.58622251e-07 1.04329615e-08 9.99999731e-01 4.02983993e-12]
[1.10754173e-13 9.99999985e-01 1.52154827e-08 2.89010964e-12]
…
[1.49267302e-21 5.93417594e-03 9.94065824e-01 2.63850642e-25]
[5.72059747e-05 5.84043189e-09 9.99942787e-01 1.11022631e-09]
[9.26906174e-01 1.13080561e-15 3.16813723e-13 7.30938256e-02]]
迭代次数: 63
估计的均值: [[38.27388054 12.24988444]
[30.52256858 40.04425748]
[14.53288421 25.61851847]
[46.77276322 16.44816031]]
估计的混合项系数: [0.15820441527017362, 0.1607779586472253, 0.3200016136861318, 0.3610160123964695]
隐藏变量：
[[1.47708218e-13 9.99999999e-01 1.29422409e-09 6.54892227e-12]
[4.67715347e-07 1.08197703e-08 9.99999521e-01 4.13150241e-12]
[1.74843251e-13 9.99999985e-01 1.47752043e-08 2.43078861e-12]
…
[3.87958184e-21 5.62332329e-03 9.94376677e-01 2.10615108e-25]
[9.71704650e-05 6.14346404e-09 9.99902822e-01 1.17640109e-09]
[9.15901837e-01 1.22651971e-15 2.97418650e-13 8.40981628e-02]]
迭代次数: 64
估计的均值: [[37.88961141 12.42206154]
[30.52507419 40.04137937]
[14.47219721 25.66247917]
[46.7586846 16.34210162]]
估计的混合项系数: [0.15481075556720164, 0.16077582816619168, 0.3183055613571243, 0.36610785490948294]
隐藏变量：
[[2.24936997e-13 9.99999999e-01 1.24737452e-09 5.60661606e-12]
[8.63622989e-07 1.12512144e-08 9.99999125e-01 4.37024034e-12]
[2.76732555e-13 9.99999986e-01 1.43202709e-08 2.09344143e-12]
…
[1.02834424e-20 5.27655085e-03 9.94723449e-01 1.73819507e-25]
[1.68430800e-04 6.49233062e-09 9.99831561e-01 1.28533454e-09]
[9.02094962e-01 1.33769692e-15 2.75786068e-13 9.79050376e-02]]
迭代次数: 65
估计的均值: [[37.47779006 12.60109607]
[30.52800608 40.04122323]
[14.40180527 25.71650602]
[46.73257528 16.24235448]]
估计的混合项系数: [0.15160844115035227, 0.16073362130535315, 0.31629734144872823, 0.3713605960955666]
隐藏变量：
[[3.45442547e-13 9.99999999e-01 1.19813110e-09 4.89806067e-12]
[1.65282712e-06 1.17410893e-08 9.99998335e-01 4.77203893e-12]
[4.43083918e-13 9.99999986e-01 1.38446341e-08 1.84221766e-12]
…
[2.83826446e-20 4.88856600e-03 9.95111434e-01 1.47441809e-25]
[3.02080435e-04 6.90246235e-09 9.99697911e-01 1.45015631e-09]
[8.84418358e-01 1.46780960e-15 2.51391490e-13 1.15581642e-01]]
迭代次数: 66
估计的均值: [[37.01661203 12.79689641]
[30.53152763 40.04463006]
[14.3167694 25.78514983]
[46.69445466 16.14571705]]
估计的混合项系数: [0.14862287255789436, 0.16063996067563394, 0.3138265696414943, 0.3769105971249776]
隐藏变量：
[[5.46450185e-13 9.99999999e-01 1.14415085e-09 4.34391803e-12]
[3.39102020e-06 1.23260713e-08 9.99996597e-01 5.39788157e-12]
[7.34672173e-13 9.99999987e-01 1.33251796e-08 1.64818041e-12]
…
[8.58003002e-20 4.44635493e-03 9.95553645e-01 1.27181096e-25]
[5.77982795e-04 7.41022467e-09 9.99422008e-01 1.69864562e-09]
[8.60616875e-01 1.62632990e-15 2.23317848e-13 1.39383125e-01]]
迭代次数: 67
估计的均值: [[36.46411752 13.02947946]
[30.53589305 40.053079 ]
[14.20688552 25.87756984]
[46.64245783 16.04652179]]
估计的混合项系数: [0.14588932149846032, 0.1604750936159448, 0.3105888822684617, 0.38304670261713264]
隐藏变量：
[[9.27034073e-13 9.99999999e-01 1.08054656e-09 3.88095836e-12]
[7.94688656e-06 1.30903547e-08 9.99992040e-01 6.39399603e-12]
[1.31896084e-12 9.99999987e-01 1.27127414e-08 1.48849143e-12]
…
[3.12821415e-19 3.92526397e-03 9.96074736e-01 1.09826416e-25]
[1.24938049e-03 8.09721124e-09 9.98750609e-01 2.09543856e-09]
[8.25550007e-01 1.83537779e-15 1.90022855e-13 1.74449993e-01]]
迭代次数: 68
估计的均值: [[35.74115028 13.33655987]
[30.54155883 40.06967601]
[14.05125878 26.01253574]
[46.57091622 15.9358256 ]]
估计的混合项系数: [0.1435070573122507, 0.16019873833298473, 0.30597399553547105, 0.3903202088192929]
隐藏变量：
[[1.80835795e-12 9.99999999e-01 9.97976591e-10 3.45851225e-12]
[2.38693462e-05 1.42289023e-08 9.99976116e-01 8.14380885e-12]
[2.77106660e-12 9.99999988e-01 1.19107527e-08 1.34515940e-12]
…
[1.63706412e-18 3.28741896e-03 9.96712581e-01 9.27948972e-26]
[3.38372548e-03 9.15171800e-09 9.96616263e-01 2.81130828e-09]
[7.66355914e-01 2.14423874e-15 1.49258022e-13 2.33644086e-01]]
迭代次数: 69
估计的均值: [[34.70650727 13.78371025]
[30.54938584 40.10092568]
[13.80905977 26.22661878]
[46.46660268 15.80097576]]
估计的混合项系数: [0.14166678166935626, 0.1597299383180467, 0.29884108996230074, 0.39976219005029595]
隐藏变量：
[[4.48993861e-12 9.99999999e-01 8.80229337e-10 3.04429363e-12]
[1.11090618e-04 1.62133550e-08 9.99888893e-01 1.18188651e-11]
[7.65076418e-12 9.99999989e-01 1.07416858e-08 1.20748358e-12]
…
[1.63652020e-17 2.50259899e-03 9.97497401e-01 7.44527312e-26]
[1.35486556e-02 1.10011779e-08 9.86451329e-01 4.37141415e-09]
[6.48997394e-01 2.64525287e-15 9.94211395e-14 3.51002606e-01]]
迭代次数: 70
估计的均值: [[33.17333966 14.44706145]
[30.56052809 40.15633747]
[13.42380137 26.57063798]
[46.30308443 15.63304666]]
估计的混合项系数: [0.14021097938818317, 0.15895839334785702, 0.2877502696370239, 0.41308035762693557]
隐藏变量：
[[1.49317722e-11 9.99999999e-01 7.12899490e-10 2.66715189e-12]
[9.70504583e-04 2.01259483e-08 9.99029475e-01 2.16726883e-11]
[2.97857460e-11 9.99999991e-01 9.01234800e-09 1.08918520e-12]
…
[4.21240813e-16 1.64257991e-03 9.98357420e-01 5.66685380e-26]
[8.95247896e-02 1.39517167e-08 9.10475188e-01 8.33718843e-09]
[4.09525046e-01 3.31099796e-15 4.64200583e-14 5.90474954e-01]]
迭代次数: 71
估计的均值: [[30.95950165 15.31216757]
[30.57336207 40.22633276]
[12.86126436 27.09834686]
[46.045063 15.45544285]]
估计的混合项系数: [0.13843343486474635, 0.15801810386239543, 0.271614234084118, 0.43193422718874025]
隐藏变量：
[[5.18862674e-11 9.99999999e-01 5.19894929e-10 2.49566882e-12]
[1.62160544e-02 2.90632175e-08 9.83783916e-01 5.67855098e-11]
[1.30314113e-10 9.99999993e-01 6.90760404e-09 1.06220721e-12]
…
[2.81422560e-14 8.99826923e-04 9.99100173e-01 4.48486434e-26]
[5.61163242e-01 1.14923117e-08 4.38836734e-01 1.25877865e-08]
[1.14859259e-01 3.39307042e-15 1.03399276e-14 8.85140741e-01]]
迭代次数: 72
估计的均值: [[27.75447595 16.1102132 ]
[30.58169421 40.26751703]
[12.00835501 28.1180865 ]
[45.68847699 15.33629634]]
估计的混合项系数: [0.14126305514391388, 0.15743584783201778, 0.24578295866804803, 0.45551813835601956]
隐藏变量：
[[7.75734118e-11 1.00000000e+00 3.63953207e-10 2.80957452e-12]
[3.80991492e-01 4.48316955e-08 6.19008463e-01 1.85639194e-10]
[2.75805390e-10 9.99999995e-01 5.15051396e-09 1.25589462e-12]
…
[2.59125894e-12 3.25762837e-04 9.99674237e-01 3.27909525e-26]
[9.73706107e-01 2.23634644e-09 2.62938860e-02 5.10336845e-09]
[8.24472702e-03 2.70582169e-15 4.95080864e-16 9.91755273e-01]]
迭代次数: 73
估计的均值: [[24.18481322 16.72521839]
[30.58800887 40.26191858]
[10.83601077 29.94099522]
[45.31122055 15.29381746]]
估计的混合项系数: [0.15512103977319425, 0.1573885153765887, 0.20937043945270897, 0.4781200053975075]
隐藏变量：
[[3.50965942e-11 1.00000000e+00 2.54643702e-10 3.64024354e-12]
[9.69711820e-01 1.36676300e-08 3.02881660e-02 1.17053648e-10]
[1.85299170e-10 9.99999996e-01 3.85466136e-09 1.70452938e-12]
…
[7.79783158e-11 7.12686544e-05 9.99928731e-01 1.71698841e-26]
[9.99435295e-01 4.79639174e-10 5.64702009e-04 2.17217390e-09]
[2.18970741e-04 2.22677834e-15 3.37759077e-18 9.99781029e-01]]
迭代次数: 74
估计的均值: [[22.16143103 17.85451889]
[30.59146778 40.1796643 ]
[ 9.43204805 31.51845207]
[45.04951426 15.23633496]]
估计的混合项系数: [0.17385816539748147, 0.1583035583442353, 0.17639541875459394, 0.4914428575036888]
隐藏变量：
[[5.80629509e-11 1.00000000e+00 7.47065506e-11 4.11046873e-12]
[9.98548160e-01 5.91107447e-09 1.45183381e-03 7.32530873e-11]
[3.70984422e-10 9.99999998e-01 1.25071111e-09 1.98361096e-12]
…
[1.18676203e-09 2.57646553e-05 9.99974234e-01 1.07104540e-26]
[9.99973756e-01 3.21863221e-10 2.62412667e-05 2.03159583e-09]
[1.26294778e-05 2.28970153e-15 1.61151948e-20 9.99987371e-01]]
迭代次数: 75
估计的均值: [[21.07057908 18.92872365]
[30.58327787 40.15870099]
[ 7.86752405 33.07317914]
[44.94592973 15.20710569]]
估计的混合项系数: [0.19961752116565756, 0.15867330470271623, 0.1456995211977265, 0.4960096529338994]
隐藏变量：
[[1.46135887e-10 1.00000000e+00 1.18901554e-11 4.25460003e-12]
[9.99925523e-01 3.78847527e-09 7.44732837e-05 5.45151474e-11]
[1.01606717e-09 9.99999999e-01 2.24766732e-10 2.07693447e-12]
…
[8.81629296e-09 1.35246042e-05 9.99986467e-01 6.77889956e-27]
[9.99998834e-01 2.86424191e-10 1.16346901e-06 2.07747606e-09]
[1.91415360e-06 2.24771057e-15 4.54072715e-23 9.99998086e-01]]
迭代次数: 76
估计的均值: [[20.46290344 19.65548765]
[30.58961798 40.2088799 ]
[ 6.46178113 34.44884838]
[44.90233988 15.19254951]]
估计的混合项系数: [0.22052491225632445, 0.15811820425237072, 0.12359386916714006, 0.49776301432416503]
隐藏变量：
[[2.81489306e-10 1.00000000e+00 1.71311214e-12 4.27558449e-12]
[9.99995027e-01 2.80103294e-09 4.96998067e-06 4.70920450e-11]
[2.04885885e-09 9.99999998e-01 3.63029413e-11 2.10471557e-12]
…
[3.29808867e-08 9.86671092e-06 9.99990100e-01 5.38461057e-27]
[9.99999935e-01 2.57800980e-10 6.26307446e-08 2.19620635e-09]
[5.89089063e-07 1.97529213e-15 1.89085096e-25 9.99999411e-01]]
迭代次数: 77
估计的均值: [[20.21669697 19.99330773]
[30.60039866 40.26974669]
[ 5.78603385 35.07081951]
[44.88470835 15.18924617]]
估计的混合项系数: [0.22991920130746002, 0.15739033846211842, 0.11422715905115136, 0.49846330117927035]
隐藏变量：
[[3.83870443e-10 1.00000000e+00 6.03074509e-13 4.28903286e-12]
[9.99998667e-01 2.35942330e-09 1.33016284e-06 4.51671827e-11]
[2.84880023e-09 9.99999997e-01 1.35534711e-11 2.12366234e-12]
…
[6.43522095e-08 9.37060736e-06 9.99990565e-01 5.37001150e-27]
[9.99999982e-01 2.35534803e-10 1.51058269e-08 2.30271283e-09]
[3.48993981e-07 1.72780104e-15 1.32912232e-26 9.99999651e-01]]
迭代次数: 78
估计的均值: [[20.14397814 20.12084303]
[30.6032376 40.30506238]
[ 5.56441975 35.23952042]
[44.87833768 15.18934672]]
估计的混合项系数: [0.2330242314210044, 0.1569689088904177, 0.11128805974430009, 0.4987187999442781]
隐藏变量：
[[4.37371971e-10 1.00000000e+00 4.16118529e-13 4.30303432e-12]
[9.99999109e-01 2.19211908e-09 8.88850375e-07 4.48241633e-11]
[3.26305373e-09 9.99999997e-01 9.55178241e-12 2.13581271e-12]
…
[8.37588449e-08 9.48986813e-06 9.99990426e-01 5.53795086e-27]
[9.99999988e-01 2.24553295e-10 9.83612267e-09 2.35695536e-09]
[2.92595615e-07 1.59930591e-15 5.84569358e-27 9.99999707e-01]]
迭代次数: 79
估计的均值: [[20.12348176 20.16513877]
[30.60459524 40.32169934]
[ 5.49645354 35.28652594]
[44.87631581 15.18958496]]
估计的混合项系数: [0.23405467200711402, 0.15676215519392384, 0.11038173829883739, 0.4988014345001249]
隐藏变量：
[[4.59337829e-10 1.00000000e+00 3.70028349e-13 4.30809591e-12]
[9.99999209e-01 2.12508062e-09 7.88804602e-07 4.47489620e-11]
[3.43204492e-09 9.99999997e-01 8.55307845e-12 2.14053372e-12]
…
[9.16903067e-08 9.55309450e-06 9.99990355e-01 5.61206951e-27]
[9.99999989e-01 2.19355185e-10 8.67327104e-09 2.37695353e-09]
[2.76798855e-07 1.54280140e-15 4.57633811e-27 9.99999723e-01]]
迭代次数: 80
估计的均值: [[20.11782904 20.1801906 ]
[30.60553293 40.32873992]
[ 5.47532397 35.30076728]
[44.87569365 15.18969428]]
估计的混合项系数: [0.2344031276976603, 0.15667033457081828, 0.11009911592178995, 0.4988274218097313]
隐藏变量：
[[4.67541888e-10 1.00000000e+00 3.56614745e-13 4.30928498e-12]
[9.99999238e-01 2.09808236e-09 7.60326552e-07 4.47289373e-11]
[3.49495899e-09 9.99999996e-01 8.26208479e-12 2.14209204e-12]
…
[9.44701434e-08 9.56895076e-06 9.99990337e-01 5.63715758e-27]
[9.99999989e-01 2.17055082e-10 8.34521061e-09 2.38369767e-09]
[2.72085222e-07 1.51993007e-15 4.24331689e-27 9.99999728e-01]]
迭代次数: 81
估计的均值: [[20.11630152 20.18530195]
[30.60603973 40.33155351]
[ 5.46868505 35.30523074]
[44.87550475 15.189735 ]]
估计的混合项系数: [0.23452201334750492, 0.1566323361541897, 0.11001016890393013, 0.4988354815943756]
隐藏变量：
[[4.70510140e-10 1.00000000e+00 3.52475420e-13 4.30944017e-12]
[9.99999246e-01 2.08746526e-09 7.51605120e-07 4.47229202e-11]
[3.51767313e-09 9.99999996e-01 8.17250935e-12 2.14257439e-12]
…
[9.54028861e-08 9.57136931e-06 9.99990333e-01 5.64513376e-27]
[9.99999989e-01 2.16099678e-10 8.24517351e-09 2.38592228e-09]
[2.70642125e-07 1.51102667e-15 4.14382513e-27 9.99999729e-01]]
迭代次数: 82
估计的均值: [[20.1159003 20.18704225]
[30.60627381 40.33264022]
[ 5.4665835 35.3066504 ]
[44.87544751 15.18974936]]
估计的混合项系数: [0.2345627479815288, 0.15661730139851904, 0.10998197185946357, 0.4988379787604886]
隐藏变量：
[[4.71570479e-10 1.00000000e+00 3.51168828e-13 4.30940765e-12]
[9.99999249e-01 2.08338854e-09 7.48862179e-07 4.47210211e-11]
[3.52577391e-09 9.99999996e-01 8.14434053e-12 2.14272054e-12]
…
[9.57128445e-08 9.57117177e-06 9.99990333e-01 5.64763275e-27]
[9.99999989e-01 2.15719777e-10 8.21380780e-09 2.38665466e-09]
[2.70194991e-07 1.50764186e-15 4.11277215e-27 9.99999730e-01]]
迭代次数: 83
估计的均值: [[20.11579942 20.18763644]
[30.6063728 40.33305044]
[ 5.46591388 35.30710594]
[44.87543014 15.18975436]]
估计的混合项系数: [0.23457674172793627, 0.1566115282588881, 0.10997297462917778, 0.49883875538399763]
隐藏变量：
[[4.71946878e-10 1.00000000e+00 3.50751668e-13 4.30936968e-12]
[9.99999250e-01 2.08185336e-09 7.47988609e-07 4.47204104e-11]
[3.52864567e-09 9.99999996e-01 8.13538485e-12 2.14276459e-12]
…
[9.58159010e-08 9.57078320e-06 9.99990333e-01 5.64841524e-27]
[9.99999989e-01 2.15573240e-10 8.20384580e-09 2.38689662e-09]
[2.70055418e-07 1.50637710e-15 4.10290490e-27 9.99999730e-01]]
迭代次数: 84
估计的均值: [[20.11577584 20.18783983]
[30.6064125 40.33320276]
[ 5.46569916 35.30725315]
[44.87542484 15.18975609]]
估计的混合项系数: [0.23458155854284657, 0.1566093577686685, 0.10997008553797614, 0.4988389981505082]
0.0007879452125143516 1.011916284210379e-05

在这里插入图片描述

# 画图
plotShow()

在这里插入图片描述

参考：
1、Machine-Learning：第3章 EM算法

2、统计学习方法

3、统计学习方法-代码解读

4、EM算法 - 期望极大算法

Axiiiz

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
Machine-Learning：第3章 EM算法

第3章 EM算法本章思维导图本章内容概要前言EM算法引入EM算法推导逼近证明收敛高斯混合分布混合高斯分布模型E步主要计算内容M步主要计算内容本章思维导图（待更新）本章内容概要前言EM算法是机器学习十大算法之一，它很简单，但是也同样很有深度，简单是因为它就分两步求解问题，E步：求期望（expectation）M步：求极大（maximization)深度在于它的数学推理涉及到...
复制链接

扫一扫