机器学习之EM（代码实践）

一、EM算法调参

GMM调参：spherical、diag、tied和full
BIC和AIC：
记：L为某模型下样本的似然函数值，k为模型中未
知参数的个数(维度)，n为样本个数，则AIC和BIC分别为：

二、EM相关代码示例

GMM导入：

from sklearn.mixture import GaussianMixture

高斯分布构造：

    mu1_fact = (0, 0, 0)
    cov1_fact = np.diag((1, 2, 3))
    data1 = np.random.multivariate_normal(mu1_fact, cov1_fact, 400)
    mu2_fact = (2, 2, 1)
    cov2_fact = np.array(((1, 1, 3), (1, 2, 1), (0, 0, 1)))
    data2 = np.random.multivariate_normal(mu2_fact, cov2_fact, 100)

E step：

norm1 = multivariate_normal(mu1, sigma1)
norm2 = multivariate_normal(mu2, sigma2)
tau1 = pi * norm1.pdf(data)
tau2 = (1 - pi) * norm2.pdf(data)
gamma = tau1 / (tau1 + tau2)

M step:

mu1 = np.dot(gamma, data) / np.sum(gamma)
mu2 = np.dot((1 - gamma), data) / np.sum((1 - gamma))
sigma1 = np.dot(gamma * (data - mu1).T, data - mu1) / np.sum(gamma)
sigma2 = np.dot((1 - gamma) * (data - mu2).T, data - mu2) / np.sum(1 - gamma)
pi = np.sum(gamma) / n

DPGMM:

# DPGMM
dpgmm=BayesianGaussianMixture(n_components=n_components,covariance_type='full',max_iter=1000,n_init=5,weight_concentration_prior_type='dirichlet_process', weight_concentration_prior=0.1)
dpgmm.fit(x)
centers = dpgmm.means_
covs = dpgmm.covariances_
y_hat = dpgmm.predict(x)