机器学习之七：K-均值聚类

一、K-均值聚类

K-均值聚类是一类简单的无监督学习方法，其核心思想是将数据按照相似程度分为K个簇。这类算法需要注意三点：

1. K的选择：K需要自己决定

2. 初始质心的选择：质心指的是每个簇的中心点，初始质心也需要自己决定，通常是随机选择；初始质心的个数等于K

3. 数据相似度的衡量：一般用特征值之间的距离之和来度量数据的相似度

其算法流程是：首先选取K个初始质心，随后对于每一条数据，计算其到每个质心的距离，将其与距离最近的质心分为同一个簇。这一步完成后，需要将每个簇的质心更新为该簇所有点的平均值。

由于K-均值聚类比较简单，笔者就不过多赘述了，直接上代码和效果图吧：

import numpy as np
import matplotlib.pyplot as plt

def loadDataset(filePath):
    dataList = []
    fr = open(filePath)
    for line in fr.readlines():
        curLine = line.strip().split('\t')
        fltLine = list(map(float,curLine))
        dataList.append(fltLine)
    return dataList

def distEclud(vecA,vecB):
    return np.sqrt(np.sum(np.power(vecA-vecB,2)))

def randCent(dataSet,k):
    n = np.shape(dataSet)[1]
    centroids = np.mat(np.zeros((k,n)))
    for j in range(n):
        minJ = min(dataSet[:,j])
        maxJ = max(dataSet[:,j])
        rangeJ = float(maxJ - minJ)
        centroids[:,j] = minJ + rangeJ*np.random.rand(k,1)
    return centroids

def kMeans(dataSet,k,distMeans=distEclud,createCent=randCent):
    m = np.shape(dataSet)[0]
    clusterAssment = np.mat(np.zeros((m,2)))
    centroids = createCent(dataSet,k)
    clusterChanged = True
    while clusterChanged:
        clusterChanged = False
        for i in range(m):
            minDist = np.inf
            minIndex = -1
            for j in range(k):
                distJI = distMeans(centroids[j,:],dataSet[i,:])
                if distJI < minDist:
                    minDist = distJI
                    minIndex = j
            if clusterAssment[i,0] != minIndex:
                clusterChanged = True
            clusterAssment[i,:] = minIndex,minDist
        #print(centroids)
        for cent in range(k):
            ptsInClust = dataSet[np.nonzero(clusterAssment[:,0].A==cent)[0]]
            centroids[cent,:] = np.mean(ptsInClust,axis=0)
    return centroids,clusterAssment

dataMat = np.mat(loadDataset(r'..\数据集\machinelearninginaction\Ch10\testSet.txt'))
myCentroids,clustAssing = kMeans(dataMat,4)
marker = np.array(['^','s','*','x','o','>','p','h','+','D','d'])             
for i in range(len(myCentroids)):
    index = np.nonzero(clustAssing[:,0]==i)[0]
    plt.scatter(np.array((dataMat[index,0])),np.array((dataMat[index,1])),marker=marker[i],c='blue')
plt.scatter(np.array(myCentroids[:,0]),np.array(myCentroids[:,1]),s=100,marker='+',c='red')
plt.show()

结果还是比较令人满意的。

二、二分K-均值算法

K-均值算法容易陷入局部最优，为了克服这个问题，人们提出了二分K-均值算法。在介绍二分K-均值之前，先介绍一下如何衡量不同聚类方式的效果。通常的衡量指标是误差平方和SSE，即每个数据点到其质心的聚类平方之和。显然，SSE越小，算法效果越好。

二分K-均值的思想是首先将所有点作为一个簇，然后将该簇一分为二。之后选择其中一个簇继续划分，选择哪一个簇取决于对其划分是否可以最大程度降低SSE的值。重复上述过程直到簇数等于K为止。二分K均值也比较简单，直接上代码吧：

def biKmeans(dataSet,k,distMeans=distEclud):
    m = np.shape(dataSet)[0]
    clusterAssment = np.mat(np.zeros((m,2)))
    centroid0 = np.mean(dataSet,axis=0).tolist()[0]
    centList = [centroid0]
    for j in range(m):
        clusterAssment[j,1] = distMeans(np.mat(centroid0),dataSet[j,:])**2
    while len(centList) < k:
        lowestSSE = np.inf
        for i in range(len(centList)):
            ptsInCurrCluster = dataSet[np.nonzero(clusterAssment[:,0].A==i)[0],:]
            centroidMat,splitClustAss = kMeans(ptsInCurrCluster,2,distMeans)
            sseSplit = sum(splitClustAss[:,1])
            sseNotSplit = sum(clusterAssment[np.nonzero(clusterAssment[:,0].A!=i)[0],1])
            if sseSplit+sseNotSplit < lowestSSE:
                bestCentToSplit = i
                bestNewCents = centroidMat
                bestClustAss = splitClustAss.copy()
                lowestSSE = sseSplit + sseNotSplit
        bestClustAss[np.nonzero(bestClustAss[:,0].A==1)[0],0] = len(centList)#新加簇
        bestClustAss[np.nonzero(bestClustAss[:,0].A==0)[0],0] = bestCentToSplit#原有簇
        centList[bestCentToSplit] = bestNewCents[0,:]
        centList.append(bestNewCents[1,:])#更新质心列表
        clusterAssment[np.nonzero(clusterAssment[:,0].A==bestCentToSplit)[0],:] = bestClustAss
    for i in range(len(centList)):
        centList[i] = list(np.array(centList[i]).flatten())
    return np.mat(centList),clusterAssment       
    
dataMat = np.mat(loadDataset(r'..\数据集\machinelearninginaction\Ch10\testSet2.txt'))
centList,myNewAssments = biKmeans(dataMat,3)
marker = np.array(['^','s','*','x','o','>','p','h','+','D','d'])    
for i in range(len(centList)):
    index = np.nonzero(myNewAssments[:,0]==i)[0]
    plt.scatter(np.array((dataMat[index,0])),np.array((dataMat[index,1])),marker=marker[i],c='blue')
plt.scatter(np.array(centList[:,0]),np.array(centList[:,1]),s=100,marker='+',c='red')
plt.show()

三、sklearn中的K均值算法

江湖惯例用sklearn中的包试一试：

from sklearn.cluster import KMeans
dataMat = np.mat(loadDataset(r'..\数据集\machinelearninginaction\Ch10\testSet2.txt'))
marker = np.array(['^','s','*','x','o','>','p','h','+','D','d'])    
clusters = KMeans(n_clusters=3, random_state=9).fit(dataMat)
y_pred = clusters.predict(dataMat)
centroids = clusters.cluster_centers_
print(centroids)
print(y_pred)
for i in range(len(centroids)):
    index = np.nonzero(y_pred==i)[0]
    plt.scatter(np.array((dataMat[index,0])),np.array((dataMat[index,1])),marker=marker[i],c='blue')
plt.scatter(np.array(centroids[:,0]),np.array(centroids[:,1]),s=100,marker='+',c='red')
plt.show()