k-means算法
k-means算法是一种聚类算法,所谓聚类,即根据相似性原则,将具有较高相似度的数据对象划分至同一类簇,将具有较高相异度的数据对象划分至不同类簇。聚类与分类最大的区别在于,分类的目标事先已知,而聚类结果的类别没有预先定义,聚类也称无监督分类。
K-means算法原理
k-means算法中的k代表类簇个数,means代表类簇内数据对象的均值(这种均值是一种对类簇中心的描述),因此,k-means算法又称为k-均值算法。k-means算法是一种基于划分的聚类算法,以距离作为数据对象间相似性度量的标准,即数据对象间的距离越小,则它们的相似性越高,则它们越有可能在同一个类簇。数据对象间距离的计算有很多种,k-means算法通常采用欧氏距离来计算数据对象间的距离。
k-means算法的工作流程:
- 随机确定 k k k个初始点作为质心
- 将数据集中的每个点分配到一个簇中
- 更新质心为该簇所有点的平均值
算法伪代码描述:
代码实现:
import numpy as np
from utils import *
def k_means(data,k,dist_means = dist_eclud, create_cent=rand_cent):
m = data.shape[0]
cluster_assment = np.matrix(np.zeros((m,2)))
centroids = create_cent(data,k)
cluster_changed = True
while cluster_changed:
cluster_changed = False
for i in range(m):
min_dist = np.inf
min_index = -1
for j in range(k):
dist_j_i = dist_means(centroids[j,:], data[i,:])
if dist_j_i<min_dist:
min_dist = dist_j_i
min_index = j
if cluster_assment[i,0] != min_index:
cluster_changed=True
cluster_assment[i,:] = min_index,min_dist**2
for cent in range(k):
ptsInClust = data[np.nonzero(cluster_assment[:,0].A==cent)[0]]
centroids[cent,:] = np.mean(ptsInClust, axis=0)
return centroids,cluster_assment
数据加载:
import numpy as np
import matplotlib.pyplot as plt
# 加载数据
def read_data(path):
data = np.loadtxt(path)
return data
# if __name__ == '__main__':
# data = read_data('./data/testSet.txt')
# plt.scatter(data[:,0], data[:,1])
# plt.show()
测试代码:
import numpy as np
import matplotlib.pyplot as plt
from load_data import *
from k_means import k_means
from bin_k_means import biKmeans
if __name__ == '__main__':
# data = read_data('./data/testSet.txt')
data = read_data('./data/testSet2.txt')
plt.scatter(data[:,0], data[:,1])
cent, clus = k_means(data,4)
plt.scatter(cent[:,0].T.tolist(), cent[:,1].T.tolist())
# cent, clus = biKmeans(data,3)
# cent_x = []
# cent_y = []
# for c in cent:
# cent_x.append(np.array(c)[0][0])
# cent_y.append(np.array(c)[0][1])
# plt.scatter(cent_x, cent_y)
plt.show()
实验结果:
![]() | ![]() |
---|---|
二分k-means算法
为克服k-means算法收敛于局部最小值的问题,提出二分k-means算法,该算法首先将所有点作为一个簇,然后将该簇一分为二,之后选择其中一个簇继续划分,选择哪个簇进行划分取决于对其划分是否可以最大程度降低SSE的值。上述基于SSE的划分过程不断重复,知道得到用指定的簇数为止。
SSE是一种用于度量聚类效果的指标,即误差平方和,为所有簇中的全部数据点到簇中心的误差距离的平方累加和。SSE的值如果越小,表示数据点越接近于它们的簇中心,即质心,聚类效果也越好。因为,对误差取平方后,就会更加重视那些远离中心的数据点。
算法流程:
将所有点看成一个簇
当簇数目小于k时
对于每一个簇
计算总误差
在给定的簇上面进行k-均值聚类(k=2)
计算将该簇一分为二之后的总误差
选择使得总误差最小的簇进行划分
代码实现:
import numpy as np
from utils import *
from k_means import k_means
def biKmeans(dataSet, k, distMeas=dist_eclud):
m = dataSet.shape[0]
clusterAssment = np.matrix(np.zeros((m, 2)))
centroid0 = np.mean(dataSet, axis=0)
centList = [centroid0]
for j in range(m):
clusterAssment[j, 1] = distMeas(np.matrix(centroid0), dataSet[j, :]) ** 2
while len(centList) < k:
lowestSSE = np.inf
for i in range(len(centList)):
ptsInCurrCluster = dataSet[np.nonzero(clusterAssment[:, 0].A == i)[0], :]
centroidMat, splitClustAss = k_means(ptsInCurrCluster, 2, distMeas)
sseSplit = sum(splitClustAss[:, 1])
sseNotSplit = sum(clusterAssment[np.nonzero(clusterAssment[:, 0].A != i)[0], 1])
print("sseSplit, and notSplit: ", sseSplit, sseNotSplit)
if (sseSplit + sseNotSplit) < lowestSSE:
bestCentToSplit = i
bestNewCents = centroidMat
bestClustAss = splitClustAss.copy()
lowestSSE = sseSplit + sseNotSplit
bestClustAss[np.nonzero(bestClustAss[:, 0].A == 1)[0], 0] = len(centList)
bestClustAss[np.nonzero(bestClustAss[:, 0].A == 0)[0], 0] = bestCentToSplit
print('the bestCentToSplit is: ', bestCentToSplit)
print('the len of bestClustAss is: ', len(bestClustAss))
centList[bestCentToSplit] = bestNewCents[0, :]
centList.append(bestNewCents[1, :])
clusterAssment[np.nonzero(clusterAssment[:, 0].A == bestCentToSplit)[0], :] = bestClustAss
return centList, clusterAssment
实验结果: