K近邻算法的具体思想如下:
(1)计算已知类别数据集中的样本与当前样本之间的距离
(2)按照距离递增次序排序
(3)选取与当前点距离最小的k个样本
(4)确定前k个样本所在类别的出现频率
(5)返回前k个样本中出现频率最高的类别作为当前点的预测分类
python实现
def define_classification(self, sample, trainSet, labels, K=1):
sortdiffidx = self.calc_distance_between_sample_trainSet(sample, trainSet, labels)
vote = {}
for i in range(K):
ith_label = labels[sortdiffidx[i]]
vote[ith_label] = vote.get(ith_label, 0) + 1
sortedvote = self.dict_sort(vote)
return sortedvote[0][0]
def calc_distance_between_sample_trainSet(self, sample, trainSet, labels):
N = trainSet.shape[0]
difference = tile(sample, (N, 1)) - trainSet
squared_difference = difference**2
squared_difference = sum(squared_difference, axis=1)
distance = squared_difference**0.5
sorted_differ = distance.argsort()
return sorted_differ
def dict_sort(self, dict_for_sort):
return sorted(dict_for_sort.iteritems(), key=lambda x: x[1], reverse=True)
def KnnClassify(self, testX, trainX, labels, K=1):
return self.define_classification(testX, trainX, labels, K)
其中testX是需要进行识别的样本,trainX是已知类别的训练集样本集合,labels对应着样本集合中每个样本的类别,返回的是类别
testX形如[1,2,3..........]
trainX形如
[
[1,2,3..........]
.........
[1,2,3..........]
]
labels形如['c1','c2','c3',........]
实际上,python库中已经做好的相应的封装,只要导入sklearn这个库,就能使用其提供的K近邻算法接口。