优点:精度高,对异常值不敏感,无数据输入假定,不做分布形式的假设.
缺点:计算复杂度高,空间复杂度高
其伪代码如下:
对未知类别属性的数据集中的每个点依次执行以下操作:
(1)计算已知类别数据集中的点与当前点之间的距离;
(2)按照距离递增次序排序
(3)选取与当前距离最小的k个点
(4)确定前k个点所在类别的出现频率
(5)返回前k个点中出现频率最高的类别作为当前点的预测分类
python实现代码如下:
import numpy as np
import operator
class KNN:
def createdata(self):
dataset=np.array([[1,2,3],[4,5,6],[3,2,4]])
labels=['A','B','B']
return dataset,labels
def classify(self,intx,dataset,labels,k):
#calculate the distance between test instance intx and other training samples
datasize=dataset.shape[0]
diffmat=np.tile(intx,(datasize,1))-dataset#repeate intx to the size of the training samples
diffmat=diffmat**2
distance=diffmat.sum(axis=1) #the sum of per line
distance=distance**0.5
sortdiffidx=distance.argsort()
vote={}
for i in range(k):
votelabel=labels[sortdiffidx[i]]
vote[votelabel]=vote.get(votelabel,0)+1 #get(votelabel,0):if dictionary 'vote' exist key 'votelabel',return vote[votelabel];else return 0
sortedvote=sorted(vote.iteritems(),key=lambda x:x[1],reverse=True)# key can be replace by key=operator.itemgetter(1),sort by the value of vote dictionary
return sortedvote[0][0]
k=KNN()
group,labels=k.createdata()
cl=k.classify([0,0,0],group,labels,3)
print cl