机器学习算法非常重要,基本的算法手动实现一次,有很好的学习效果。下面实现k紧邻算法、
k近邻算法简单、直观。它的核心思想非常简单,给定一个训练数据,对于新的输入实例,在训练数据集中赵傲与该实例最邻近额度k个实例,这k个实例的最多数属于某个类,就把输入实例分成这个类。k近邻模型实际上是对特徵空间的划分,模型的三个基本要素为:距离度量,k值选择和分类决策规则决定。对于距离的度量,我们一般选择为欧氏距离,分类决策的规则为民主投票,即哪一类的结果最多,就分为哪一类。
下面以手写数字识别为例,给出实现的代码:
from numpy import *
import operator
from os import listdir
def classify0(inX, dataSet, labels, k):
dataSetSize = dataSet.shape[0]
diffMat = tile(inX, (dataSetSize,1)) - dataSet # A沿各个维度重复的次数
sqDiffMat = diffMat**2
sqDistances = sqDiffMat.sum(axis=1)#矩阵的每一行相加
distances = sqDistances**0.5
sortedDistIndicies = distances.argsort()
classCount = {}
for i in range(k):
voteIlabel = labels[sortedDistIndicies[i]]
classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1
sortedClassCount = sorted(classCount.iteritems(),key=operator.itemgetter(1),reverse=True)
return sortedClassCount[0][0]
def img2vector(filename):
returnvector = zeros((1,1024))
fr = open(filename)
for i in range(32):
lineStr = fr.readline()
for j in range(32):
returnvector[0,32*i+j] = int(lineStr[j])
return returnvector
def handwritingClasstest():
hwLabels = []
trainingFilelist = listdir('trainingDigits')
m = len(trainingFilelist)
trainingMat = zeros((m,1024))
for i in range(m):
filenameStr = trainingFilelist[i]
fileStr = filenameStr.split('.')[0]
classNumStr = int(fileStr.split('_')[0])
hwLabels.append(classNumStr)
trainingMat[i,:] = img2vector('trainingDigits/%s' % filenameStr)
testFileList = listdir('testDigits')
errorCount = 0.0
mTest = len(testFileList)
for i in range(mTest):
filenameStr = testFileList[i]
fileStr = filenameStr.split('.')[0]
classNumStr = int(fileStr.split('_')[0])
vectorUnderTest = img2vector('testDigits/%s' % filenameStr)
classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, 3)
print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, classNumStr)
if (classifierResult != classNumStr): errorCount += 1.0
print "\nthe total number of errors is: %d" % errorCount
print "\nthe total error rate is: %f" % (errorCount / float(mTest))
if __name__ == '__main__':
handwritingClasstest()
运行之后的结果为:
我们可以看到,使用kNN识别的错误率为1.16%已经非常高了。
例子中使用的数据集下载地址为:http://archive.ics.uci.edu/ml 作者是土耳其的一位教授