1.k邻近算法原理
k邻近算法通过计算不同特征值间的距离进行分类
存在一个样本数据集合,也称为训练样本集,样本集中每个数据都存在标签。输入没有标签的新数据后,将输入数据中的每个特征值和训练样本集进行比较,然后选出特征最相似的数据集的标签作为输入数据集标签。一般来说我们选取训练样本集中前k个相似数据,然后选取k个相似数据中出现最多的标签作为输入数据标签
2.python代码实现
import numpy as np
import pylab as pl
import operator
#k邻近算法
def classify0(inX, dataSet, labels, k):
m = dataSet.shape[0]
diff = np.tile(inX,(m,1)) - dataSet
sqDiff = diff ** 2
sqDistances = sqDiff.sum(axis=1)
distance = sqDistances ** 0.5
sortedDistIndicies = distance.argsort()
classCount = {}
for i in range(k):
label = labels[sortedDistIndicies[i]]
classCount[label] = classCount.get(label,0) + 1
sortedClassCount = sorted(classCount.iteritems(),key=operator.itemgetter(1),reverse=True)
return sortedClassCount[0][0]
#将文件转化为数组
def file2data(filename):
fr = open(filename)
lines = fr.readlines()
n = len(lines)
returnData = np.zeros((n,3))
labels = []
index = 0
for line in lines:
line = line.strip()
listFromLine = line.split('\t')
returnData[index,:] = listFromLine[0:3]
labels.append(int(listFromLine[-1]))
index += 1
return returnData, labels
#将数据归一化处理
def autoNorm(dataSet):
minVals = dataSet.min(0)
maxVals = dataSet.man(0)
ranges = maxVals - minVals
normDataSet = np.zeros(np.shape(dataSet))
m = dataSet.shape[0]
normDataSet = dataSet - np.tile(minVals, (m,1))
normDataSet = normDataSet/np.tile(ranges,(m,1))
return normDataSet, ranges, minVals
之后我们将k邻近算法用于手写识别系统
我们的数据放在两个目录里面,其中一个为训练数据,包含大约2000个样本,一个是测试数据包含大约为900个样本,其中的每个样本数据如上图所示,由0和1组成,其中有32行和32列
#将图转化为数组
def img2vector(filename):
vector = np.zeros((1,1024))
fr = open(filename)
for i in range(32):
line = fr.readline()
for j in range(32):
vector[0,32*i+j] = int(line[j])
return vector
#手写字体的处理
def handwritingClassTest():
labels = []
trainingFileList = os.listdir('trainingDigits')
m = len(trainingFileList)
trainData = np.zeros((m,1024))
for i in range(m):
fileNameStr = trainingFileList[i]
fileName = fileNameStr.split('.')[0]
classNum = int(fileName.split('_')[0])
labels.append(classNum)
trainData[i,:] = img2vector('trainingDigits/%s'%fileNameStr)
testFileList = os.listdir('testDigits')
errorCount = 0.0
mTest = len(testFileList)
for i in range(mTest):
fileNameStr = testFileList[i]
fileName = fileNameStr.split('.')[0]
classNum = int(fileName.split('_')[0])
vectorTest = img2vector('testDigits/%s'%fileNameStr)
classResult = classify0(vectorTest, trainData, labels, 3)
if (classResult != classNum) : errorCount += 1.0
print 'error rate is %f' %(errorCount/float(mTest))