机器学习实战笔记（k邻近算法）

1.k邻近算法原理

k邻近算法通过计算不同特征值间的距离进行分类

存在一个样本数据集合，也称为训练样本集，样本集中每个数据都存在标签。输入没有标签的新数据后，将输入数据中的每个特征值和训练样本集进行比较，然后选出特征最相似的数据集的标签作为输入数据集标签。一般来说我们选取训练样本集中前k个相似数据，然后选取k个相似数据中出现最多的标签作为输入数据标签

2.python代码实现

import numpy as np
import pylab as pl
import operator
#k邻近算法
def classify0(inX, dataSet, labels, k):
    m = dataSet.shape[0]
    diff = np.tile(inX,(m,1)) - dataSet
    sqDiff = diff ** 2
    sqDistances = sqDiff.sum(axis=1)
    distance = sqDistances ** 0.5
    sortedDistIndicies = distance.argsort()
    classCount = {}
    for i in range(k):
        label = labels[sortedDistIndicies[i]]
        classCount[label] = classCount.get(label,0) + 1
    sortedClassCount = sorted(classCount.iteritems(),key=operator.itemgetter(1),reverse=True)
    return sortedClassCount[0][0]
#将文件转化为数组
def file2data(filename):
    fr = open(filename)
    lines = fr.readlines()
    n = len(lines)
    returnData = np.zeros((n,3))
    labels = []
    index = 0
    for line in lines:
        line = line.strip()
        listFromLine = line.split('\t')
        returnData[index,:] = listFromLine[0:3]
        labels.append(int(listFromLine[-1]))
        index += 1
    return returnData, labels
#将数据归一化处理
def autoNorm(dataSet):
    minVals = dataSet.min(0)
    maxVals = dataSet.man(0)
    ranges = maxVals - minVals
    normDataSet = np.zeros(np.shape(dataSet))
    m = dataSet.shape[0]
    normDataSet = dataSet - np.tile(minVals, (m,1))
    normDataSet = normDataSet/np.tile(ranges,(m,1))
    return normDataSet, ranges, minVals

之后我们将k邻近算法用于手写识别系统

我们的数据放在两个目录里面，其中一个为训练数据，包含大约2000个样本，一个是测试数据包含大约为900个样本，其中的每个样本数据如上图所示，由0和1组成，其中有32行和32列

#将图转化为数组
def img2vector(filename):
    vector = np.zeros((1,1024))
    fr = open(filename)
    for i in range(32):
        line = fr.readline()
        for j in range(32):
            vector[0,32*i+j] = int(line[j])
    return vector
#手写字体的处理
def handwritingClassTest():
    labels = []
    trainingFileList = os.listdir('trainingDigits')
    m = len(trainingFileList)
    trainData = np.zeros((m,1024))
    for i in range(m):
        fileNameStr = trainingFileList[i]
        fileName = fileNameStr.split('.')[0]
        classNum = int(fileName.split('_')[0])
        labels.append(classNum)
        trainData[i,:] = img2vector('trainingDigits/%s'%fileNameStr)
    testFileList = os.listdir('testDigits')
    errorCount = 0.0
    mTest = len(testFileList)
    for i in range(mTest):
        fileNameStr = testFileList[i]
        fileName = fileNameStr.split('.')[0]
        classNum = int(fileName.split('_')[0])
        vectorTest = img2vector('testDigits/%s'%fileNameStr)
        classResult = classify0(vectorTest, trainData, labels, 3)
        if (classResult != classNum) : errorCount += 1.0
    print 'error rate is %f' %(errorCount/float(mTest))