《机器学习实战》第二章，之K-邻近算法-实验

一、准备：使用Python导入数据：

 

1、首先，创建名为kNN.py的Python模块，在kNN.py文件中添加下面代码：

 

from numpy import *

import operator

def createDataSet():

    group = array([[1.0,1.1],[1.0,1.0],[0,0],[0,0.1]])

    labels = ['A','A','B','B']

return group, labels

 

进入Python开发环境之后，输入下面命令：

 

 

二、实施kNN分类算法：

 

1、在kNN.py中再添加下面代码：

 

def classify0(inX, dataSet, labels, k):

    dataSetSize = dataSet.shape[0]

    diffMat = tile(inX, (dataSetSize,1)) - dataSet

    sqDiffMat = diffMat**2

    sqDistances = sqDiffMat.sum(axis=1)

    distances = sqDistances**0.5

    sortedDistIndicies = distances.argsort()

    classCount={}

    for i in range(k):

        voteIlabel = labels[sortedDistIndicies[i]]

        classCount[voteIlabel] = classCount.get

(voteIlabel,0) + 1

    sortedClassCount = sorted(classCount.items(),

     key=operator.itemgetter(1), reverse=True)

    return sortedClassCount[0][0]

 

进入Python开发环境之后，输入下面命令：

 

 

得到结果是B，正确。

 

三、使用K-近邻算法改进约会网站的配对效果

 

准备数据：从文本文件中解析数据：

    

1、在kNN.py添加下面代码：

    

    def file2matrix(filename):

    fr = open(filename)

    arrayOLines = fr.readlines()

    numberOfLines = len(arrayOLines)

    returnMat = zeros((numberOfLines,3))

    classLabelVector = []

    index = 0

    for line in arrayOLines:

        line = line.strip()

        listFromLine = line.split('\t')

        returnMat[index,:] = listFromLine[0:3]

        classLabelVector.append(int(listFromLine[-1]))

        index += 1

return returnMat,classLabelVector

 

 

在Python命令提示符里输入reload(kNN)时，始终都是报错，后来发现是Python的版本的问题，我的版本是Python3.6.3，正确用法如下图：

 

四、分析数据：使用Matplotlib创建散点图

 

1、首先我们使用Matplotlib制作原始数据的散点图，看下图：

 

 

2、由于没有使用样本分来的特征值，我们很难从图中看到任何有用的数据模式信息。Matplotlib库提供scstter函数支持个性化标记散点图上的点。调用scatter函数。看下图：

 

在Python命令提示符中会出现上图错误，此错误可能是我的Matplotlib库版本的问题，有待解决。

在Jupyter中运行可以成功，datingDataMat矩阵的第二第三列属性，看下图：

 

 

datingDataMat矩阵的第一第二列属性，看下图：

 

 

 

 

五、准备数据：归一化数值

 

1、在kNN.py中添加下面代码：

 

 def autoNorm(dataSet):

    minVals = dataSet.min(0)

    maxVals = dataSet.max(0)

    ranges = maxVals - minVals

    normDataSet = zeros(shape(dataSet))

    m = dataSet.shape[0]

    normDataSet = dataSet - tile(minVals, (m,1))

    normDataSet = normDataSet/tile(ranges, (m,1))

    #element wise divide

return normDataSet, ranges, minVals

 

六、测试算法：作为完整程序验证分类器

 

1、在kNN.py中添加下面代码：

 

def datingClassTest():

    hoRatio = 0.1

    datingDataMat,datingLabels = file2matrix('datingTestSet2.txt')

    normMat, ranges, minVals = autoNorm(datingDataMat)

    m = normMat.shape[0]

    numTestVecs = int(m*hoRatio)

    errorCount = 0.0

    for i in range(numTestVecs):

        classifierResult=  classify0(normMat[i,:],normMat[numTestVecs:m,:],datingLabels[numTestVecs:m],3)

        Print（ "the classifier came back with: %d, the real answer is: %d" % (classifierResult, datingLabels[i])）

        if (classifierResult != datingLabels[i]): errorCount += 1.0

print ("the total error rate is: %f" % (errorCount/float(numTestVecs)))

 

 

七、使用算法：构建完整可用系统

 

1、在kNN.py里添加下面代码：

 

def classifyPerson():  

    resultList = ["not at all","in small does","in large does"]  

    percentTats = float(input("percentage of time spent playing video games?"))  

    ffMiles = float(input("frequent flier miles earned per year?"))  

    iceCream = float(input("liters of ice cream consumes per year?"))  

    datingDataMat,datingLabels = file2matrix('datingTestSet2.txt')  

    normMat,ranges,minVals = autoNorm(datingDataMat)  

    inArr = array([ffMiles,percentTats,iceCream])  

classifierResult = classify0(((inArr-minVals)/ranges),datingDataMat,datingLabels,3)

print("You will probably like this person:",resultList[classifierResult - 1])

 

 

八、手写识别系统

    

准备数据：将图像转换为测试向量

 

1、在kNN.py中添加下面代码：

 

def img2vector(filename):

    returnVect = zeros((1,1024))

    fr = open(filename)

    for i in range(32):

        lineStr = fr.readline()

        for j in range(32):

            returnVect[0,32*i+j] = int(lineStr[j])

return returnVect

 

 

九、测试算法：使用k-近邻算法识别手写数字

 

1、在kNN.py中添加下面代码：

 

from os import listdir

def handwritingClassTest():

    hwLabels = []

    trainingFileList = listdir('trainingDigits')

    m = len(trainingFileList)

    trainingMat = zeros((m,1024))

    for i in range(m):

        fileNameStr = trainingFileList[i]

        fileStr = fileNameStr.split('.')[0]

        classNumStr = int(fileStr.split('_')[0])

        hwLabels.append(classNumStr)

        trainingMat[i,:] = img2vector('trainingDigits/%s' % fileNameStr)

    testFileList = listdir('testDigits')

    errorCount = 0.0

    mTest = len(testFileList)

    for i in range(mTest):

        fileNameStr = testFileList[i]

        fileStr = fileNameStr.split('.')[0]

        classNumStr = int(fileStr.split('_')[0])

        vectorUnderTest = img2vector('testDigits/%s' % fileNameStr)

        classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, 3)

        print ("the classifier came back with: %d, the real answer is: %d" % (classifierResult, classNumStr))

        if (classifierResult != classNumStr):errorCount += 1.0

    print ("\nthe total number of errors is: %d" % errorCount)

    print ("\nthe total error rate is: %f" % (errorCount/float(mTest)))

操作异常问题与解决方案

一、准备：使用Python导入数据：

1、在Python环境中输入命令import kNN 提示错误， No module named numpy，

发现自己电脑里没有安装numpy，通过网上下载相应版本和安装之后，问题得到解决。

 

二、实施kNN分类算法：

1、在Python3.X中，item（）替换了iteritems()，课本源码是iteritems()，我用的是Python3.6.3版本，所以把iteritems()改成item（），问题得到解决。

2、修改完代码之后，重启Python环境，直接输入kNN.classify0([0.0],group,labels,3),发现报错，此时需要重复步骤一命令即可。

 

三、准备数据：从文本文件中解析数据

1、在Python命令提示符里输入reload(kNN)时，始终都是报错，后来发现是Python的版本的问题，我的版本是Python3.6.3，正确用法如下图：

2、在Python命令提示符中：

ax.scatter(datingDataMat[:,1],datingDataMat[:,2],15.0*array(datingLabels),15.0*array(datingLabels))无法编译通过，提示信息是：name ‘array’is not defined,此错误是我的Python3.6.3和Matplotlib版本的问题。

 

六、测试算法：作为完整程序验证分类器

1、课表代码两条print语句各缺了一对括号，并把print请raw_去掉。

 

七、使用算法：构建完整可用系统

1、课表代码一条print语句各缺了一对括号。

 

九、测试算法：使用k-近邻算法识别手写数字

 

1、如果是在Python提示符中做实验，那么testDigits和trainingDigits文件要放在Python的安装目录下。

2、如果是在jupyter中实验，那么路径要改为详细路径，如：F:\Softwares\Python\testDigits；F:\Softwares\Python\trainingDigits

 

from numpy import *

import operator


def createDataSet():

    group = array([[1.0,1.1],[1.0,1.0],[0,0],[0,0.1]])

    labels = ['A','A','B','B']

    return group, labels



def classify0(inX, dataSet, labels, k):

    dataSetSize = dataSet.shape[0]

    diffMat = tile(inX, (dataSetSize,1)) - dataSet

    sqDiffMat = diffMat**2

    sqDistances = sqDiffMat.sum(axis=1)

    distances = sqDistances**0.5

    sortedDistIndicies = distances.argsort()
    classCount={}
    for i in range(k):

        voteIlabel = labels[sortedDistIndicies[i]]

        classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1

    sortedClassCount = sorted(classCount.items(),
     key=operator.itemgetter(1), reverse=True)

    return sortedClassCount[0][0]

def file2matrix(filename):
    fr = open(filename)
    arrayOLines = fr.readlines()
    numberOfLines = len(arrayOLines)
    returnMat = zeros((numberOfLines,3))
    classLabelVector = []
    index = 0 
    for line in arrayOLines:
        line = line.strip()
        listFromLine = line.split('\t')
        returnMat[index,:] = listFromLine[0:3]
        classLabelVector.append(int(listFromLine[-1]))
        index += 1
    return returnMat,classLabelVector

def autoNorm(dataSet):

    minVals = dataSet.min(0)

    maxVals = dataSet.max(0)

    ranges = maxVals - minVals

    normDataSet = zeros(shape(dataSet))

    m = dataSet.shape[0]

    normDataSet = dataSet - tile(minVals, (m,1))

    normDataSet = normDataSet/tile(ranges, (m,1))
   

    return normDataSet, ranges, minVals

def datingClassTest():

    hoRatio = 0.1
    datingDataMat,datingLabels = file2matrix('datingTestSet2.txt')
    normMat, ranges, minVals = autoNorm(datingDataMat)
    m = normMat.shape[0]
    numTestVecs = int(m*hoRatio)
    errorCount = 0.0
    for i in range(numTestVecs):
        classifierResult = classify0(normMat[i,:],normMat[numTestVecs:m,:],datingLabels[numTestVecs:m],3)
        print ("the classifier came back with: %d,the real answer is: %d"%(classifierResult, datingLabels[i]))
        if (classifierResult != datingLabels[i]): errorCount += 1.0
    print ("the total error rate is: %f" % (errorCount/float(numTestVecs)))

def classifyPerson():  
    resultList = ["not at all","in small does","in large does"]  
    percentTats = float(input("percentage of time spent playing video games?"))  
    ffMiles = float(input("frequent flier miles earned per year?"))  
    iceCream = float(input("liters of ice cream consumes per year?"))  
    datingDataMat,datingLabels = file2matrix('datingTestSet2.txt')  
    normMat,ranges,minVals = autoNorm(datingDataMat)  
    inArr = array([ffMiles,percentTats,iceCream])  
    classifierResult = classify0(((inArr-minVals)/ranges),datingDataMat,datingLabels,3)  
    print("You will probably like this person:",resultList[classifierResult - 1])

def img2vector(filename):

    returnVect = zeros((1,1024))

    fr = open(filename)

    for i in range(32):

        lineStr = fr.readline()

        for j in range(32):

            returnVect[0,32*i+j] = int(lineStr[j])

    return returnVect

from os import listdir
def handwritingClassTest():

    hwLabels = []

    trainingFileList = listdir('trainingDigits')
    m = len(trainingFileList)
    trainingMat = zeros((m,1024))

    for i in range(m):

        fileNameStr = trainingFileList[i]

        fileStr = fileNameStr.split('.')[0]
        classNumStr = int(fileStr.split('_')[0])

        hwLabels.append(classNumStr)

        trainingMat[i,:] = img2vector('trainingDigits/%s' % fileNameStr)

    testFileList = listdir('testDigits')
    errorCount = 0.0

    mTest = len(testFileList)

    for i in range(mTest):

        fileNameStr = testFileList[i]

        fileStr = fileNameStr.split('.')[0]
        classNumStr = int(fileStr.split('_')[0])

        vectorUnderTest = img2vector('testDigits/%s' % fileNameStr)

        classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, 3)

        print ("the classifier came back with: %d, the real answer is: %d" % (classifierResult, classNumStr))

        if (classifierResult != classNumStr):errorCount += 1.0
    
    print ("\nthe total number of errors is: %d" % errorCount)
    print ("\nthe total error rate is: %f" % (errorCount/float(mTest)))