机器学习---K-Nearest Neighbor算法应用二(手动实现KNN算法)

最新推荐文章于 2023-07-01 15:15:11 发布

四果汤多加陈皮才酸爽

最新推荐文章于 2023-07-01 15:15:11 发布

阅读量479

点赞数 1

分类专栏：机器学习

机器学习专栏收录该内容

10 篇文章 0 订阅

订阅专栏

原始数据

测试数据集irisdata.txt
格式如下,
其中，Iris-setosa/Iris-versicolor/Iris-virginica表示类别

5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa
4.7,3.2,1.3,0.2,Iris-setosa
4.6,3.1,1.5,0.2,Iris-setosa
5.0,3.6,1.4,0.2,Iris-setosa
5.4,3.9,1.7,0.4,Iris-setosa
4.6,3.4,1.4,0.3,Iris-setosa
5.0,3.4,1.5,0.2,Iris-setosa
4.4,2.9,1.4,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
5.4,3.7,1.5,0.2,Iris-setosa
4.8,3.4,1.6,0.2,Iris-setosa
4.8,3.0,1.4,0.1,Iris-setosa
4.3,3.0,1.1,0.1,Iris-setosa
5.8,4.0,1.2,0.2,Iris-setosa
5.7,4.4,1.5,0.4,Iris-setosa
5.4,3.9,1.3,0.4,Iris-setosa
5.1,3.5,1.4,0.3,Iris-setosa
5.7,3.8,1.7,0.3,Iris-setosa
5.1,3.8,1.5,0.3,Iris-setosa
5.4,3.4,1.7,0.2,Iris-setosa
5.1,3.7,1.5,0.4,Iris-setosa
4.6,3.6,1.0,0.2,Iris-setosa
5.1,3.3,1.7,0.5,Iris-setosa
4.8,3.4,1.9,0.2,Iris-setosa
5.0,3.0,1.6,0.2,Iris-setosa
5.0,3.4,1.6,0.4,Iris-setosa
5.2,3.5,1.5,0.2,Iris-setosa
5.2,3.4,1.4,0.2,Iris-setosa
4.7,3.2,1.6,0.2,Iris-setosa
4.8,3.1,1.6,0.2,Iris-setosa
5.4,3.4,1.5,0.4,Iris-setosa
5.2,4.1,1.5,0.1,Iris-setosa
5.5,4.2,1.4,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
5.0,3.2,1.2,0.2,Iris-setosa
5.5,3.5,1.3,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
4.4,3.0,1.3,0.2,Iris-setosa
5.1,3.4,1.5,0.2,Iris-setosa
5.0,3.5,1.3,0.3,Iris-setosa
4.5,2.3,1.3,0.3,Iris-setosa
4.4,3.2,1.3,0.2,Iris-setosa
5.0,3.5,1.6,0.6,Iris-setosa
5.1,3.8,1.9,0.4,Iris-setosa
4.8,3.0,1.4,0.3,Iris-setosa
5.1,3.8,1.6,0.2,Iris-setosa
4.6,3.2,1.4,0.2,Iris-setosa
5.3,3.7,1.5,0.2,Iris-setosa
5.0,3.3,1.4,0.2,Iris-setosa
7.0,3.2,4.7,1.4,Iris-versicolor
6.4,3.2,4.5,1.5,Iris-versicolor
6.9,3.1,4.9,1.5,Iris-versicolor
5.5,2.3,4.0,1.3,Iris-versicolor
6.5,2.8,4.6,1.5,Iris-versicolor
5.7,2.8,4.5,1.3,Iris-versicolor
6.3,3.3,4.7,1.6,Iris-versicolor
4.9,2.4,3.3,1.0,Iris-versicolor
6.6,2.9,4.6,1.3,Iris-versicolor
5.2,2.7,3.9,1.4,Iris-versicolor
5.0,2.0,3.5,1.0,Iris-versicolor
5.9,3.0,4.2,1.5,Iris-versicolor
6.0,2.2,4.0,1.0,Iris-versicolor
6.1,2.9,4.7,1.4,Iris-versicolor
5.6,2.9,3.6,1.3,Iris-versicolor
6.7,3.1,4.4,1.4,Iris-versicolor
5.6,3.0,4.5,1.5,Iris-versicolor
5.8,2.7,4.1,1.0,Iris-versicolor
6.2,2.2,4.5,1.5,Iris-versicolor
5.6,2.5,3.9,1.1,Iris-versicolor
5.9,3.2,4.8,1.8,Iris-versicolor
6.1,2.8,4.0,1.3,Iris-versicolor
6.3,2.5,4.9,1.5,Iris-versicolor
6.1,2.8,4.7,1.2,Iris-versicolor
6.4,2.9,4.3,1.3,Iris-versicolor
6.6,3.0,4.4,1.4,Iris-versicolor
6.8,2.8,4.8,1.4,Iris-versicolor
6.7,3.0,5.0,1.7,Iris-versicolor
6.0,2.9,4.5,1.5,Iris-versicolor
5.7,2.6,3.5,1.0,Iris-versicolor
5.5,2.4,3.8,1.1,Iris-versicolor
5.5,2.4,3.7,1.0,Iris-versicolor
5.8,2.7,3.9,1.2,Iris-versicolor
6.0,2.7,5.1,1.6,Iris-versicolor
5.4,3.0,4.5,1.5,Iris-versicolor
6.0,3.4,4.5,1.6,Iris-versicolor
6.7,3.1,4.7,1.5,Iris-versicolor
6.3,2.3,4.4,1.3,Iris-versicolor
5.6,3.0,4.1,1.3,Iris-versicolor
5.5,2.5,4.0,1.3,Iris-versicolor
5.5,2.6,4.4,1.2,Iris-versicolor
6.1,3.0,4.6,1.4,Iris-versicolor
5.8,2.6,4.0,1.2,Iris-versicolor
5.0,2.3,3.3,1.0,Iris-versicolor
5.6,2.7,4.2,1.3,Iris-versicolor
5.7,3.0,4.2,1.2,Iris-versicolor
5.7,2.9,4.2,1.3,Iris-versicolor
6.2,2.9,4.3,1.3,Iris-versicolor
5.1,2.5,3.0,1.1,Iris-versicolor
5.7,2.8,4.1,1.3,Iris-versicolor
6.3,3.3,6.0,2.5,Iris-virginica
5.8,2.7,5.1,1.9,Iris-virginica
7.1,3.0,5.9,2.1,Iris-virginica
6.3,2.9,5.6,1.8,Iris-virginica
6.5,3.0,5.8,2.2,Iris-virginica
7.6,3.0,6.6,2.1,Iris-virginica
4.9,2.5,4.5,1.7,Iris-virginica
7.3,2.9,6.3,1.8,Iris-virginica
6.7,2.5,5.8,1.8,Iris-virginica
7.2,3.6,6.1,2.5,Iris-virginica
6.5,3.2,5.1,2.0,Iris-virginica
6.4,2.7,5.3,1.9,Iris-virginica
6.8,3.0,5.5,2.1,Iris-virginica
5.7,2.5,5.0,2.0,Iris-virginica
5.8,2.8,5.1,2.4,Iris-virginica
6.4,3.2,5.3,2.3,Iris-virginica
6.5,3.0,5.5,1.8,Iris-virginica
7.7,3.8,6.7,2.2,Iris-virginica
7.7,2.6,6.9,2.3,Iris-virginica
6.0,2.2,5.0,1.5,Iris-virginica
6.9,3.2,5.7,2.3,Iris-virginica
5.6,2.8,4.9,2.0,Iris-virginica
7.7,2.8,6.7,2.0,Iris-virginica
6.3,2.7,4.9,1.8,Iris-virginica
6.7,3.3,5.7,2.1,Iris-virginica
7.2,3.2,6.0,1.8,Iris-virginica
6.2,2.8,4.8,1.8,Iris-virginica
6.1,3.0,4.9,1.8,Iris-virginica
6.4,2.8,5.6,2.1,Iris-virginica
7.2,3.0,5.8,1.6,Iris-virginica
7.4,2.8,6.1,1.9,Iris-virginica
7.9,3.8,6.4,2.0,Iris-virginica
6.4,2.8,5.6,2.2,Iris-virginica
6.3,2.8,5.1,1.5,Iris-virginica
6.1,2.6,5.6,1.4,Iris-virginica
7.7,3.0,6.1,2.3,Iris-virginica
6.3,3.4,5.6,2.4,Iris-virginica
6.4,3.1,5.5,1.8,Iris-virginica
6.0,3.0,4.8,1.8,Iris-virginica
6.9,3.1,5.4,2.1,Iris-virginica
6.7,3.1,5.6,2.4,Iris-virginica
6.9,3.1,5.1,2.3,Iris-virginica
5.8,2.7,5.1,1.9,Iris-virginica
6.8,3.2,5.9,2.3,Iris-virginica
6.7,3.3,5.7,2.5,Iris-virginica
6.7,3.0,5.2,2.3,Iris-virginica
6.3,2.5,5.0,1.9,Iris-virginica
6.5,3.0,5.2,2.0,Iris-virginica
6.2,3.4,5.4,2.3,Iris-virginica
5.9,3.0,5.1,1.8,Iris-virginica

源码

# !/usr/bin/env python
# -*- coding: utf-8 -*-
# Author: Justin Chan

import csv
import random
import math
import operator

#装载数据集函数
#filename数据集文件
#filename数据集分为两部分，一个是训练集trainingSet,一个是测试集testSet.
#split用来将原始数据集分为训练集和测试集的界限。

def loadDataset(filename, split, trainingSet = [], testSet = []):
    with open(filename,'rb') as csvfile:
        lines = csv.reader(csvfile)#读取所有行
        dataset = list(lines)
        for x in range(len(dataset)-1):
            for y in range(4):
                dataset[x][y] = float(dataset[x][y])
            if random.random() < split:#随机一个数与split比较，大的装载到测试集，小的装载的数据集。
                trainingSet.append(dataset[x])
            else:
                testSet.append(dataset[x])

#计算坐标两个实例的直线距离,length表示维度
def euclideanDistance(instance1,instance2,length):
    distance = 0
    for x in range(length):
        distance += pow((instance1[x] - instance2[x]),2)#计算两个点所有维度的平方和
    return math.sqrt(distance)

#返回最近的k个邻居
def getNeighbors(trainingSet,testInstance,k):
    distances = []
    length = len(testInstance)-1 #测试实例的维度
    for x in range(len(trainingSet)):
        dist = euclideanDistance(testInstance,trainingSet[x],length)
        distances.append((trainingSet[x],dist))
    distances.sort(key=operator.itemgetter(1))#把距离从小到大排序
    neighbors = []
    for x in range(k):
        neighbors.append(distances[x][0])
    return neighbors

#在邻居中取得最近的那个邻居
def getResponse(neighbors):
    classVotes = {}
    for x in range(len(neighbors)):
        response = neighbors[x][-1]
        if response in classVotes:
            classVotes[response] += 1
        else:
            classVotes[response] = 1
    sortedVotes = sorted(classVotes.items(),key=operator.itemgetter(1),reverse=True)#每个类进行降序排列
    return sortedVotes[0][0]

#计算测试集的准确率
def getAccuracy(testSet,predictions):
    correct = 0#预测对的个数
    for x in range(len(testSet)):
        if testSet[x][-1] == predictions[x]:#下标-1即表示最后一个值,这里就是label
            correct += 1
    return (correct/float(len(testSet)))*100.0

def main():
    #prepare data
    trainingSet = []
    testSet = []
    split = 0.67
    loadDataset(r'/home/justin18chan/PycharmProjects/tensorflowProj/KNearestNeighbor/irisdata.txt', split, trainingSet, testSet)#在传路径时，加r表示后面传的是原始字符串，忽略特殊字符
    print('Train set:' + repr(len(trainingSet)))
    print('Test set:' + repr(len(testSet)))
    #gengrate prediction
    predictions = []
    k = 3
    for x in range(len(testSet)):
        neighbors = getNeighbors(trainingSet, testSet[x], k)
        result = getResponse(neighbors)
        predictions.append(result)
        print('> predicted' + repr(result) + ',actual' + repr(testSet[x][-1]))
    accuracy = getAccuracy(testSet,predictions)
    print('Accuracy:' + repr(accuracy) + '%')

main()

遇到的问题

1.在读取csv文件时，打开文件格式错误，要用‘rt’打开。

    with open(filename,'rb') as csvfile:
        lines = csv.reader(csvfile)#读取所有行
        dataset = list(lines)

_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?
原因是csv文件是文本文件，不是二进制文件，不能用‘rb’格式打开。文本文件打开要用rt。

2.在python3.X中是classVotes.items()，在2.X是iteritems().

sorted(classVotes.items(),key=operator.itemgetter(1),reverse=True)

运算结果

/usr/bin/python3.5 xxxxx/KNearestNeighbor/KnnImplementation.py
Train set:92
Test set:57
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-setosa',actual'Iris-setosa'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-virginica',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-versicolor',actual'Iris-versicolor'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-versicolor',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-versicolor',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
> predicted'Iris-virginica',actual'Iris-virginica'
Accuracy:94.73684210526315%

四果汤多加陈皮才酸爽

关注

1
点赞
踩
3

收藏

觉得还不错? 一键收藏
0
评论
机器学习---K-Nearest Neighbor算法应用二(手动实现KNN算法)

测试数据集 irisdata.txt 格式如下 5.1,3.5,1.4,0.2,Iris-setosa 7.0,3.2,4.7,1.4,Iris-versicolor 6.3,3.3,6.0,2.5,Iris-virginica5.1,3.5,1.4,0.2,Iris-setosa4.9,3.0,1.4,0.2,Iris-setosa4.7,3.2,1.3,0.2,Iris-setos
复制链接

扫一扫