机器学习之python 实现knn算法

机器学习之实现KNN算法

• 本篇博客使用python简易的实现一个 knn 算法，方便更好的理解knn算法 ，不涉及调库。适合于新手但已学过knn的小白，在学完knn后，动手做一个实现knn算法的小项目可以更好的理解knn
• 本篇写的是一个分类的问题
• 任务 预测某一点属于哪一类
• 训练数据的点t及其对应分类

  rain_example = [[1,1],[2,2],[3,3],[20,20],[25,25],[30,30]]
  train_labels = ['a','a','a','b','b','b']
  

• 待分类的数据

  test = [[4,4],[5,5],[27,27]]
  

• 首先实现距离度量的函数，本篇采用实现欧式距离

  def get_distance (x,y):
      if len(x) == len (y):
          sum = 0
          for i in range(len(x)):
              sum =sum+ math.pow(abs(x[i]-y[i]),2)
          return pow(sum,1/2)
      else:
          return 0
  

• 有了距离计算函数后，knn中会根据距离选出最近的k个，接下来实现 选出最近的k个的功能
  • 实现思路：先遍历一次训练集，计算出待预测点与所有样本的距离，并将其距离和标签存入distance_queue list中，存入时以元组形式存入
  • 接下来，只需对distance_queue 按距离进行排序
  • 取出前k个（即最近的k个值所对应的标签）返回

  
  def get_nearst_K_labels(trainset,trainlabels,testset,k):
      distance_queue = []
      for i in range(len(trainset)):
          x = trainset[i]
          distance = get_distance(testset,x)
          distance_queue.append((distance,trainlabels[i]))
          # print (distance_queue)
      distance_queue.sort(key=operator.itemgetter(0))
      print (distance_queue)
      k_neighbors = []
      for i in range(k):
          k_neighbors.append(distance_queue[i][1])
      return k_neighbors
  
  

• 预测分类
  • 将得到的前k个标签，进行统计，选出次数最多的作位 预测值（多数表决）

  def get_max_labels(k_neighbors):
      label = {}
      for i in k_neighbors:
          if i in label:
              label[i] = label[i]+1
          else:
              label[i] = 1
      max_label = sorted(label.items(),key=operator.itemgetter(1),reverse=True)
      return max_label[0][0]
  

• 综上，已经完成了实现knn算法。
• 全部代码：

import math 
import operator
def get_distance (x,y):
    if len(x) == len (y):
        sum = 0
        for i in range(len(x)):
            sum =sum+ math.pow(abs(x[i]-y[i]),2)
        return pow(sum,1/2)
    else:
        return 0



def get_nearst_K_labels(trainset,trainlabels,testset,k):
    distance_queue = []
    for i in range(len(trainset)):
        x = trainset[i]
        distance = get_distance(testset,x)
        distance_queue.append((distance,trainlabels[i]))
        # print (distance_queue)
    distance_queue.sort(key=operator.itemgetter(0))
    print (distance_queue)
    k_neighbors = []
    for i in range(k):
        k_neighbors.append(distance_queue[i][1])
    return k_neighbors

def get_max_labels(k_neighbors):
    label = {}
    for i in k_neighbors:
        if i in label:
            label[i] = label[i]+1
        else:
            label[i] = 1
    max_label = sorted(label.items(),key=operator.itemgetter(1),reverse=True)
    return max_label[0][0]

train_example = [[1,1],[2,2],[3,3],[20,20],[25,25],[30,30]]
train_labels = ['a','a','a','b','b','b']
test = [[4,4],[5,5],[27,27]]
for i in test:
    labels_example = get_nearst_K_labels(train_example,train_labels,i,3)
    print(get_max_labels(labels_example))

• 结果