KNN分类器

K-Nearest Neighbor

在训练数据中寻找与测试数据最相似的K个数据，再让这K个数据对测试数据进行投票，最后把票数最高的标签作为对测试图片的预测。

---

1.数据集的处理
下载地址：官网下载地址
对训练集进行的处理：

import pickle
import numpy as np
def down_load(filename):
    with open(filename,'rb') as file:
        data = pickle.load(file,encoding = 'latin1')
    return data
 #整合训练数据
Train_data = []
Train_label=[]
for i in range(1,6):
    filename = 'data_batch_'+str(i)
    data_ = down_load(filename)
    Train_label.append(data_['labels'])
    Train_data.append(data_['data'])
Train_data = np.array(Train_data)
Train_label = np.array(Train_label)
Train_data = Train_data.reshape(Train_data.shape[0]*Train_data.shape[1],Train_data.shape[2])
Train_label = Train_label.reshape(Train_label.shape[0]*Train_label.shape[1])
#处理测试集
test_data_ = down_load('test_batch')
test_data = test_data_['data']
test_label = test_data_['labels']

---

2.模型的构建
首先，我们讨论k=1的情况，也就是Nearest Neighbor分类器(这里使用L1距离）$$d_1(I_1,I_2)=\sum |I_1^p-I_2^p|$$
$$d_2(I_1,I_2)=\sqrt{\sum (I_1^p-I_2^p{)}^2}$$

class NearestNeighbor(object):
    def __inif__(self):
        pass
    def train_model(self,train_data,train_label):
        self.train_d = train_data
        self.train_l = train_label
    def predict(self,test_data):
        pre_ans = np.zeros(test_data.shape[0])
        for i in range(test_data.shape[0]):
            distances = np.sum(np.abs(self.train_d - test_data[i,:]),axis = 1)
            min_dex = np.argmin(distances)
            pre_ans[i] = self.train_l[min_dex]
        return pre_ans

进行训练和预测：

nn = NearestNeighbor()
nn.train_model(Train_data,Train_label)
prediction = nn.predict(test_data)
accuracy = np.mean(prediction==test_label)

最后得到的accuracy为0.2492.可以看到，这个准确率是比较低的。

之后我们来看k>1的情况。此时我们要先找到与预测数据最接近的k张图片，然后对这k张图片进行投票。

from collections import Counter
class k_NearestNeighbor(object):
    def __init__(self):
        pass
    def train_model(self,train_data,train_label):
        self.train_d = train_data
        self.train_l = train_label
    def predict(self,k,test_data):
        pre_ans = np.zeros(test_data.shape[0])
        for i in range(test_data.shape[0]):
            ans_pic = []
            distance = np.sum(np.abs(self.train_d - test_data[i,:]),axis = 1)
            index_order = np.argsort(distance)
            ans_index = index_order[:k]
            #找到最接近预测图像的K张图片
            for j in range(k):
                ans_pic.append(self.train_l[ans_index[j]])
            dic = Counter(ans_pic)
            #对K张图片进行投票
            max_ = max(dic.values())
            pre_ans[i] = list(dic.keys())[list(dic.values()).index(max_)]
        return pre_ans

为了加快训练速度，我们选取部分训练集和部分测试集进行实验

knn = k_NearestNeighbor()
knn.train_model(Train_data[:10000,:],Train_label[:10000])
accuracy = []
for i in range(1,10):
    prediction = knn.predict(i,test_data[:1000,:])
    accuracy.append(np.mean(prediction == test_label[:1000]))

得到k=1~9时的accuracy：[0.208, 0.208, 0.217, 0.229, 0.225, 0.227, 0.222, 0.224, 0.22]

---

KNN优缺点
优点：
1.思想简单，理论成熟
2.可以处理非线性分类和多类别的分类
缺点：
1.训练时间短，测试耗时长
2.两点间的距离公式不能提供足够的信息