K-Nearest Neighbor
在训练数据中寻找与测试数据最相似的K个数据,再让这K个数据对测试数据进行投票,最后把票数最高的标签作为对测试图片的预测。
1.数据集的处理
下载地址:官网下载地址
对训练集进行的处理:
import pickle
import numpy as np
def down_load(filename):
with open(filename,'rb') as file:
data = pickle.load(file,encoding = 'latin1')
return data
#整合训练数据
Train_data = []
Train_label=[]
for i in range(1,6):
filename = 'data_batch_'+str(i)
data_ = down_load(filename)
Train_label.append(data_['labels'])
Train_data.append(data_['data'])
Train_data = np.array(Train_data)
Train_label = np.array(Train_label)
Train_data = Train_data.reshape(Train_data.shape[0]*Train_data.shape[1],Train_data.shape[2])
Train_label = Train_label.reshape(Train_label.shape[0]*Train_label.shape[1])
#处理测试集
test_data_ = down_load('test_batch')
test_data = test_data_['data']
test_label = test_data_['labels']
2.模型的构建
首先,我们讨论k=1的情况,也就是Nearest Neighbor分类器(这里使用L1距离)
d
1
(
I
1
,
I
2
)
=
∑
∣
I
1
p
−
I
2
p
∣
d_1(I_1,I_2) = \sum|I_1^p - I_2^p|
d1(I1,I2)=∑∣I1p−I2p∣
d
2
(
I
1
,
I
2
)
=
∑
(
I
1
p
−
I
2
p
)
2
d_2(I_1,I_2) = \sqrt{\sum(I_1^p - I_2^p)^2}
d2(I1,I2)=∑(I1p−I2p)2
class NearestNeighbor(object):
def __inif__(self):
pass
def train_model(self,train_data,train_label):
self.train_d = train_data
self.train_l = train_label
def predict(self,test_data):
pre_ans = np.zeros(test_data.shape[0])
for i in range(test_data.shape[0]):
distances = np.sum(np.abs(self.train_d - test_data[i,:]),axis = 1)
min_dex = np.argmin(distances)
pre_ans[i] = self.train_l[min_dex]
return pre_ans
进行训练和预测:
nn = NearestNeighbor()
nn.train_model(Train_data,Train_label)
prediction = nn.predict(test_data)
accuracy = np.mean(prediction==test_label)
最后得到的accuracy为0.2492.可以看到,这个准确率是比较低的。
之后我们来看k>1的情况。此时我们要先找到与预测数据最接近的k张图片,然后对这k张图片进行投票。
from collections import Counter
class k_NearestNeighbor(object):
def __init__(self):
pass
def train_model(self,train_data,train_label):
self.train_d = train_data
self.train_l = train_label
def predict(self,k,test_data):
pre_ans = np.zeros(test_data.shape[0])
for i in range(test_data.shape[0]):
ans_pic = []
distance = np.sum(np.abs(self.train_d - test_data[i,:]),axis = 1)
index_order = np.argsort(distance)
ans_index = index_order[:k]
#找到最接近预测图像的K张图片
for j in range(k):
ans_pic.append(self.train_l[ans_index[j]])
dic = Counter(ans_pic)
#对K张图片进行投票
max_ = max(dic.values())
pre_ans[i] = list(dic.keys())[list(dic.values()).index(max_)]
return pre_ans
为了加快训练速度,我们选取部分训练集和部分测试集进行实验
knn = k_NearestNeighbor()
knn.train_model(Train_data[:10000,:],Train_label[:10000])
accuracy = []
for i in range(1,10):
prediction = knn.predict(i,test_data[:1000,:])
accuracy.append(np.mean(prediction == test_label[:1000]))
得到k=1~9时的accuracy:[0.208, 0.208, 0.217, 0.229, 0.225, 0.227, 0.222, 0.224, 0.22]
KNN优缺点
优点:
1.思想简单,理论成熟
2.可以处理非线性分类和多类别的分类
缺点:
1.训练时间短,测试耗时长
2.两点间的距离公式不能提供足够的信息