KNN入门

版权声明：未经允许，随意转载，请附上本文链接谢谢（づ￣3￣）づ╭❤～
http://blog.csdn.net/xiaoduan_/article/details/79328487

KNN

• k近邻算法是非常特殊的，可以被认为是没有模型的算法
• 为了和其他算法统一，可以认为训练数据集就是模型本身

Sklearn中KNN的简单调用

from sklearn.neighbors import KNeighborsClassifier
kNN_classifier = KNeighborsClassifier(n_neighbors=6)
kNN_classifier.fit(X_train, y_train)
y_predict = kNN_classifier.predict(X_predict)

超参数

• 超参数

在算法运行前需要决定的参数

• 模型参数

算法运行过程中学习的参数

• KNN中没有模型参数

• KNN中的K是典型的超参数

• KNN中的另一个超参数weights

1.考虑距离

2.解决平票问题

参数的选择
uniform：不考虑距离
distance：使用距离
p: 距离的定义

自己封装一个简易版KNN

import numpy as np
from math import sqrt
from collections import Counter


class KNNClassifier:

    def __init__(self, k):
        """初始化kNN分类器"""
        assert k >= 1, "k must be valid"
        self.k = k
        self._X_train = None
        self._y_train = None

    def fit(self, X_train, y_train):
        """根据训练数据集X_train和y_train训练kNN分类器"""
        assert X_train.shape[0] == y_train.shape[0], \
            "the size of X_train must be equal to the size of y_train"
        assert self.k <= X_train.shape[0], \
            "the size of X_train must be at least k."

        self._X_train = X_train
        self._y_train = y_train
        return self

    def predict(self, X_predict):
        """给定待预测数据集X_predict，返回表示X_predict的结果向量"""
        assert self._X_train is not None and self._y_train is not None, \
                "must fit before predict!"
        assert X_predict.shape[1] == self._X_train.shape[1], \
                "the feature number of X_predict must be equal to X_train"

        y_predict = [self._predict(x) for x in X_predict]
        return np.array(y_predict)

    def _predict(self, x):
        """给定单个待预测数据x，返回x的预测结果值"""
        assert x.shape[0] == self._X_train.shape[1], \
            "the feature number of x must be equal to X_train"

        distances = [sqrt(np.sum((x_train - x) ** 2))
                     for x_train in self._X_train]
        nearest = np.argsort(distances)

        topK_y = [self._y_train[i] for i in nearest[:self.k]]
        votes = Counter(topK_y)

        return votes.most_common(1)[0][0]

    def __repr__(self):
        return "KNN(k=%d)" % self.k

KNN的特点

• 解决分类问题
• 天然解决多分类问题
• 思想简单，效果强大
• 还可以用来解决回归问题
  

缺点

• 效率低下
  
• 高度数据相关
• 预测结果具有不可解释性
• 维度灾难
  

KNN整体流程