KNN-sklearn实现

简单学习及使用scikit-learn库中的neighbors.KNeighborsClassifier

数据集采用UCI中的Breast Cancer Wisconsin (Diagnostic).

from sklearn.datasets import load_breast_cancer 
from sklearn.model_selection import train_test_split

data = load_breast_cancer()
data["data"] #X
data["target"]#y

clf = KNeighborsClassifier(n_neighbors=5)   #实例化，k = 5
#划分训练集与测试集
xtrain,xtest,ytrain,ytest = train_test_split(data["data"],data["target"],test_size=0.3)
#拟合数据
clf.fit(xtrain,ytrain)
[clf.score(xtrain,ytrain),clf.score(xtest,ytest)]

可以得到,
score = [0.949748743718593, 0.9239766081871345]

描绘一条学习曲线，观测不同k值下的score(前面我设置的k值为5).

score_list = []
k_range = range(1,12)
for i in k_range:
    clf1 = KNeighborsClassifier(n_neighbors=i)
    clf1.fit(xtrain1,ytrain1)
    score_list.append(clf1.score(xtest1,ytest1))
score_list
plt.plot(k_range,score_list)

学习曲线如下：
再进行K折交叉验证：

from sklearn.model_selection import cross_val_score  as cvs

score1_list = []
score_var = []
k1_range = range(1,11)

for i in k1_range:
    clf2 = KNeighborsClassifier(n_neighbors=i)
    fold2 = cvs(clf2,data["data"],data["target"],cv=5)
    score_var.append(fold2.var())
    score1_list.append(fold2.mean())
    
plt.plot(k1_range,score1_list,color="orange")
#绘制2倍方差下的score曲线
plt.plot(k1_range,np.array(score1_list)+np.array(score_var)*2,color = "red",linestyle = "--")
plt.plot(k1_range,np.array(score1_list)-np.array(score_var)*2,color = "red",linestyle = "--")

学习曲线如下：

最终可以选取k值为5。

由于数据集的不同的特征值相差较大，对邻近距离影响较大，应当将数据集进行标准化处理。

from sklearn.preprocessing import *

X_01 = MinMaxScaler(feature_range=(0,1)).fit_transform(data1.iloc[:,:-1])
zdata1 = pd.DataFrame(X_01,columns=breast_cancer.feature_names)
zdata1.insert(len(zdata1.columns),"y",breast_cancer.target)

zX = zdata1.iloc[:,:-1]
zy = zdata1.y

加上权重后，重新绘制学习曲线如下: