简单学习及使用scikit-learn库中的neighbors.KNeighborsClassifier
数据集采用UCI中的Breast Cancer Wisconsin (Diagnostic).
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
data = load_breast_cancer()
data["data"] #X
data["target"]#y
clf = KNeighborsClassifier(n_neighbors=5) #实例化,k = 5
#划分训练集与测试集
xtrain,xtest,ytrain,ytest = train_test_split(data["data"],data["target"],test_size=0.3)
#拟合数据
clf.fit(xtrain,ytrain)
[clf.score(xtrain,ytrain),clf.score(xtest,ytest)]
可以得到,
score = [0.949748743718593, 0.9239766081871345]
描绘一条学习曲线,观测不同k值下的score(前面我设置的k值为5).
score_list = []
k_range = range(1,12)
for i in k_range:
clf1 = KNeighborsClassifier(n_neighbors=i)
clf1.fit(xtrain1,ytrain1)
score_list.append(clf1.score(xtest1,ytest1))
score_list
plt.plot(k_range,score_list)
学习曲线如下:
再进行K折交叉验证:
from sklearn.model_selection import cross_val_score as cvs
score1_list = []
score_var = []
k1_range = range(1,11)
for i in k1_range:
clf2 = KNeighborsClassifier(n_neighbors=i)
fold2 = cvs(clf2,data["data"],data["target"],cv=5)
score_var.append(fold2.var())
score1_list.append(fold2.mean())
plt.plot(k1_range,score1_list,color="orange")
#绘制2倍方差下的score曲线
plt.plot(k1_range,np.array(score1_list)+np.array(score_var)*2,color = "red",linestyle = "--")
plt.plot(k1_range,np.array(score1_list)-np.array(score_var)*2,color = "red",linestyle = "--")
学习曲线如下:
最终可以选取k值为5。
由于数据集的不同的特征值相差较大,对邻近距离影响较大,应当将数据集进行标准化处理。
from sklearn.preprocessing import *
X_01 = MinMaxScaler(feature_range=(0,1)).fit_transform(data1.iloc[:,:-1])
zdata1 = pd.DataFrame(X_01,columns=breast_cancer.feature_names)
zdata1.insert(len(zdata1.columns),"y",breast_cancer.target)
zX = zdata1.iloc[:,:-1]
zy = zdata1.y
加上权重后,重新绘制学习曲线如下: