kNN算法中的超参数的问题

1、超参数：是指在运行机器学习算法之前需要指定的参数。

2、如何寻找到最好的超参数？

               可以使用循环搜索的方法来选择出最好的超参数。

      k近邻（kNN）的超参数一个是k值的选择另一个是距离的权重，我们将距离的倒数作为距离的权重，在sklearn中k近邻算法有一个weights参数，其默认值为uniform，此时是不考虑距离权重的，当weights=distance时，sklearn就会考虑距离的权重

加载数据，训练数学模型代码如下：

"""coding:utf-8"""
from sklearn import datasets
digits = datasets.load_digits()
X = digits.data
y = digits.target
print(X.shape)
print(y.shape)
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=666)
from sklearn.neighbors import KNeighborsClassifier
knn_clf = KNeighborsClassifier(n_neighbors=3)
knn_clf.fit(X_train,y_train)
#predict = knn_clf.predict(X_test)
accuracy = knn_clf.score(X_test,y_test)
print(accuracy)

 寻找最好的K值

best_score = 0
best_k = -1
for k in range(1,11):
    knn_clf = KNeighborsClassifier(n_neighbors=k)
    knn_clf.fit(X_train,y_train)
    score = knn_clf.score(X_test,y_test)
    if score > best_score:
        best_score = score
        best_k = k
print(best_k)
print(best_score)

考虑距离权重参数

best_k = -1
best_score = 0.0
best_method = ""
for method in ["uniform","distance"]:
    for k in range(1,11):
        knn_clf = KNeighborsClassifier(n_neighbors=k,weights=method)
        knn_clf.fit(X_train,y_train)
        knn_clf.score(X_test,y_test)
        if score > best_score:
            best_score = score
            best_k = k
            best_method = method

print(best_score)
print(best_k)
print(best_method)

搜索明科夫斯基距离相应的P

best_score = 0.0
best_k = -1
best_p = -1

for k in range(1, 11):
    for p in range(1, 6):
        knn_clf = KNeighborsClassifier(n_neighbors=k, weights="distance", p=p)
        knn_clf.fit(X_train, y_train)
        score = knn_clf.score(X_test, y_test)
        if score > best_score:
            best_k = k
            best_p = p
            best_score = score
        
print("best_k =", best_k)
print("best_p =", best_p)
print("best_score =", best_score)