KNN分类模型
- 概念:简单地说,K-近邻算法采用测量不同特征值之间的距离方法进行分类(k-Nearest Neighbor ,KNN)
- k值的作用
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/d6a124ae2999219449941e25fa9f28ee.png)
- 欧几里得距离
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/c62a385df1e3221373485ad37d373fdf.png)
- 在scikit-learn库中使用k-近邻算法
import sklearn.datasets as ds
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
if __name__ == '__main__':
iris = ds.load_iris()
feature = iris['data']
target = iris['target']
x_train,x_test,y_train,y_test = train_test_split(feature,target,test_size=0.2,random_state=2021)
x_train.shape
knn = KNeighborsClassifier(n_neighbors=3)
knnModel = knn.fit(x_train,y_train)
print(knnModel)
y_pred= knnModel.predict(x_test)
y_true = y_test
print('模型的分类结果:',y_pred)
print('真是的分类结果:',y_true)
score = knnModel.score(x_test,y_test)
print('score=',score)
targetResult = knnModel.predict([[6.1,5.1,4.5,3.6],[2.1,3.1,4.5,5.6]])
print('预测结果:',targetResult)
import numpy as np
scores = []
ks = []
for i in range(1,50):
knn = KNeighborsClassifier(n_neighbors=i)
knn.fit(x_train,y_train)
score = knn.score(x_test,y_test)
ks.append(i)
scores.append(score)
scores_arr = np.array(scores)
ks_arr = np.array(ks)
import matplotlib.pyplot as plt
plt.plot(ks_arr,scores_arr)
plt.xlabel('k_value')
plt.ylabel('score')
plt.show()
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/02ee72dd63f9dcdd574566ef1047c6d9.png)
K折交叉验证
- 目的:
- 选出最为适合的模型超参数的取值,然后将超参数的值作用到模型的创建中。
- 思想:
- 将样本的训练数据交叉的拆分出不同的训练集和验证集,使用交叉拆分出不同的训练集和验证集分布测试模型的精准度,然后求出的精准度的均值就是此处交叉验证的结果。将交叉验证作到不同的超参数中,选取出精准度最高的超参数作为模型的超参数即可。
- 实现思路
- 将数据集评价分割成K个等份
- 使用1份数据作为测试数据,其余作为训练数据
- 计算测试准确率
- 使用不同的测试集,重复2、3步骤
- 对准确率做平均,作为对未知数据预测准确率的估计
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/568846f7956ec21c1cb61fd16a24fa3c.png)
- API
- from sklearn.model_selection import cross_val_score
- cross_val_score(estimator,X,y,cv)
- estimator:模型对象
- X,y;训练集数据
- cv:折数
- 交叉验证在KNN中的基本使用
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
from sklearn.model_selection import cross_val_score
import sklearn.datasets as ds
if __name__ == '__main__':
iris = ds.load_iris()
feature = iris['data']
target = iris['target']
x_train,x_test,y_train,y_test = train_test_split(feature,target,test_size=0.2,random_state=2021)
x_train.shape
knn = KNeighborsClassifier(n_neighbors=5)
crossScore= cross_val_score(knn,x_train,y_train,cv=5).mean()
print(crossScore)
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
from sklearn.model_selection import cross_val_score
import sklearn.datasets as ds
import matplotlib.pyplot as plt
import numpy as np
if __name__ == '__main__':
iris = ds.load_iris()
feature = iris['data']
target = iris['target']
x_train, x_test, y_train, y_test = train_test_split(feature, target, test_size=0.2, random_state=2020)
scores = []
ks = []
for k in range(3,20):
knn = KNeighborsClassifier(n_neighbors=k)
cross_score = cross_val_score(knn,x_train,y_train,cv=6).mean()
scores.append(cross_score)
ks.append(k)
ks_arr = np.array(ks)
scores_arr = np.array(scores)
plt.plot(ks_arr, scores_arr)
plt.xlabel('k_value')
plt.ylabel('score')
plt.show()
max_idx = scores_arr.argmax()
max_k = ks[max_idx]
print('最大值下标:',max_idx)
print('最大值对应的k值:',max_k)
![在这里插入图片描述](https://i-blog.csdnimg.cn/blog_migrate/5d2d4e31f96825f852e89c1152b8ba79.png)
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
from sklearn.model_selection import cross_val_score
import sklearn.datasets as ds
if __name__ == '__main__':
iris = ds.load_iris()
feature = iris['data']
target = iris['target']
x_train, x_test, y_train, y_test = train_test_split(feature, target, test_size=0.2, random_state=2020)
from sklearn.linear_model import LogisticRegression
knn = KNeighborsClassifier(n_neighbors=7)
print('KNN模型准确率',cross_val_score(knn,x_train,y_train,cv=10).mean())
lr = LogisticRegression()
print('LR模型准确率',cross_val_score(lr,x_train,y_train,cv=10).mean())
K-Fold(作为了解)
- Scikit中提供了K-Fold的API
- n-split就是折数
- shuffle指是否对数据洗牌
- random_state为随机种子,固定随机性
from numpy import array
from sklearn.model_selection import KFold
if __name__ == '__main__':
data = array([0.1,0.2,0.3,0.4,0.5,0.6])
kfold = KFold(n_splits=3,shuffle=True,random_state=1)
for train,test in kfold.split(data):
print('train: %s,test:%s' % (data[train],data[test]))
- Scikit中交叉验证接口sklearn.model_selection.cross_val_score,但是该接口没有数据shuffle功能,所以一般结合Kfold一起使用。如果Train数据在分组前已经经过了shuffle处理,比如使用train_test_split分组,那就可以直接使用cross_val_score接口
from sklearn.model_selection import cross_val_score
import sklearn.datasets as ds
from sklearn.neighbors import KNeighborsClassifier
iris = ds.load_iris()
X,y = iris.data,iris.target
knn = KNeighborsClassifier(n_neighbors=5)
n_folds = 5
kf = KFold(n_folds,shuffle=True,random_state=42).get_n_splits(X)
scores = cross_val_score(knn,X,y,cv=kf)
print(scores.mean())