机器学习11:用sns.pairplot()做特征工程

用一个糖尿病确诊预测案例,从头到尾了解一下机器学习的工作流程。

导入相关库,读取数据

# Import our libraries
import pandas as pd
import numpy as np
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split, RandomizedSearchCV
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier
import matplotlib.pyplot as plt
from sklearn.svm import SVC
import seaborn as sns
sns.set(style="ticks")

import check_file as ch

%matplotlib inline

# Read in our dataset
diabetes = pd.read_csv('diabetes.csv')

# Take a look at the first few rows of the dataset
diabetes.head()
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAgeOutcome
061487235033.60.627501
11856629026.60.351310
28183640023.30.672321
318966239428.10.167210
40137403516843.12.288331

我们将从加载数据和导入相关库开始。然后,你将了解如何使用网格搜索来优化多个模型。因为这门课程的目的是了解机器学习技术,所以我们在很大程度上省略了创建机器学习模型前的数据分析阶段的相关过程,包括探索性数据分析、特征工程、数据清理和数据整理等。

步骤 1: 让我们完成下面几个步骤。先看一下这个数据集的一些摘要统计数据,以便准确地将统计值与下面的字典中的适当键进行匹配。

# Cells for work
diabetes.describe()
PregnanciesGlucoseBloodPressureSkinThicknessInsulinBMIDiabetesPedigreeFunctionAgeOutcome
count768.000000768.000000768.000000768.000000768.000000768.000000768.000000768.000000768.000000
mean3.845052120.89453169.10546920.53645879.79947931.9925780.47187633.2408850.348958
std3.36957831.97261819.35580715.952218115.2440027.8841600.33132911.7602320.476951
min0.0000000.0000000.0000000.0000000.0000000.0000000.07800021.0000000.000000
25%1.00000099.00000062.0000000.0000000.00000027.3000000.24375024.0000000.000000
50%3.000000117.00000072.00000023.00000030.50000032.0000000.37250029.0000000.000000
75%6.000000140.25000080.00000032.000000127.25000036.6000000.62625041.0000001.000000
max17.000000199.000000122.00000099.000000846.00000067.1000002.42000081.0000001.000000

1.sns.pairplot

http://seaborn.pydata.org/generated/seaborn.pairplot.html

#可以看到对角线上是各个属性的直方图(分布图),而非对角线上是两个不同属性之间的相关图
sns.pairplot(diabetes)
plt.savefig("./pairplot000.png")

在这里插入图片描述

'''
hue :针对某一字段进行分类
sns.pairplot(diabetes, hue="Outcome");

kind:用于控制非对角线上的图的类型,可选"scatter"与"reg"
diag_kind:控制对角线上的图的类型,可选"hist"与"kde"
sns.pairplot(diabetes,kind="reg",diag_kind="kde")

palette:控制色调
sns.pairplot(data,hue="种类",palette="husl")

#markers:控制散点的样式 
sns.pairplot(data,hue="Outcome",markers=["+", "s", "D"])


#单独用vars参数选择"萼片长 "和"花瓣长"两种属性
sns.pairplot(data,vars=["Pregnancies","Glucose"])

# 用x_vars和 y_vars参数指定
# 需要注意的是,x_vars和y_vars要同时指定
sns.pairplot(data,x_vars=["Pregnancies","Glucose"],
             y_vars=["BloodPressure","SkinThickness"]) 

'''

sns.pairplot(diabetes, hue="Outcome");
plt.savefig("./pairplot01.png")

在这里插入图片描述

2.heatmap热度图

'''
#heatmap热度图,seaborn中常用的图

#重要点思维:拿到一批数据一般会求特征之间的相关系数,可以用pands直接求出来相关系数,放到heatmap,

可以很清楚的看到两个特征的相关程度,这是一个固定的数据思维
'''
sns.heatmap(diabetes.corr(), annot=True, cmap="YlGnBu");
plt.savefig("./heatmap.png")

在这里插入图片描述

3.hist()直方图

diabetes.hist()
plt.savefig("./hist.png")

在这里插入图片描述

# Possible keys for the dictionary
a = '0.65'
b = '0'
c = 'Age'
d = '0.35'
e = 'Glucose'
f = '0.5'
g = "More than zero"

# Fill in the dictionary with the correct values here
answers_one = {
    'The proportion of diabetes outcomes in the dataset':d,
    'The number of missing data points in the dataset': b,
    'A dataset with a symmetric distribution': e,
    'A dataset with a right-skewed distribution': c, 
    'This variable has the strongest correlation with the outcome': e
}

# Just to check your answer, don't change this
ch.check_one(answers_one)
Awesome! These all look great!

步骤 2:因为我们的数据集非常干净,我们将直接开始机器学习步骤。我们的目标是能够预测糖尿病病例。首先,需要确定y向量和X矩阵。然后,用下面的代码将数据集划分为训练数据和测试数据。

y = diabetes['Outcome']
X = diabetes[['Pregnancies','Glucose', 'BloodPressure', 'SkinThickness','Insulin', 'BMI', 'DiabetesPedigreeFunction', 'Age']]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

现在你已经有一个训练集和一个测试集,我们需要创建一些模型,并最终从中选择一个最好的模型。然而,与前面的课程中使用默认值不同,现在我们要对这些模型进行调优。

通过测试所有可能的超参数组合来找到最佳模型通常很困难(而且非常耗时)。因此,通常我们使用随机搜索。

在实践中,虽然优化的很好,但使用随机搜索来查找超参数会更耗时。这是与此主题相关的一篇文章。这是两篇介绍如何在sklearn中使用随机搜索的文档 12

为了有效地使用随机化搜索,你最好对超参数的常用分布有很好的理解。了解你的超参数有哪些可能值将使你能编写一个好的网格搜素算法(不中断的那种)。

步骤 3:在这一步中,我将向你展示如何使用随机搜索,然后你可以在步骤4中为其他模型设置网格搜索。但是因为我不记得 SVMs 的每一个超参数具体做什么,请你帮我将每个超参数与其相应的调优函数进行匹配。

用随机搜索算法

# build a classifier
clf_rf = RandomForestClassifier()

# Set up the hyperparameter search
param_dist = {"max_depth": [3, None],
              "n_estimators": list(range(10, 200)),
              "max_features": list(range(1, X_test.shape[1]+1)),
              "min_samples_split": list(range(2, 11)),
              "min_samples_leaf": list(range(1, 11)),
              "bootstrap": [True, False],
              "criterion": ["gini", "entropy"]}


# Run a randomized search over the hyperparameters
random_search = RandomizedSearchCV(clf_rf, param_distributions=param_dist)

# Fit the model on the training data
random_search.fit(X_train, y_train)

# Make predictions on the test data
rf_preds = random_search.best_estimator_.predict(X_test)

ch.print_metrics(y_test, rf_preds, 'random forest')
Accuracy score for random forest : 0.7597402597402597
Precision score random forest : 0.65
Recall score random forest : 0.7090909090909091
F1 score random forest : 0.6782608695652174

步骤4:现在你已经了解了如何使用随机森林模型来运行随机网格搜索,请尝试使用 AdaBoost 和 SVC 分类器执行此操作。你也可以尝试使用在本课程前面见过的其他分类器, 看看哪些分类器最有效。

# build a classifier for ada boost
clf_ada = AdaBoostClassifier()

# Set up the hyperparameter search
# look at  setting up your search for n_estimators, learning_rate
# http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html
param_dist = {"n_estimators": [10, 100, 200, 400],
              "learning_rate": [0.001, 0.005, .01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 1, 2, 10, 20]}

# Run a randomized search over the hyperparameters
ada_search = RandomizedSearchCV(clf_ada, param_distributions=param_dist)

# Fit the model on the training data
ada_search.fit(X_train, y_train)

# Make predictions on the test data
ada_preds = ada_search.best_estimator_.predict(X_test)

# Return your metrics on test data
ch.print_metrics(y_test, ada_preds, 'adaboost')
Accuracy score for adaboost : 0.7792207792207793
Precision score adaboost : 0.7441860465116279
Recall score adaboost : 0.5818181818181818
F1 score adaboost : 0.6530612244897959
# build a classifier for support vector machines
clf_svc = SVC()

# Set up the hyperparameter search
# look at setting up your search for C (recommend 0-10 range), 
# kernel, and degree
# http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
param_dist = {"C": [0.1, 0.5, 1, 3, 5],
              "kernel": ['linear','rbf']
             }


# Run a randomized search over the hyperparameters
svc_search = RandomizedSearchCV(clf_svc, param_distributions=param_dist)

# Fit the model on the training data
svc_search.fit(X_train, y_train)

# Make predictions on the test data
svc_preds = svc_search.best_estimator_.predict(X_test)


# Return your metrics on test data
ch.print_metrics(y_test, svc_preds, 'svc')
Accuracy score for svc : 0.7532467532467533
Precision score svc : 0.6545454545454545
Recall score svc : 0.6545454545454545
F1 score svc : 0.6545454545454545

步骤5:使用下面的测试查看一下,你选择的最佳模型是否和我们通过网格搜索找到的一样。

a = 'randomforest'
b = 'adaboost'
c = 'supportvector'

best_model =  b# put your best model here as a string or variable

# See if your best model was also mine.  
# Notice these might not match depending your search!
ch.check_best(best_model)
Nice!  It looks like your best model matches the best model I found as well!  It makes sense to use f1 score to determine best in this case given the imbalance of classes.  There might be justification for precision or recall being the best metric to use as well - precision showed to be best with adaboost again.  With recall, SVMs proved to be the best for our models.

4.特征重要性feature_importances_

一旦你找到了最佳模型,了解它为什么运行良好也很重要。在可以看到权重的回归模型中,可以更容易地解释结果。

步骤6:尽管你的模型很难解释,但是有一些方法可以让我们知道哪些特征是重要的。使用前一个问题中选择的“最佳模型”,来查找帮助判断个体是否患有糖尿病的最重要的特征。你的结论是否与你在本notebook的数据探索阶段的预期相匹配?

print(diabetes.shape)
(768, 9)
# Show your work here - the plot below was helpful for me
# https://stackoverflow.com/questions/44101458/random-forest-feature-importance-chart-using-python
features = diabetes.columns[:diabetes.shape[1]]
importances = random_search.best_estimator_.feature_importances_
indices = np.argsort(importances)

plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='b', align='center')
plt.yticks(range(len(indices)), features[indices])
plt.xlabel('Relative Importance');

plt.savefig("./importances.png")

在这里插入图片描述

For the most part these match what I would expect based on the correlations we saw earlier between each variable and Outcome. However, one interesting finding is that pregnancy looked to be correlated with the Outcome in the exploratory findings. Though, that is likely only because pregnancy is correlated with age, and age is truly the variable that is a better indicator of diabetes.

步骤 7:使用上面的结果来完成下面的字典。

# Check your solution by matching the correct values in the dictionary
# and running this cell
a = 'Age'
b = 'BloodPressure'
c = 'BMI'
d = 'DiabetesPedigreeFunction'
e = 'Insulin'
f = 'Glucose'
g = 'Pregnancy'
h = 'SkinThickness'



sol_seven = {
    'The variable that is most related to the outcome of diabetes' : f,
    'The second most related variable to the outcome of diabetes' : c,
    'The third most related variable to the outcome of diabetes' : a,
    'The fourth most related variable to the outcome of diabetes' : d
}

ch.check_q_seven(sol_seven)
That's right!  Some of these were expected, but some were a bit unexpected too!

步骤 8: 现在总结一下,在这个notebook中你做过什么,并且你怎样把这个结果向一个非技术背景的人解释。完成后,单击左上角的橙色图标可以查看解决方案的notebook。

In this case study, we looked at predicting diabetes for 768 patients. There was a reasonable amount of class imbalance with just under 35% of patients having diabetes. There were no missing data, and initial looks at the data showed it would be difficult to separate patients with diabetes from those that did not have diabetes.

Three advanced modeling techniques were used to predict whether or not a patient has diabetes. The most successful of these techniques proved to be an AdaBoost Classification technique, which had the following metrics:

Accuracy score for adaboost : 0.7792207792207793

Precision score adaboost : 0.7560975609756098

Recall score adaboost : 0.5636363636363636

F1 score adaboost : 0.6458333333333333

Based on the initial look at the data, it is unsurprising that Glucose, BMI, and Age were important in understanding if a patient has diabetes. These were consistent with more sophisticated approaches. Interesting findings were that pregnancy looked to be correlated when initially looking at the data. However, this was likely due to its large correlation with age.

  • 25
    点赞
  • 170
    收藏
  • 打赏
    打赏
  • 7
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
©️2022 CSDN 皮肤主题:编程工作室 设计师:CSDN官方博客 返回首页
评论 7

打赏作者

科学边界

你的鼓励将是我创作的最大动力

¥2 ¥4 ¥6 ¥10 ¥20
输入1-500的整数
余额支付 (余额:-- )
扫码支付
扫码支付:¥2
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值