Machine Learning -- algorithms performance measure (heart disease data)

最新推荐文章于 2022-12-04 21:00:42 发布
最新推荐文章于 2022-12-04 21:00:42 发布
阅读量259 收藏 1
点赞数
分类专栏： Machine Learning
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/Opium_Z/article/details/89490946
版权

OverView:

In this chapter, I’m gonna do performance stuffs for machine learning algorithms. From previous chapter, I have already built four model:
SVM-RBF, SVM-PLOY, Bagging, AdaBoost.

Step 1: Confusion Matrix

SVM-RBF (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

SVM-POLY (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

Bagging (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

Adaboost (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

Step 2: Metrics

Based on the confusion matrix, I will calculate:

  1. F-measure score
  2. accuracy
  3. recall ( sensitivity, true positive rate )
  4. precision ( positive predictive value )
  5. AUC ( Area under the ROC curve )

–> This is a screen shot for all the metrics of each algorithm.

在这里插入图片描述

–> Put them into excel

在这里插入图片描述

–> Rank them to have an intuitive unstanding

在这里插入图片描述

Step 3: Plot ROC Curve

在这里插入图片描述



import pandas as pd
import numpy as np
from sklearn.preprocessing import OneHotEncoder
from sklearn import tree
from sklearn.ensemble import BaggingClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn import svm
from sklearn.model_selection import StratifiedKFold, GridSearchCV
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.model_selection import cross_val_score
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix, f1_score, accuracy_score, recall_score, precision_score, roc_auc_score, \
    roc_curve





## load data
from plot_confusion_matrix import plot_confusion_matrix

trainSet = pd.read_csv("clevelandtrain.csv")
testSet = pd.read_csv("clevelandtest.csv")

xtrain = (trainSet.drop(["heartdisease::category|0|1"], axis=1)).iloc[:,:].values  # (152, 13)
ytrain = trainSet["heartdisease::category|0|1"].iloc[:].values                     # (152,)

xtest = (testSet.drop(["heartdisease::category|0|1"], axis=1)).iloc[:,:].values    # (145, 13)
ytest = testSet["heartdisease::category|0|1"].iloc[:].values                       # (145,)



## data preprocessing

 # without: one-hot-encoder:

# xTrain = xtrain
# yTrain = ytrain
# xTest = xtest
# yTest = ytest

  # one-hot-encoder: #9 (cp), #19 (restecg),  #41 (slope), #51 (thal)

xtrain_pre = trainSet.drop(["cp", "restecg", "slope", "thal", "heartdisease::category|0|1"], axis=1).iloc[:,:].values # (152, 9)
xtrain_cp = trainSet["cp"].iloc[:].values
xtrain_restecg = trainSet["restecg"].iloc[:].values
xtrain_slope = trainSet["slope"].iloc[:].values
xtrain_thal = trainSet["thal"].iloc[:].values

ohe1 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')
ohe2 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')
ohe3 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')
ohe4 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')

xtrain_cp = ohe1.fit_transform(xtrain_cp.reshape(-1,1))                    # (152, 4)
xtrain_restecg = ohe2.fit_transform(xtrain_restecg.reshape(-1,1))          # (152, 3)
xtrain_slope = ohe3.fit_transform(xtrain_slope.reshape(-1,1))              # (152, 3)
xtrain_thal = ohe4.fit_transform(xtrain_thal.reshape(-1,1))                # (152, 3)


xTrain = np.hstack((xtrain_pre, xtrain_cp, xtrain_restecg, xtrain_slope, xtrain_thal))   # (152, 22)
yTrain = ytrain                                                                          # (152,)



xtest_pre = testSet.drop(["cp", "restecg", "slope", "thal", "heartdisease::category|0|1"], axis=1).iloc[:,:].values   # (145, 9)
xtest_cp = testSet["cp"].iloc[:].values
xtest_restecg = testSet["restecg"].iloc[:].values
xtest_slope = testSet["slope"].iloc[:].values
xtest_thal = testSet["thal"].iloc[:].values

xtest_cp = ohe1.transform(xtest_cp.reshape(-1,1))                 # (145, 4)
xtest_restecg = ohe2.transform(xtest_restecg.reshape(-1,1))       # (145, 3)
xtest_slope = ohe3.transform(xtest_slope.reshape(-1,1))           # (145, 3)
xtest_thal = ohe4.transform(xtest_thal.reshape(-1,1))             # (145, 3)

xTest = np.hstack((xtest_pre, xtest_cp, xtest_restecg, xtest_slope, xtest_thal))   # (145, 22)
yTest = ytest                                                                      # (145,)


print("-----------------------------------------------------------------")


class_names = np.array([1,0])


svcRBF = SVC(C=300.0,gamma=0.0001,kernel='rbf',probability=True)
svcRBF.fit(xTrain,yTrain)
svcRBFScore = svcRBF.score(xTest, yTest)
prediction_svcRBF = svcRBF.predict(xTest)



# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcRBF, classes=class_names, title='Confusion matrix svcRBF, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcRBF, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix svcRBF')
#
# plt.show()
#
#
# f1 = f1_score(yTest, prediction_svcRBF)
# acc1 = accuracy_score(yTest, prediction_svcRBF)
# rec1 = recall_score(yTest, prediction_svcRBF)
# pre1 = precision_score(yTest, prediction_svcRBF)
# auc1 = roc_auc_score(yTest, prediction_svcRBF)
# print(f1)
# print(acc1)
# print(rec1)
# print(pre1)
# print(auc1)
# print("-----------------------------------------------------------------")




svcPoly = SVC(C=1.0,degree = 8.666666,coef0=1.0,gamma = 'scale',max_iter=-1,kernel='poly',probability=True)
svcPoly.fit(xTrain,yTrain)
svcPolyScore = svcPoly.score(xTest, yTest)
prediction_svcPLOY = svcPoly.predict(xTest)

# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcPLOY, classes=class_names, title='Confusion matrix svcPLOY, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcPLOY, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix svcPLOY')
#
# plt.show()
#
# f2 = f1_score(yTest, prediction_svcPLOY)
# acc2 = accuracy_score(yTest, prediction_svcPLOY)
# rec2 = recall_score(yTest, prediction_svcPLOY)
# pre2 = precision_score(yTest, prediction_svcPLOY)
# auc2 = roc_auc_score(yTest, prediction_svcPLOY)
# print(f2)
# print(acc2)
# print(rec2)
# print(pre2)
# print(auc2)
# print("-----------------------------------------------------------------")



decisonTree = tree.DecisionTreeClassifier()
decisonTreeBagging = BaggingClassifier(decisonTree,max_samples=0.7, max_features=1.0)
decisonTreeBagging.fit(xTrain,yTrain)
Bagging_score = decisonTreeBagging.score(xTest,yTest)

prediction_Bagging = decisonTreeBagging.predict(xTest)



# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_Bagging, classes=class_names, title='Confusion matrix decisonTreeBagging, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_Bagging, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix decisonTreeBagging')
#
# plt.show()
#
# f3 = f1_score(yTest, prediction_Bagging)
# acc3 = accuracy_score(yTest, prediction_Bagging)
# rec3 = recall_score(yTest, prediction_Bagging)
# pre3 = precision_score(yTest, prediction_Bagging)
# auc3 = roc_auc_score(yTest, prediction_Bagging)
# print(f3)
# print(acc3)
# print(rec3)
# print(pre3)
# print(auc3)
# print("-----------------------------------------------------------------")



decisonTreeAda = AdaBoostClassifier(decisonTree,n_estimators=10,random_state=np.random.RandomState(1))
decisonTreeAda.fit(xTrain,yTrain)
AdaBoost_score = decisonTreeAda.score(xTest,yTest)

prediction_AdaBoost = decisonTreeAda.predict(xTest)


# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_AdaBoost, classes=class_names, title='Confusion matrix AdaBoost, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_AdaBoost, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix AdaBoost')
#
# plt.show()
#
#
# f4 = f1_score(yTest, prediction_AdaBoost)
# acc4 = accuracy_score(yTest, prediction_AdaBoost)
# rec4 = recall_score(yTest, prediction_AdaBoost)
# pre4 = precision_score(yTest, prediction_AdaBoost)
# auc4 = roc_auc_score(yTest, prediction_AdaBoost)
# print(f4)
# print(acc4)
# print(rec4)
# print(pre4)
# print(auc4)
# print("-----------------------------------------------------------------")





## plot roc curve

prediction_prob_svmRBF = svcRBF.predict_proba(xTest)[:, 1]
prediction_prob_svmPOLY = svcPoly.predict_proba(xTest)[:, 1]
prediction_prob_Bagging = decisonTreeBagging.predict_proba(xTest)[:, 1]
prediction_prob_Adaboost = decisonTreeAda.predict_proba(xTest)[:, 1]

fpr_svmRBF, tpr_svmRBF, _ = roc_curve(yTest, prediction_prob_svmRBF)
fpr_svmPOLY, tpr_svmPOLY, _ = roc_curve(yTest, prediction_prob_svmPOLY)
fpr_Bagging, tpr_Bagging, _ = roc_curve(yTest, prediction_prob_Bagging)
fpr_Adaboost, tpr_Adaboost, _ = roc_curve(yTest, prediction_prob_Adaboost)

plt.figure(1)
plt.plot([0, 1], [0, 1], 'k--')

plt.plot(fpr_svmRBF, tpr_svmRBF, label='SVM-RBF')
plt.plot(fpr_svmPOLY, tpr_svmPOLY, label='SVM-POLY')
plt.plot(fpr_Bagging, tpr_Bagging, label='Bagging')
plt.plot(fpr_Adaboost, tpr_Adaboost, label='Adaboost')

plt.xlabel('False positive rate')
plt.ylabel('True positive rate')
plt.title('ROC curve')
plt.legend(loc='best')
plt.show()
Opium_Z
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Machine-Learning-Algorithms-master 配套数据集及源代码
01-02
Machine_Learning_Algorithms-master，Machine_Learning_Algorithms-master 配套数据集及源代码 Machine_Learning_Algorithms-master，Machine_Learning_Algorithms-master 配套数据集及源代码
【Machine Learning系列】带你快速学习十大机器学习算法
最新发布
陈橘又青的博客
07-03 9926
机器学习算法的学习是一个长期的过程，需要不断地练习和实践才能够掌握。通过坚持不懈地学习和实践，你将能够快速掌握机器学习算法并应用于实际问题中。
吴恩达机器学习笔记一（Machine learning algorithms）
weixin_43294620的博客
12-29 722
Machine learning algorithms （纯粹促进自己学习整理） 1. Machine learning algorithms: Main two types: Supervised learning 监督学习 In supervised learning, the idea is that we’re going to teach the computer how to do something. Unsupervised learning 无监督学习 In unsupervised
Machine Learning：十大机器学习算法
茅坤宝骏氹的博客
04-17 5400
转载自 Machine Learning：十大机器学习算法摘要: - 机器学习算法分类：监督学习、无监督学习、强化学习 - 基本的机器学习算法：线性回归、支持向量机(SVM)、最近邻居(KNN)、逻辑回归、决策树、k平均、随机森林、朴素贝叶斯、降维、梯度增强机器学习算法大致可以分为三类：监督学习算法 (Supervised Algorithms）:在监督学习训练过程中，可以由训练数据集学到或建立一...
Machine Learning Algorithms
beijingbuaaer的博客
07-22 347
I mainly use the python module scikit-learn, a.k.a. sklearn to learn the basic machine learning algorithms. So, this blog is the content of my continuing blogs tends to Machine-Learning Algorithm
吴恩达机器学习笔记二（Machine learning algorithms--Linear regression）
weixin_43294620的博客
01-06 789
Linear regression For example: We’re going to use a data set of housing prices from the city of Portland, Oregon. And I’m gonna plot my data set of a number of houses that were different sizes that were sold for a range of different prices. Based on the d
machine-learning-algorithms-and-data-structures:从头开始构建的机器学习算法和数据结构的集合
04-01
机器学习算法和数据结构 机器学习算法 神经网络 分类 回归 数据结构 [列表]（（ ）
understanding-machine-learning-theory-algorithms
10-31
understanding-machine-learning-theory-algorithms
Machine-Learning-Algorithms
04-14
机器学习算法 嗯，此仓库中的所有算法都是我自己尝试制作并成功的算法。 我知道它们可能并不完美，也可能没有效率，但是，嘿，这是我的第一次尝试。
机器学习模型的性能指标
weixin_26752765的博客
09-04 2585
There are various metrics that we can use to evaluate the performance of ML algorithms, classification as well as regression algorithms. We must carefully choose the metrics for evaluating ML performa...
周志华《Machine Learning》学习笔记（2）--性能度量
VAY-长跑
11-20 7592
本篇主要是对第二章剩余知识的理解，包括：性能度量、比较检验和偏差与方差，写到后面发现篇幅冗长读起来十分没有快感，故本篇主要知识点为性能度量。
机器学习分类问题：9个常用的评估指标总结
悟道西方
12-04 195
你好，我是zhenguo对机器学习的评估度量是机器学习核心部分，本文总结分类问题常用的metrics分类问题评估指标在这里，将讨论可用于评估分类问题预测的各种性能指标1 Confusion Matrix这是衡量分类问题性能的最简单方法，其中输出可以是两种或更多类型的类。混淆矩阵只不过是一个具有两个维度的表，即“实际”和“预测”，此外，这两个维度都有“真阳性（TP）”、“真阴性（TN）”、“假阳性（...
Machine Learning Algorithms: Which One to Choose for Your Problem
xiangz_csdn的博客
10-28 811
Intuition of using different kinds of algorithms in different tasks
数据挖掘常用的心脏病数据(From UCI)
wonder的地下室
06-05 9274
http://archive.ics.uci.edu/ml/machine-learning-databases/statlog/heart/ 该数据经常作为数据挖掘的示例。 This database contains 13 attributes (which have been extracted froma larger set of 75)         Attribut
机器学习（Machine Learning）和传统的数据统计分析（Data Statistics）有什么区别？
梅森上校的博客 业精于勤荒于嬉，形成于思毁于随。
05-03 1999
机器学习（Machine Learning）和传统的数据统计（Data Statistics）有什么区别？ 机器学习 强调预测 通过预测性能评估结果 关注过度拟合而不是模型本身的复杂性 注重表现 通过在新数据集上的性能获得可概括性 通常，没有指定超人口模型 对性能和健壮性的关注 传统统计分析 强调超总体推断 关注a-先验假设 更简单的模型优于复杂的模型（简约），即使更复杂的模...
[转]机器学习相关数据库 备份
aodiyu3146的博客
10-11 364
原文地址：http://www.cppblog.com/cdy20/archive/2012/10/10/193134.html 资料库 KDD杯的中心，所有的数据，任务和结果。 UCI机器学习和知识发现研究中使用的大型数据集KDD数据库存储库。 UCI机器学习数据库。 AWS（亚马逊网络服务）公共数据集，提供了一个集中的资料库，可以无缝集成到基于AWS的云应用程序的公...
周志华《Machine Learning》学习笔记 2 ——性能度量
zhuangV587的博客
06-11 475
       本篇主要是对第二章剩余知识的理解，包括：性能度量、比较检验 和 偏差与方差 。在上一篇中，我们解决了评估学习器泛化性能的方法，即用测试集的“测试误差”作为“泛化误差”的近似，当我们划分好 训练/测试集后，那如何计算“测试误差”呢？这就是性能度量，例如：均方差，错误率等，即“测试误差”的一个评价标准。有了评估方法和性能度量，就可以计算出学习器的“测试误差”，但由于“测试误差”受到很多因...
Pattern Recognition and Machine Learning-01-Preface
06-08
He also discusses the challenges and opportunities in this field, such as dealing with high-dimensional data and designing effective algorithms. Bishop explains that the book is intended to be a ...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值