模型评估算法

学习四种不同的分离数据集的方法，用来分离训练数据集和评估数据集。

• 分离训练数据集和评估数据集
• K折交叉验证分离
• 弃一交叉验证分离
• 重复随机评估、训练数据集分离

1.分离训练数据集和评估数据集

通常将67%的数据作为训练集，33%的数据作为评估数据集，如下：

from pandas import read_csv
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
# 导入数据
filename='pima_data.csv'
names=['preg','plas','pres','skin','test','mass','pedi','age','class']
dataset=read_csv(filename,names=names)
# 将数据划分成输入数据和输出数据
array=dataset.values
X=array[:, 0:8]
Y=array[:, 8]
test_size=0.33
seed=4
# 按需划分训练集和评估集
X_train,X_test,Y_train,Y_test=train_test_split(X,Y,test_size=test_size,random_state=seed)
model=LogisticRegression()
model.fit(X_train,Y_train)
result=model.score(X_test,Y_test)
print("算法评估结果：%.3lf%%" %(result*100))

2.K折交叉验证分离

K折交叉验证是将原始数据分成K组（一般是均分），将每个子集数据分别做一次验证集，其余K-1组子集数据作为训练集，这样会得到K个模型，再用这K个模型的准确率的平均值。K一般从3开始取，如3，5，10

from pandas import read_csv
from sklearn.model_selection import KFold
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# 导入数据
filename = 'pima_data.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataset = read_csv(filename, names=names)
# 将数据划分成输入数据和输出数据
array = dataset.values
X = array[:, 0:8]
Y = array[:, 8]
num_folds = 10
seed = 7
kfold = KFold(n_splits=num_folds, random_state=seed)
model = LogisticRegression()
result = cross_val_score(model, X, Y, cv=kfold)
print("算法评估结果为：%.3lf%% (%.3lf%%)" % (result.mean() * 100, result.std() * 100))

3.弃一交叉验证分离

即每个样本单独作为验证集，其余的N-1个样本作为训练集，所以会得到N个模型。优点：结果可靠；实验可复制。

from pandas import read_csv
from sklearn.model_selection import LeaveOneOut
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# 导入数据
filename = 'pima_data.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataset = read_csv(filename, names=names)
# 将数据划分成输入数据和输出数据
array = dataset.values
X = array[:, 0:8]
Y = array[:, 8]
loocv=LeaveOneOut()
model = LogisticRegression()
result = cross_val_score(model, X, Y, cv=loocv)
print("算法评估结果为：%.3lf%% (%.3lf%%)" % (result.mean() * 100, result.std() * 100))

4.重复随机分离评估数据集与训练集

即按照67%：33%的比例分离，评估，重复10次。

from pandas import read_csv
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import ShuffleSplit

# 导入数据
filename = 'pima_data.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataset = read_csv(filename, names=names)
# 将数据划分成输入数据和输出数据
array = dataset.values
X = array[:, 0:8]
Y = array[:, 8]
n_splits=10
test_size=0.33
seed=7
ss=ShuffleSplit(n_splits=n_splits,test_size=test_size,random_state=seed)
model=LogisticRegression()
result = cross_val_score(model, X, Y, cv=ss)
print("算法评估结果为：%.3lf%% (%.3lf%%)" % (result.mean() * 100, result.std() * 100))