文章目录
前言
入门机器学习，第一次在CSDN上写学习笔记，如有错误请多指正。
参考书目:机器学习及Python应用
参考了赛博机器喵的【学习笔记】 陈强-机器学习-Python-Ch5 逻辑回归
一、数据处理
1.导入模块和数据文件
案例采用了titanic数据，该数据包括了泰坦尼克号乘客的存活数据。
import pandas as pd
import numpy as np
titanic=pd.read_csv('D:\数据集\\MLPython_Data\\titanic.csv')
titanic数据框如下:
Class Sex Age Survived Freq
0 1st Male Child No 0
1 2nd Male Child No 0
2 3rd Male Child No 35
3 Crew Male Child No 0
4 1st Female Child No 0
5 2nd Female Child No 0
6 3rd Female Child No 17
7 Crew Female Child No 0
8 1st Male Adult No 118
9 2nd Male Adult No 154
10 3rd Male Adult No 387
11 Crew Male Adult No 670
12 1st Female Adult No 4
13 2nd Female Adult No 13
14 3rd Female Adult No 89
15 Crew Female Adult No 3
16 1st Male Child Yes 5
17 2nd Male Child Yes 11
18 3rd Male Child Yes 13
19 Crew Male Child Yes 0
20 1st Female Child Yes 1
21 2nd Female Child Yes 13
22 3rd Female Child Yes 14
23 Crew Female Child Yes 0
24 1st Male Adult Yes 57
25 2nd Male Adult Yes 14
26 3rd Male Adult Yes 75
27 Crew Male Adult Yes 192
28 1st Female Adult Yes 140
29 2nd Female Adult Yes 80
30 3rd Female Adult Yes 76
31 Crew Female Adult Yes 20
2.展开数据框
将数据框完全展开，根据变量Freq让不同观测值在数据框中以相应频次出现。
freq=titanic.Freq.to_numpy()#将Freq变为数组
index=np.repeat(np.arange(32),freq)#将np.arange(32)中元素按freq频率重复
titanic=titanic.iloc[index,:]#利用数据框索引，输出新样本
titanic=titanic.drop('Freq',axis=1)#在新样本中去除Freq；axis=1时，数组变化是横向的，体现列增加和减少
新样本如下所示：
Class Sex Age Survived
2 3rd Male Child No
2 3rd Male Child No
2 3rd Male Child No
2 3rd Male Child No
2 3rd Male Child No
.. ... ... ... ...
31 Crew Female Adult Yes
31 Crew Female Adult Yes
31 Crew Female Adult Yes
31 Crew Female Adult Yes
31 Crew Female Adult Yes
3.数据概况
print(pd.crosstab(titanic.Sex,titanic.Survived))#考察不同性别存亡人数
print(pd.crosstab(titanic.Sex,titanic.Survived,normalize='index'))
#考察不同性别存活率，normalize='index'表示行标准化
print(pd.crosstab(titanic.Age,titanic.Survived,normalize='index'))#不同年龄存活率
print(pd.crosstab(titanic.Class,titanic.Survived,normalize='index'))#不同仓位存活率
结果如下：
Survived No Yes
Sex
Female 126 344
Male 1364 367
Survived No Yes
Sex
Female 0.268085 0.731915
Male 0.787984 0.212016
Survived No Yes
Age
Adult 0.687380 0.312620
Child 0.477064 0.522936
Survived No Yes
Class
1st 0.375385 0.624615
2nd 0.585965 0.414035
3rd 0.747875 0.252125
Crew 0.760452 0.239548
4.对数据进行抽样、处理
from sklearn.model_selection import train_test_split
import statsmodels.api as sm
from patsy import dmatrices
#导入模块
train,test=train_test_split(titanic,test_size=0.3,stratify=titanic.Survived,random_state=0)
#抽取训练集和测试集样本
#test_size=0.3表示测试集占30%
#stratify=titanic.Survived表示根据titanic.Survived进行分层抽样，确保每个类别在训练集和测试集比重相同
y_train,X_train=dmatrices('Survived~Class+Sex+Age',data=train,return_type='dataframe')
#dmatrices()函数使字符型分类变量转化为数值型分类变量
#return_type='dataframe'表示返回数据库，默认返回矩阵
#此命令表示根据训练集train，通过公式'Survived~Class+Sex+Age'，生成响应变量y_train和数据X_train
考察y_train, X_train前5个观测值
pd.options.display.max_columns=10#最大显示列数为10
print(X_train.head())
print(y_train.head())
输出结果如下：
Intercept Class[T.2nd] Class[T.3rd] Class[T.Crew] Sex[T.Male] \
11 1.0 0.0 0.0 1.0 1.0
27 1.0 0.0 0.0 1.0 1.0
22 1.0 0.0 1.0 0.0 0.0
11 1.0 0.0 0.0 1.0 1.0
27 1.0 0.0 0.0 1.0 1.0
Age[T.Child]
11 0.0
27 0.0
22 1.0
11 0.0
27 0.0
#以上为X_train观测值
Survived[No] Survived[Yes]
11 1.0 0.0
27 0.0 1.0
22 0.0 1.0
11 1.0 0.0
27 0.0 1.0
#以上为y_train观测值
由于y_train包含两个虚拟变量，仅保留Survived[Yes]
y_train=y_train.iloc[:,1]
对测试集也进行类似操作
y_test,X_test=dmatrices('Survived~Class+Sex+Age',data=test,return_type='dataframe')
y_test=y_test.iloc[:,1]
二.逻辑回归
1.调用statsmodels模块的Logit类进行逻辑回归
model=sm.Logit(y_train,X_train)#创建Logit实例
results=model.fit()
结果如下：
Optimization terminated successfully.
Current function value: 0.501935#损失函数值
Iterations 6#迭代6次，算法收敛
2.考察回归系数估计值
print(results.params)
Intercept 2.023452
Class[T.2nd] -1.045877
Class[T.3rd] -1.869105
Class[T.Crew] -0.882650
Sex[T.Male] -2.355785
Age[T.Child] 1.126943
dtype: float64
由于变量为离散变量，可用几率比对系数进行解释
print(np.exp(results.params))#将回归系数转化为几率比
Intercept 7.564391
Class[T.2nd] 0.351384#当乘客仓位由一等舱变为二等舱时，其存活新几率是原几率的0.35倍
Class[T.3rd] 0.154262#当乘客仓位由一等舱变为三等舱时，其存活新几率是原几率的0.15倍
Class[T.Crew] 0.413685#当乘客仓位由一等舱变为船员时，其存活新几率是原几率的0.41倍
Sex[T.Male] 0.094819#当乘客由女性变为男性时，其存活新几率是原几率的0.09倍
Age[T.Child] 3.086208#当乘客由成年变为小孩时，其存活新几率是原几率的3.09倍
dtype: float64
3.平均边际效应
margeff=results.get_margeff()
print(margeff.summary())
Logit Marginal Effects
=====================================
Dep. Variable: Survived[Yes]
Method: dydx
At: overall
=================================================================================
dy/dx std err z P>|z| [0.025 0.975]
---------------------------------------------------------------------------------
Class[T.2nd] -0.1708 0.037 -4.595 0.000 -0.244 -0.098
Class[T.3rd] -0.3053 0.031 -9.788 0.000 -0.366 -0.244
Class[T.Crew] -0.1442 0.031 -4.711 0.000 -0.204 -0.084
Sex[T.Male] -0.3848 0.021 -18.409 0.000 -0.426 -0.344
Age[T.Child] 0.1841 0.046 3.980 0.000 0.093 0.275
=================================================================================
三.预测
1.计算测试集误差
prob=results.predict(X_test)#测试集中个体存货概率
pred=(prob>=0.5)#以存活概率0.5作为门槛，预测测试集个体是否存活
table=pd.crosstab(y_test,pred,colnames=['Preicted'])#展示混淆矩阵
print(table)
输出：
Preicted False True
Survived[Yes]
0.0 415 32
1.0 110 104
计算混淆矩阵一系列指标，考察测试集预测效果：
table=np.array(table)#为便于计算，将数据框变为数组
TN=table[0,0]#真阴性
FN=table[0,1]#假阴性
FP=table[1,0]#假阳性
TP=table[1,1]#真阳性
Accuracy=(TN+TP)/np.sum(table)#准确率
Error_rate=1-Accuracy#错误率
Sensitivity=TP/(FP+TP)#灵敏率或真阳率
Specificity=TN/(TN+FN)#特异度或真阴率
Recall=TP/(FN+TP)#查全率或召回率
print(Accuracy)
print(Error_rate)
print(Sensitivity)
print(Specificity)
print(Recall)
输出结果如下：
0.7851739788199698
0.21482602118003025
0.48598130841121495
0.9284116331096197
0.7647058823529411
2.ROC曲线图
from sklearn.metrics import roc_curve,RocCurveDisplay,roc_auc_score
import matplotlib.pyplot as plt
#导入计算模块和画图模块
fpr,tpr,thresholds=roc_curve(y_test,prob)
#传入样本真实标签和样本属于正类的概率，计算真阳率和假阳率
auc_score=roc_auc_score(y_test,prob)
#计算ROC曲线下面积
RocCurveDisplay(fpr=fpr, tpr=tpr,roc_auc=auc_score).plot()
plt.title('ROC Curve')
plt.show()
3.计算科恩kappa指标
from sklearn.metrics import cohen_kappa_score
print(cohen_kappa_score(y_test,pred))
输出：
0.4578900555613312
预测值和实际值具备中等一致性
699
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
