数据挖掘案例: 泰坦尼克号

*. 数据挖掘流程

（一）数据读取：

• 读取数据，并进行展示
• 统计数据各项指标
• 明确数据规模与要完成任务

（二）特征理解分析

• 单特征分析，逐个变量分析其对结果的影响
• 多变量统计分析，综合考虑多种情况影响
• 统计绘图得出结论

（三）数据清洗与预处理

• 对缺失值进行填充
• 特征标准化/归一化
• 筛选有价值的特征
• 分析特征之间的相关性

（四）建立模型

• 特征数据与标签准备
• 数据集切分
• 多种建模算法对比
• 集成策略等方案改进

一. 数据读取与统计分析

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
#plt.style.use('fivethirtyeight')
sns.set_style('darkgrid')
%matplotlib inline
import warnings
warnings.filterwarnings('ignore')

data = pd.read_csv(r'F:\51学习\study\数据挖掘案例\泰坦尼克号\train.csv')
print(data.shape)
data.head()

# data.info()
# data.dtypes
# data.describe()
# import pandas_profiling
# pandas_profiling.ProfileReport(data)
data.isnull().sum()

• 统计获救情况

plt.figure(figsize = (14, 6))
plt.subplot(1, 2, 1)
data['Survived'].value_counts().plot.pie(explode = [0, 0.08], autopct = '%.2f%%', shadow = True)
plt.title('Survived')
plt.ylabel('')
plt.subplot(1, 2, 2)
sns.countplot('Survived', data = data)
plt.title('Survived')

二. 特征分析 & 缺失值填充

2.1 性别与获救

data.groupby(['Sex','Survived'])['Survived'].count()

fig, ax = plt.subplots(1, 2, figsize = (14, 6))
data[['Sex', 'Survived']].groupby(['Sex']).mean().plot.bar(ax = ax[0])
ax[0].set_title('Survived vs Sex')
sns.countplot('Sex', hue = 'Survived', data = data, ax = ax[1])
ax[1].set_title('Sex: Survived vs Dead')

2.2 船舱等级与获救

pd.crosstab(data.Pclass, data.Survived, margins = True).style.background_gradient(cmap = 'autumn')

fig, ax = plt.subplots(1, 2, figsize = (14, 6))
data['Pclass'].value_counts().plot.bar(ax = ax[0])
ax[0].set(title = 'Number Of Passengers By Pclass', ylabel = 'Count')

sns.countplot('Pclass', hue = 'Survived', data = data, ax = ax[1])
ax[1].set_title('Pclass: Survived vs Dead')

• 船舱等级和性别对结果的影响

pd.crosstab([data.Pclass, data.Sex], data.Survived, margins = True).style.background_gradient(cmap = 'autumn')

pd.pivot_table(data, index = 'Pclass', columns = 'Sex', values = 'Survived')

sns.factorplot('Pclass', 'Survived', hue = 'Sex', data = data)

#sns.pointplot('Pclass', 'Survived', hue = 'Sex', data = data)

2.3 年龄与获救

print('Oldest Passenger was of:',data['Age'].max(),'Years')
print('Youngest Passenger was of:',data['Age'].min(),'Years')
print('Average Age on the ship:',data['Age'].mean(),'Years')

plt.figure(figsize = (14, 6))

plt.subplot(1, 2, 1)
sns.violinplot('Pclass', 'Age', hue = 'Survived', data = data, split = True)
plt.title('Pclass and Age vs Survived')
plt.yticks(range(0, 110, 10));

plt.subplot(1, 2, 2)
sns.violinplot('Sex', 'Age', hue = 'Survived', data = data, split = True)
plt.title('Sex and Age vs Survived')
plt.yticks(range(0, 110, 10));

Oldest Passenger was of: 80.0 Years
Youngest Passenger was of: 0.42 Years
Average Age on the ship: 29.69911764705882 Years

2.4 姓名(称谓) 与获救

data['Initial'] = 0
data['Initial'] = data.Name.str.extract('([A-Za-z]+)\.')
pd.crosstab(data.Initial, data.Sex).T.style.background_gradient(cmap = 'autumn')

data['Initial'].replace(['Mlle','Mme','Ms','Dr','Major','Lady','Countess','Jonkheer','Col','Rev','Capt','Sir','Don'],
                        ['Miss','Miss','Miss','Mr','Mr','Mrs','Mrs','Other','Other','Other','Mr','Mr','Mr'], inplace = True)
pd.crosstab(data.Initial, data.Sex).style.background_gradient(cmap = 'summer')

data.groupby('Initial')['Age'].mean()

2.5 填充缺失值

# 使用每组的均值来进行填充
data.loc[(data.Age.isnull()) & (data.Initial == 'Mr'), 'Age'] = 33
data.loc[(data.Age.isnull()) & (data.Initial == 'Mrs'),'Age']=36
data.loc[(data.Age.isnull()) & (data.Initial == 'Master'),'Age']=5
data.loc[(data.Age.isnull()) & (data.Initial == 'Miss'),'Age']=22
data.loc[(data.Age.isnull()) & (data.Initial == 'Other'),'Age']=46
data.Age.isnull().any()

False

plt.style.use('seaborn-darkgrid')
plt.figure(figsize = (8, 6))
data[data['Survived'] == 0].Age.plot.hist(<