泰坦尼克号
（根据网上课程编写，并修改了课程的错误和版本问题）
- 数据预处理： 缺失值补充
- 新库中KFold参数为两个：fold = KFold(n_split,shuffle)， 调用时用fold.split(titanic)
- 正则：([A-Za-z]+)\. 解释如下：[A-Za-z]代表任一字母（大写或小写） + 代表+号前的字符出现1次或多次 \. 代表 .
- 特征选择函数 SelectKBest(chi2, k=3)，k表示维度，chi2指定，默认为f_classif
- RandomForestClassifier()
- model_selection.cross_val_score()
import pandas #ipython notebook
titanic = pandas.read_csv("titanic_train.csv")
titanic.head(5)
# print (titanic.describe())
| PassengerId | Survived | Pclass | Name | Sex | Age | SibSp | Parch | Ticket | Fare | Cabin | Embarked | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 0 | 3 | Braund, Mr. Owen Harris | male | 22.0 | 1 | 0 | A/5 21171 | 7.2500 | NaN | S |
| 1 | 2 | 1 | 1 | Cumings, Mrs. John Bradley (Florence Briggs Th... | female | 38.0 | 1 | 0 | PC 17599 | 71.2833 | C85 | C |
| 2 | 3 | 1 | 3 | Heikkinen, Miss. Laina | female | 26.0 | 0 | 0 | STON/O2. 3101282 | 7.9250 | NaN | S |
| 3 | 4 | 1 | 1 | Futrelle, Mrs. Jacques Heath (Lily May Peel) | female | 35.0 | 1 | 0 | 113803 | 53.1000 | C123 | S |
| 4 | 5 | 0 | 3 | Allen, Mr. William Henry | male | 35.0 | 0 | 0 | 373450 | 8.0500 | NaN | S |
#age样本缺失值处理
titanic["Age"] = titanic["Age"].fillna(titanic["Age"].median())
print(titanic.describe())
PassengerId Survived Pclass Age SibSp \
count 891.000000 891.000000 891.000000 891.000000 891.000000
mean 446.000000 0.383838 2.308642 29.361582 0.523008
std 257.353842 0.486592 0.836071 13.019697 1.102743
min 1.000000 0.000000 1.000000 0.420000 0.000000
25% 223.500000 0.000000 2.000000 22.000000 0.000000
50% 446.000000 0.000000 3.000000 28.000000 0.000000
75% 668.500000 1.000000 3.000000 35.000000 1.000000
max 891.000000 1.000000 3.000000 80.000000 8.000000
Parch Fare
count 891.000000 891.000000
mean 0.381594 32.204208
std 0.806057 49.693429
min 0.000000 0.000000
25% 0.000000 7.910400
50% 0.000000 14.454200
75% 0.000000 31.000000
max 6.000000 512.329200
print(titanic["Sex"].unique())
# Replace all the occurences of male with the number 0.
titanic.loc[titanic["Sex"] == "male", "Sex"] = 0
titanic.loc[titanic["Sex"] == "female", "Sex"] = 1
['male' 'female']
# 登船地点
print(titanic["Embarked"].unique())
titanic["Embarked"] = titanic["Embarked"].fillna('S')
titanic.loc[titanic["Embarked"] == "S", "Embarked"] = 0
titanic.loc[titanic["Embarked"] == "C", "Embarked"] = 1
titanic.loc[titanic["Embarked"] == "Q", "Embarked"] = 2
['S' 'C' 'Q' nan]
# Import the linear regression class
from sklearn.linear_model import LinearRegression
# Sklearn also has a helper that makes it easy to do cross validation
from sklearn.model_selection import KFold
# from sklearn.model_selection import train_test_split
# The columns we'll use to predict the target
predictors = ["Pclass", "Sex", "Age", "SibSp", "Parch", "Fare", "Embarked"]
# Initialize our algorithm class
alg = LinearRegression()
# Generate cross validation folds for the titanic dataset. It return the row indices corresponding to train and test.
# We set random_state to ensure we get the same splits every time we run this.
kf = KFold( 3, shuffle=False)
predictions = []
for train, test in kf.split(titanic):
# The predictors we're using the train the algorithm. Note how we only take the rows in the train folds.
train_predictors = (titanic[predictors].iloc[train,:])
# The target we're using to train the algorithm.
train_target = titanic["Survived"].iloc[train]
# Training the algorithm using the predictors and target.
alg.fit(train_predictors, train_target)
# We can now make predictions on the test fold
test_predictions = alg.predict(titanic[predictors].iloc[test,:])
predictions.append(test_predictions)
import numpy as np
# The predictions are in three separate numpy arrays. Concatenate them into one.
# We concatenate them on axis 0, as they only have one axis.
predictions = np.concatenate(predictions, axis=0)
# Map predictions to outcomes (only possible outcomes are 1 and 0)
predictions[predictions > .5] = 1
predictions[predictions <=.5] = 0
accuracy = sum(predictions == titanic["Survived"]) / len(predictions)
print(accuracy)
0.7833894500561167
from sklearn import model_selection
from sklearn.linear_model import LogisticRegression
# Initialize our algorithm
alg = LogisticRegression(random_state=1)
# Compute the accuracy score for all the cross validation folds. (much simpler than what we did before!)
scores = model_selection.cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=3)
# Take the mean of the scores (because we have one for each fold)
print(scores.mean())
0.7957351290684623
titanic_test = pandas.read_csv("test.csv")
titanic_test["Age"] = titanic_test["Age"].fillna(titanic["Age"].median())
titanic_test["Fare"] = titanic_test["Fare"].fillna(titanic_test["Fare"].median())
titanic_test.loc[titanic_test["Sex"] == "male", "Sex"] = 0
titanic_test.loc[titanic_test["Sex"] == "female", "Sex"] = 1
titanic_test["Embarked"] = titanic_test["Embarked"].fillna("S")
titanic_test.loc[titanic_test["Embarked"] == "S", "Embarked"] = 0
titanic_test.loc[titanic_test["Embarked"] == "C", "Embarked"] = 1
titanic_test.loc[titanic_test["Embarked"] == "Q", "Embarked"] = 2
from sklearn import model_selection
from sklearn.ensemble import RandomForestClassifier
predictors = ["Pclass", "Sex", "Age", "SibSp", "Parch", "Fare", "Embarked"]
# Initialize our algorithm with the default paramters
# n_estimators is the number of trees we want to make
# min_samples_split is the minimum number of rows we need to make a split
# min_samples_leaf is the minimum number of samples we can have at the place where a tree branch ends (the bottom points of the tree)
alg = RandomForestClassifier(random_state=1, n_estimators=10, min_samples_split=2, min_samples_leaf=1)
# Compute the accuracy score for all the cross validation folds. (much simpler than what we did before!)
kf = model_selection.KFold(3, shuffle=True, random_state=1)
scores = model_selection.cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=kf)
# Take the mean of the scores (because we have one for each fold)
print(scores.mean())
0.7957351290684626
#树多些，， 限制放松使得树深一些
alg = RandomForestClassifier(random_state=1, n_estimators=100, min_samples_split=4, min_samples_leaf=2)
kf = model_selection.KFold(3, shuffle=True, random_state=1)
scores = model_selection.cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=kf)
# Take the mean of the scores (because we have one for each fold)
print(scores.mean())
0.8215488215488215
#加特征
titanic["FamilySize"] = titanic["SibSp"] + titanic["Parch"]
titanic["NameLength"] = titanic["Name"].apply(lambda x: len(x))
import re
# A function to get the title from a name.
def get_title(name):
tters, and end with a period.
#[A-Za-z]代表任一字母（大写或小写）
#+ 代表+号前的字符出现1次或多次
#\. 代表 .
title_search = re.search(' ([A-Za-z]+)\.', name)
# If the title exists, extract and return it.
if title_search:
return title_search.group(1)
return ""
# Get all the titles and print how often each one occurs.
titles = titanic["Name"].apply(get_title)
print(pandas.value_counts(titles))
# Map each title to an integer. Some titles are very rare, and are compressed into the same codes as other titles.
title_mapping = {"Mr": 1, "Miss": 2, "Mrs": 3, "Master": 4, "Dr": 5, "Rev": 6, "Major": 7, "Col": 7, "Mlle": 8, "Mme": 8, "Don": 9, "Lady": 10, "Countess": 10, "Jonkheer": 10, "Sir": 9, "Capt": 7, "Ms": 2}
for k,v in title_mapping.items():
titles[titles == k] = v
# Verify that we converted everything.
print(pandas.value_counts(titles))
# Add in the title column.
titanic["Title"] = titles
Mr 517
Miss 182
Mrs 125
Master 40
Dr 7
Rev 6
Mlle 2
Major 2
Col 2
Countess 1
Capt 1
Jonkheer 1
Sir 1
Lady 1
Don 1
Mme 1
Ms 1
Name: Name, dtype: int64
1 517
2 183
3 125
4 40
5 7
6 6
7 5
10 3
8 3
9 2
Name: Name, dtype: int64
#特征筛选，查看特征重要性 feature_selection
import numpy as np
from sklearn.feature_selection import SelectKBest, f_classif
import matplotlib.pyplot as plt
predictors = ["Pclass", "Sex", "Age", "SibSp", "Parch", "Fare", "Embarked", "FamilySize", "Title", "NameLength"]
# Perform feature selection
selector = SelectKBest(f_classif, k=3)
selector.fit(titanic[predictors], titanic["Survived"])
# Get the raw p-values for each feature, and transform from p-values into scores
scores = -np.log10(selector.pvalues_)
# Plot the scores. See how "Pclass", "Sex", "Title", and "Fare" are the best?
plt.bar(range(len(predictors)), scores)
plt.xticks(range(len(predictors)), predictors, rotation='vertical')
plt.show()
# Pick only the four best features.
predictors = ["Pclass", "Sex", "Fare", "Title"]
alg = RandomForestClassifier(random_state=1, n_estimators=50, min_samples_split=8, min_samples_leaf=4)
#集成算法
from sklearn.ensemble import GradientBoostingClassifier
import numpy as np
# The algorithms we want to ensemble.
# We're using the more linear predictors for the logistic regression, and everything with the gradient boosting classifier.
algorithms = [
[GradientBoostingClassifier(random_state=1, n_estimators=25, max_depth=3), ["Pclass", "Sex", "Fare", "FamilySize", "Title", "Age", "Embarked"]],
[LogisticRegression(random_state=1), ["Pclass", "Sex", "Fare", "FamilySize", "Title", "Age", "Embarked"]]
]
# Initialize the cross validation folds
kf = KFold(3,shuffle=True, random_state=1).split(titanic)
predictions = []
for train, test in kf:
train_target = titanic["Survived"].iloc[train]
full_test_predictions = []
# Make predictions for each algorithm on each fold
for alg, predictors in algorithms:
# Fit the algorithm on the training data.
alg.fit(titanic[predictors].iloc[train,:], train_target)
# Select and predict on the test fold.
# The .astype(float) is necessary to convert the dataframe to all floats and avoid an sklearn error.
test_predictions = alg.predict_proba(titanic[predictors].iloc[test,:].astype(float))[:,1]
full_test_predictions.append(test_predictions)
# Use a simple ensembling scheme -- just average the predictions to get the final classification.
test_predictions = (full_test_predictions[0]*3 + full_test_predictions[1]) / 4
# Any value over .5 is assumed to be a 1 prediction, and below .5 is a 0 prediction.
test_predictions[test_predictions <= .5] = 0
test_predictions[test_predictions > .5] = 1
predictions.append(test_predictions)
# Put all the predictions together into one array.
predictions = np.concatenate(predictions, axis=0)
# Compute accuracy by comparing to the training data.
accuracy = sum(predictions == titanic["Survived"]) / len(predictions)
print(accuracy)
0.5499438832772167
titles = titanic_test["Name"].apply(get_title)
# We're adding the Dona title to the mapping, because it's in the test set, but not the training set
title_mapping = {"Mr": 1, "Miss": 2, "Mrs": 3, "Master": 4, "Dr": 5, "Rev": 6, "Major": 7, "Col": 7, "Mlle": 8, "Mme": 8, "Don": 9, "Lady": 10, "Countess": 10, "Jonkheer": 10, "Sir": 9, "Capt": 7, "Ms": 2, "Dona": 10}
for k,v in title_mapping.items():
titles[titles == k] = v
titanic_test["Title"] = titles
# Check the counts of each unique title.
print(pandas.value_counts(titanic_test["Title"]))
# Now, we add the family size column.
titanic_test["FamilySize"] = titanic_test["SibSp"] + titanic_test["Parch"]
1 240
2 79
3 72
4 21
7 2
6 2
10 1
5 1
Name: Title, dtype: int64
predictors = ["Pclass", "Sex", "Age", "Fare", "Embarked", "FamilySize", "Title"]
algorithms = [
[GradientBoostingClassifier(random_state=1, n_estimators=25, max_depth=3), predictors],
[LogisticRegression(random_state=1), ["Pclass", "Sex", "Fare", "FamilySize", "Title", "Age", "Embarked"]]
]
full_predictions = []
for alg, predictors in algorithms:
# Fit the algorithm using the full training data.
alg.fit(titanic[predictors], titanic["Survived"])
# Predict using the test dataset. We have to convert all the columns to floats to avoid an error.
predictions = alg.predict_proba(titanic_test[predictors].astype(float))[:,1]
full_predictions.append(predictions)
# The gradient boosting classifier generates better predictions, so we weight it higher.
predictions = (full_predictions[0] * 3 + full_predictions[1]) / 4
predictions
D:\Anoconda\lib\site-packages\sklearn\linear_model\_logistic.py:818: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
Increase the number of iterations (max_iter) or scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG,
array([0.11608616, 0.47313163, 0.12527407, 0.1305029 , 0.52004311,
0.14392415, 0.63984342, 0.18138299, 0.6778233 , 0.12095368,
0.12054308, 0.21154493, 0.91148439, 0.10814111, 0.89180609,
0.87818912, 0.1650196 , 0.13910405, 0.53990167, 0.551889 ,
0.22474624, 0.53695164, 0.90686439, 0.39533671, 0.88099059,
0.10288684, 0.9105501 , 0.13737153, 0.31376951, 0.12631623,
0.11615631, 0.18430576, 0.54919197, 0.49415142, 0.42910392,
0.14215743, 0.50825222, 0.52417057, 0.13230705, 0.28222509,
0.11076129, 0.47224516, 0.09914069, 0.83492474, 0.90028243,
0.14946376, 0.31905677, 0.13752021, 0.89021874, 0.53830768,
0.36218205, 0.17945767, 0.83411368, 0.87864985, 0.17768223,
0.13743949, 0.10594413, 0.12307288, 0.12054944, 0.912131 ,
0.13115382, 0.15454243, 0.13002401, 0.66586834, 0.66161485,
0.87338584, 0.67298304, 0.29010609, 0.35826283, 0.84875193,
0.6621158 , 0.12688447, 0.55224753, 0.37379733, 0.91064833,
0.40981952, 0.12962877, 0.83764819, 0.15754935, 0.6621158 ,
0.68151841, 0.199957 , 0.20583641, 0.12054308, 0.1876183 ,
0.13090738, 0.65624963, 0.53041797, 0.65419471, 0.80328565,
0.53643959, 0.12053355, 0.89344775, 0.12962877, 0.29113646,
0.12308551, 0.86391073, 0.14611431, 0.58582725, 0.12192637,
0.90507833, 0.1487208 , 0.13752021, 0.1222069 , 0.62296229,
0.13079695, 0.1462909 , 0.13752021, 0.12968111, 0.17805571,
0.14320048, 0.65420052, 0.89686624, 0.67202899, 0.87958304,
0.14013126, 0.11761531, 0.69866896, 0.369716 , 0.86312987,
0.87861506, 0.12587932, 0.90367706, 0.12049219, 0.13752021,
0.56989533, 0.12588394, 0.63632568, 0.13347645, 0.13308618,
0.12658342, 0.51402225, 0.23707771, 0.10764469, 0.09812515,
0.12399516, 0.13310033, 0.16414568, 0.51965018, 0.12217545,
0.20839696, 0.905255 , 0.19234654, 0.16288954, 0.43252003,
0.10460664, 0.34145834, 0.1349816 , 0.47224516, 0.34281095,
0.9150228 , 0.13168701, 0.10605776, 0.48641364, 0.11240039,
0.12396799, 0.91030126, 0.57927986, 0.43252003, 0.51097235,
0.65419138, 0.57892305, 0.82284027, 0.12047252, 0.28604791,
0.58375961, 0.3031538 , 0.14606114, 0.90355986, 0.52205389,
0.12051582, 0.13260088, 0.12396504, 0.13162456, 0.13180211,
0.87544577, 0.87770527, 0.29748826, 0.83379914, 0.85394535,
0.15754935, 0.33614481, 0.90334825, 0.13752021, 0.91674518,
0.13621589, 0.85707224, 0.12265744, 0.1414593 , 0.13585733,
0.13547124, 0.2615505 , 0.49879649, 0.12640652, 0.72288221,
0.10727619, 0.85708385, 0.59003429, 0.16902874, 0.53774638,
0.64770495, 0.66407876, 0.6048431 , 0.87648409, 0.16545898,
0.26353344, 0.62898006, 0.16722597, 0.89151239, 0.12309417,
0.127611 , 0.12047645, 0.24831085, 0.79802557, 0.41149964,
0.30039786, 0.65422041, 0.21582703, 0.89844275, 0.12962877,
0.81556725, 0.13597309, 0.84397565, 0.12687469, 0.87848249,
0.59630913, 0.12492255, 0.65419471, 0.11396013, 0.14478507,
0.25316202, 0.89445045, 0.11621755, 0.13753447, 0.34495662,
0.12790127, 0.19280511, 0.1403001 , 0.81278248, 0.89757781,
0.8764603 , 0.8246372 , 0.32985937, 0.12054243, 0.33097517,
0.28960309, 0.88001689, 0.16051699, 0.86312987, 0.58951613,
0.74977708, 0.15427546, 0.39857128, 0.13320041, 0.12632896,
0.12051582, 0.13752021, 0.12962877, 0.83210887, 0.12687265,
0.10829711, 0.1268804 , 0.85114827, 0.65202653, 0.16804208,
0.12054308, 0.22527114, 0.12051582, 0.50825222, 0.14031173,
0.34621912, 0.13752021, 0.91585286, 0.63242282, 0.13162416,
0.85927351, 0.16037479, 0.12507105, 0.14380903, 0.17086699,
0.51978252, 0.66306258, 0.65419471, 0.64321196, 0.71428972,
0.10527009, 0.12049219, 0.36739267, 0.13162456, 0.12962877,
0.33858104, 0.59221078, 0.13162456, 0.50262011, 0.120044 ,
0.12221704, 0.78214546, 0.12631623, 0.33515934, 0.11974185,
0.11749083, 0.17758584, 0.12164696, 0.133141 , 0.65419471,
0.8211081 , 0.33592591, 0.67771416, 0.20869482, 0.42050239,
0.13919512, 0.13794091, 0.12051779, 0.61683764, 0.89999581,
0.67464103, 0.23647358, 0.1765905 , 0.1213635 , 0.18705969,
0.1222069 , 0.13464434, 0.16414568, 0.46045705, 0.90515213,
0.12485392, 0.86892053, 0.34709001, 0.14585039, 0.17320122,
0.81934824, 0.33252581, 0.13162416, 0.64291933, 0.12136688,
0.25679033, 0.15446923, 0.09326367, 0.21115849, 0.35136293,
0.17856622, 0.11747726, 0.14689857, 0.91334888, 0.33472757,
0.61877993, 0.16414568, 0.62126457, 0.16774844, 0.85266978,
0.89655915, 0.16545898, 0.24611915, 0.16006513, 0.70327315,
0.15827211, 0.85634043, 0.12054177, 0.13752021, 0.56988285,
0.10391274, 0.87788259, 0.86962861, 0.1305029 , 0.92015643,
0.15582768, 0.1309077 , 0.53155834, 0.89622127, 0.17553662,
0.15585283, 0.90915277, 0.16579567, 0.13122589, 0.87592006,
0.90842303, 0.48844403, 0.17292445, 0.19909204, 0.13524414,
0.13752021, 0.13987028, 0.5390601 , 0.5941846 , 0.16075475,
0.83407033, 0.12398808, 0.11946654, 0.14628145, 0.18797311,
0.38986125, 0.87810898, 0.56353165, 0.12784258, 0.1030095 ,
0.91276766, 0.14226138, 0.88795149, 0.12588196, 0.12914704,
0.90748673, 0.12667147, 0.91041256, 0.36696583, 0.30783741,
0.19327157, 0.15249874, 0.26395095, 0.65418889, 0.64831126,
0.65419471, 0.90753918, 0.56786184, 0.12962877, 0.85998813,
0.10048191, 0.12962877, 0.41631829])
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
