复习:在前面我们已经学习了Pandas基础,第二章我们开始进入数据分析的业务部分,在第二章第一节的内容中,我们学习了 数据的清洗 ,这一部分十分重要,只有数据变得相对干净,我们之后对数据的分析才可以更有力。而这一节,我们要做的是数据重构,数据重构依旧属于数据理解(准备)的范围。
开始之前,导入numpy、pandas包和数据
import numpy as np
import pandas as pd
text_left_up = pd. read_csv( r"C:\Users\Administrator\Desktop\hands-on-data-analysis-master\hands-on-data-analysis-master\第二章项目集合\data\train-left-up.csv" )
text_left_up. head( )
PassengerId Survived Pclass Name 0 1 0 3 Braund, Mr. Owen Harris 1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... 2 3 1 3 Heikkinen, Miss. Laina 3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) 4 5 0 3 Allen, Mr. William Henry
2 第二章:数据重构
2.4 数据的合并
2.4.1 任务一:将data文件夹里面的所有数据都载入,观察数据的之间的关系
text_left_down = pd. read_csv( r"C:\Users\Administrator\Desktop\hands-on-data-analysis-master\hands-on-data-analysis-master\第二章项目集合\data\train-left-down.csv" )
text_right_down = pd. read_csv( r"C:\Users\Administrator\Desktop\hands-on-data-analysis-master\hands-on-data-analysis-master\第二章项目集合\data\train-right-down.csv" )
text_right_up = pd. read_csv( r"C:\Users\Administrator\Desktop\hands-on-data-analysis-master\hands-on-data-analysis-master\第二章项目集合\data\train-right-up.csv" )
text_left_down . head( )
PassengerId Survived Pclass Name 0 440 0 2 Kvillner, Mr. Johan Henrik Johannesson 1 441 1 2 Hart, Mrs. Benjamin (Esther Ada Bloomfield) 2 442 0 3 Hampe, Mr. Leon 3 443 0 3 Petterson, Mr. Johan Emil 4 444 1 2 Reynaldo, Ms. Encarnacion
text_right_down . head( )
Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 male 31.0 0 0 C.A. 18723 10.500 NaN S 1 female 45.0 1 1 F.C.C. 13529 26.250 NaN S 2 male 20.0 0 0 345769 9.500 NaN S 3 male 25.0 1 0 347076 7.775 NaN S 4 female 28.0 0 0 230434 13.000 NaN S
text_right_up. head( )
Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 male 22.0 1 0 A/5 21171 7.2500 NaN S 1 female 38.0 1 0 PC 17599 71.2833 C85 C 2 female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 3 female 35.0 1 0 113803 53.1000 C123 S 4 male 35.0 0 0 373450 8.0500 NaN S
【提示】结合之前我们加载的train.csv数据,大致预测一下上面的数据是什么
2.4.2:任务二:使用concat方法:将数据train-left-up.csv和train-right-up.csv横向合并为一张表,并保存这张表为result_up
result_up = pd. concat( [ text_left_up, text_right_up] , axis = 1 )
result_up
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C 2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S ... ... ... ... ... ... ... ... ... ... ... ... ... 434 435 0 1 Silvey, Mr. William Baird male 50.0 1 0 13507 55.9000 E44 S 435 436 1 1 Carter, Miss. Lucile Polk female 14.0 1 2 113760 120.0000 B96 B98 S 436 437 0 3 Ford, Miss. Doolina Margaret "Daisy" female 21.0 2 2 W./C. 6608 34.3750 NaN S 437 438 1 2 Richards, Mrs. Sidney (Emily Hocking) female 24.0 2 3 29106 18.7500 NaN S 438 439 0 1 Fortune, Mr. Mark male 64.0 1 4 19950 263.0000 C23 C25 C27 S
439 rows × 12 columns
2.4.3 任务三:使用concat方法:将train-left-down和train-right-down横向合并为一张表,并保存这张表为result_down。然后将上边的result_up和result_down纵向合并为result。
result_down = pd. concat( [ text_left_down, text_right_down] , axis = 1 )
result_down. head( )
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 440 0 2 Kvillner, Mr. Johan Henrik Johannesson male 31.0 0 0 C.A. 18723 10.500 NaN S 1 441 1 2 Hart, Mrs. Benjamin (Esther Ada Bloomfield) female 45.0 1 1 F.C.C. 13529 26.250 NaN S 2 442 0 3 Hampe, Mr. Leon male 20.0 0 0 345769 9.500 NaN S 3 443 0 3 Petterson, Mr. Johan Emil male 25.0 1 0 347076 7.775 NaN S 4 444 1 2 Reynaldo, Ms. Encarnacion female 28.0 0 0 230434 13.000 NaN S
result = pd. concat( [ result_up, result_down] )
result. head( )
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C 2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
2.4.4 任务四:使用DataFrame自带的方法join方法和append:完成任务二和任务三的任务
result_up = text_left_up. join( text_right_up)
result_down = text_left_down. join( text_right_down)
result = result_up. append( result_down)
result. head( )
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C 2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
2.4.5 任务五:使用Panads的merge方法和DataFrame的append方法:完成任务二和任务三的任务
result_up = pd. merge( text_left_up, text_right_up, left_index = True , right_index = True )
result_down = pd. merge( text_left_down, text_right_down, left_index = True , right_index = True )
result = result_up. append( result_down)
result. head( )
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C 2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
【思考】对比merge、join以及concat的方法的不同以及相同。思考一下在任务四和任务五的情况下,为什么都要求使用DataFrame的append方法,如何只要求使用merge或者join可不可以完成任务四和任务五呢?
2.4.6 任务六:完成的数据保存为result.csv
result. to_csv( r"result.csv" )
2.5 换一种角度看数据
2.5.1 任务一:将我们的数据变为Series类型的数据
text = pd. read_csv( r"result.csv" )
type ( text)
result= text. stack( ) . head( 20 )
result. head( )
0 Unnamed: 0 0
PassengerId 1
Survived 0
Pclass 3
Name Braund, Mr. Owen Harris
dtype: object
result. to_csv( r"result_new.csv" )
read_text = pd. read_csv( r"result_new.csv" )
read_text. head( )
Unnamed: 0 Unnamed: 1 0 0 0 Unnamed: 0 0 1 0 PassengerId 1 2 0 Survived 0 3 0 Pclass 3 4 0 Name Braund, Mr. Owen Harris