Python -Pandas 数据粘合

数据拼接

具有相同的列

import pandas as pd
df1 = pd.read_csv('../data/xxx_1.csv')
df2 = pd.read_csv('../data/xxx_2.csv')
df3 = pd.read_csv('../data/xxx_3.csv')

row_concat = pd.concat([df1, df2, df3])

将序列转换成数据框才能正确拼接

new_row_pd = pd.DataFrame([['n1', 'n2', 'n3', 'n4']], columns=['A', 'B', 'C', 'D'])
pd.concat([df1, new_row_pd])

对字典类型的数据进行拼接, 需要设置参数 ignore_index

data_dict = {'A': 'n1', 'B': 'n2', 'C': 'n3', 'D': 'n4'}
df1.append(data_dict, ignore_index=True)

concat 拼接, 默认会保留原来的行号. ignore_index=True 会重置行号

row_concat = pd.concat([df1, df2, df3], ignore_index=True)

列拼接:

col_concat = pd.concat([df1, df2, df3], axis=1)

col_concat['A']

col_concat['new_col_list'] = ['n1', 'n2', 'n3', 'n4']

col_concat['new_col_list'] = pd.Series(['n1', 'n2', 'n3', 'n4'])

重置列名

col_concat = pd.concat([df1, df2, df3], axis=1, ignore_index=True)

列名不同

不存在的列的数据为 NaN

df1.columns = ['A', 'B', 'C', 'D']
df2.columns = ['E', 'F', 'G', 'H']
df3.columns = ['A', 'C', 'F', 'H']

row_concat = pd.concat([df1, df2, df3])

得到行列都有值的部分 (join='inner'):

row_concat = pd.concat([df1, df3], join='inner')

一对一合并

person = pd.read_csv('../data/surver_person.csv')
survey = pd.read_csv('../data/surver_site.csv')
visited = pd.read_csv('../data/surver_visited.csv')

# 行号为 0, 2, 6 的行
visited_subset = visited.ix[[0, 2, 6]]

o2o_merge = site.merge(visited_subset, left_on='name', right_on='site')

一对多合并

不含重复值的数据框会按照有重复值的数据框进行合并

o2m_merge = site.merge(visited, left_on='name', right_on='site')

多对多合并

笛卡尔积

ps = person.merge(survey, left_on='ident', right_on='person')

vs = visited.merge(survey, left_on='ident', right_on='taken')