数据拼接
具有相同的列
import pandas as pd
df1 = pd.read_csv('../data/xxx_1.csv')
df2 = pd.read_csv('../data/xxx_2.csv')
df3 = pd.read_csv('../data/xxx_3.csv')
row_concat = pd.concat([df1, df2, df3])
将序列转换成数据框才能正确拼接
new_row_pd = pd.DataFrame([['n1', 'n2', 'n3', 'n4']], columns=['A', 'B', 'C', 'D'])
pd.concat([df1, new_row_pd])
对字典类型的数据进行拼接, 需要设置参数 ignore_index
data_dict = {'A': 'n1', 'B': 'n2', 'C': 'n3', 'D': 'n4'}
df1.append(data_dict, ignore_index=True)
concat
拼接, 默认会保留原来的行号. ignore_index=True
会重置行号
row_concat = pd.concat([df1, df2, df3], ignore_index=True)
列拼接:
col_concat = pd.concat([df1, df2, df3], axis=1)
col_concat['A']
col_concat['new_col_list'] = ['n1', 'n2', 'n3', 'n4']
col_concat['new_col_list'] = pd.Series(['n1', 'n2', 'n3', 'n4'])
重置列名
col_concat = pd.concat([df1, df2, df3], axis=1, ignore_index=True)
列名不同
不存在的列的数据为 NaN
df1.columns = ['A', 'B', 'C', 'D']
df2.columns = ['E', 'F', 'G', 'H']
df3.columns = ['A', 'C', 'F', 'H']
row_concat = pd.concat([df1, df2, df3])
得到行列都有值的部分 (join='inner'
):
row_concat = pd.concat([df1, df3], join='inner')
一对一合并
person = pd.read_csv('../data/surver_person.csv')
survey = pd.read_csv('../data/surver_site.csv')
visited = pd.read_csv('../data/surver_visited.csv')
# 行号为 0, 2, 6 的行
visited_subset = visited.ix[[0, 2, 6]]
o2o_merge = site.merge(visited_subset, left_on='name', right_on='site')
一对多合并
不含重复值的数据框会按照有重复值的数据框进行合并
o2m_merge = site.merge(visited, left_on='name', right_on='site')
多对多合并
笛卡尔积
ps = person.merge(survey, left_on='ident', right_on='person')
vs = visited.merge(survey, left_on='ident', right_on='taken')