分组 (groupby)
-
对数据集进行分组,然后对每组进行统计分析
-
SQL能够对数据进行过滤,分组聚合
-
pandas能利用groupby进行更加复杂的分组运算
-
分组运算过程:split->apply->combine
-
拆分:进行分组的根据
-
应用:每个分组运行的计算规则
-
合并:把每个分组的计算结果合并起来
-
- 示例代码:
import pandas as pd
import numpy as np
dict_obj = {'key1' : ['a', 'b', 'a', 'b',
'a', 'b', 'a', 'a'],
'key2' : ['one', 'one', 'two', 'three',
'two', 'two', 'one', 'three'],
'data1': np.random.randn(8),
'data2': np.random.randn(8)}
df_obj = pd.DataFrame(dict_obj)
print(df_obj)
- 运行结果:
data1 data2 key1 key2
0 0.974685 -0.672494 a one
1 -0.214324 0.758372 b one
2 1.508838 0.392787 a two
3 0.522911 0.630814 b three
4 1.347359 -0.177858 a two
5 -0.264616 1.017155 b two
6 -0.624708 0.450885 a one
7 -1.019229 -1.143825 a three
1、GroupBy对象:DataFrameGroupBy,SeriesGroupBy
1. 分组操作
groupby()
进行分组,GroupBy
对象没有进行实际运算,只是包含分组的中间数据按列名分组:
obj.groupby(‘label’)
- 示例代码:
# dataframe根据key1进行分组
print(type(df_obj.groupby('key1')))
# dataframe的 data1 列根据 key1 进行分组
print(type(df_obj['data1'].groupby(df_obj['key1'])))
- 运行结果:
<class 'pandas.core.groupby.DataFrameGroupBy'>
<class 'pandas.core.groupby.SeriesGroupBy'>
2. 分组运算
对GroupBy对象进行分组运算or多重分组运算,如mean()
非数值数据不进行分组运算
- 示例代码:
# 分组运算
grouped1 = df_obj.groupby('key1')
print(grouped1.mean())
grouped2 = df_obj['data1'].groupby(df_obj['key1'])
print(grouped2.mean())
- 运行结果:
data1 data2
key1
a 0.437389 -0.230101
b 0.014657 0.802114
key1
a 0.437389
b 0.014657
Name: data1, dtype: float64
size()
返回每个分组的元素个数
- 示例代码:
# size
print(grouped1.size())
print(grouped2.size())
- 运行结果:
key1
a 5
b 3
dtype: int64
key1
a 5
b 3
dtype: int64
3. 按自定义的key分组
obj.groupby(self_def_key)
自定义的key可为列表或多层列表
obj.groupby([‘label1’, ‘label2’])
->多层dataframe
# 按自定义key分组,列表
self_def_key = [0, 1, 2, 3, 3, 4, 5, 7]
print(df_obj.groupby(self_def_key).size())
# 按自定义key分组,多层列表
print(df_obj.groupby([df_obj['key1'], df_obj['key2']]).size())
# 按多个列多层分组
grouped2 = df_obj.groupby(['key1', 'key2'])
print(grouped2.size())
# 多层分组按key的顺序进行
grouped3 = df_obj.groupby(['key2', 'key1'])
print(grouped3.mean())
# unstack可以将多层索引的结果转换成单层的dataframe
print(grouped3.mean().unstack())
- 运行结果:
0 1
1 1
2 1
3 2
4 1
5 1
7 1
dtype: int64
key1 key2
a one 2
three 1
two 2
b one 1
three 1
two 1
dtype: int64
key1 key2
a one 2
three 1
two 2
b one 1
three 1
two 1
dtype: int64
data1 data2
key2 key1
one a 0.174988 -0.110804
b -0.214324 0.758372
three a -1.019229 -1.143825
b 0.522911 0.630814
two a 1.428099 0.107465
b -0.264616 1.017155
data1 data2
key1 a b a b
key2
one 0.174988 -0.214324 -0.110804 0.758372
three -1.019229 0.522911 -1.143825 0.630814
two 1.428099 -0.264616 0.107465 1.017155