pandas:分组（GroupBy）

最新推荐文章于 2024-04-25 21:07:42 发布

梦寐_

最新推荐文章于 2024-04-25 21:07:42 发布

阅读量2.9k

点赞数 1

分类专栏： pands 文章标签： group

本文链接：https://blog.csdn.net/HHG20171226/article/details/101647441

版权

pands 专栏收录该内容

24 篇文章 0 订阅

订阅专栏

andas提供了一个灵活高效的groupby功能，它使你能以一种自然的方式对数据集进行切片、切块、摘要等操作。根据一个或多个键（可以是函数、数组或DataFrame列名）拆分pandas对象。计算分组摘要统计，如计数、平均值、标准差，或用户自定义函数。对DataFrame的列应用各种各样的函数。应用组内转换或其他运算，如规格化、线性回归、排名或选取子集等。计算透视表或交叉表。执行分位数分析以及其他分组分析。

groupby 核心为 split-apply-combine

分割-应用-组合

分割，就是按照一些标准进行数据的拆分
应用，给分好的组，应用一些函数上去
组合，分组的数据，捣鼓捣鼓，弄成一个新的数据结构，方便后面应用

将数据拆分成组

将一个dataframe对象分割成组

import pandas as pd
df = pd.DataFrame([('class1','boy',100),
                   ('class2', 'boy', 90),
                   ('class3', 'girl', 80),
                   ('class1', 'unkown', None),
                   ('class2', 'unkown', 90)],index=['dawa','erwa','sanwa','siwa','wuwa'],columns=['class','sex','weight'])

print(df)
'''
        class     sex  weight
dawa   class1     boy   100.0
erwa   class2     boy    90.0
sanwa  class3    girl    80.0
siwa   class1  unkown     NaN
wuwa   class2  unkown    90.0
'''

print(df.groupby('class'))
print(df.groupby('sex',axis='columns'))
print(df.groupby(['class','sex']))

多列分组

df = pd.DataFrame({'A': ['girl', 'boy', 'girl', 'boy',
                         'girl', 'boy', 'girl', 'girl'],
                   'B': ['one', 'one', 'two', 'three',
                         'two', 'two', 'one', 'three'],
                   'C': [3,1,4,5,6,7,8,1],
                   'D': [9,1,2,3,1,6,4,3]})

print(df)
'''
      A      B  C  D
0  girl    one  3  9
1   boy    one  1  1
2  girl    two  4  2
3   boy  three  5  3
4  girl    two  6  1
5   boy    two  7  6
6  girl    one  8  4
'''

grouped = df.groupby('A')
print(grouped)
'''
<pandas.core.groupby.generic.DataFrameGroupBy object at 0x0000019FAE11DE08>
'''
grouped = df.groupby(['A','B'])
print(grouped)
#<pandas.core.groupby.generic.DataFrameGroupBy object at 0x000002AA51E3E2C8>

对于一个待分组的dataframe对象来说
你可以直接用’A’或者’B’中的一个进行分组
当然也可以直接用[‘A’,‘B’]进行分组

获取分组之后的数据

get_group('key name')

df = pd.DataFrame({'A': ['girl', 'boy', 'girl', 'boy',
                         'girl', 'boy', 'girl', 'girl'],
                   'B': ['one', 'one', 'two', 'three',
                         'two', 'two', 'one', 'three'],
                   'C': [3,1,4,5,6,7,8,1],
                   'D': [9,1,2,3,1,6,4,3]})

print(df)
'''
      A      B  C  D
0  girl    one  3  9
1   boy    one  1  1
2  girl    two  4  2
3   boy  three  5  3
4  girl    two  6  1
5   boy    two  7  6
6  girl    one  8  4
'''

grouped = df.groupby('A')
print(grouped)
'''
<pandas.core.groupby.generic.DataFrameGroupBy object at 0x0000019FAE11DE08>
'''

print(grouped.get_group('boy'))
'''
     A      B  C  D
1  boy    one  1  1
3  boy  three  5  3
5  boy    two  7  6
'''
print(grouped.get_group('girl'))
'''
      A      B  C  D
0  girl    one  3  9
2  girl    two  4  2
4  girl    two  6  1
6  girl    one  8  4
7  girl  three  1  3
'''

除了获取分组之后的数据 groups属性

grouped = df.groupby('A')
print(grouped)
print(grouped.groups)
'''
<pandas.core.groupby.generic.DataFrameGroupBy object at 0x0000016A25B5C3C8>
{'boy': Int64Index([1, 3, 5], dtype='int64'), 
'girl': Int64Index([0, 2, 4, 6, 7], dtype='int64')}
'''

将数据拆分成组

Pandas对象可以分成任何对象。有多种方式来拆分对象，如 -

obj.groupby(‘key’)
obj.groupby([‘key1’,’key2’])
obj.groupby(key,axis=1)

import pandas as pd

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)

print (df.groupby('Team'))
#<pandas.core.groupby.DataFrameGroupBy object at 0x00000245D60AD518>

查看分组

import pandas as pd
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)

print (df.groupby('Team').groups)
'''
{'Devils': Int64Index([2, 3], dtype='int64'), 
'Kings': Int64Index([4, 6, 7], dtype='int64'), 
'Riders': Int64Index([0, 1, 8, 11], dtype='int64'), 
'Royals': Int64Index([9, 10], dtype='int64'), 
'kings': Int64Index([5], dtype='int64')}

'''

按多列分组

import pandas as pd
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)
print (df.groupby(['Team','Year']).groups)
'''
{
('Devils', 2014): Int64Index([2], dtype='int64'), 
('Devils', 2015): Int64Index([3], dtype='int64'), 
('Kings', 2014): Int64Index([4], dtype='int64'),
('Kings', 2016): Int64Index([6], dtype='int64'),
('Kings', 2017): Int64Index([7], dtype='int64'), 
('Riders', 2014): Int64Index([0], dtype='int64'), 
('Riders', 2015): Int64Index([1], dtype='int64'), 
('Riders', 2016): Int64Index([8], dtype='int64'), 
('Riders', 2017): Int64Index([11], dtype='int64'),
('Royals', 2014): Int64Index([9], dtype='int64'), 
('Royals', 2015): Int64Index([10], dtype='int64'), 
('kings', 2015): Int64Index([5], dtype='int64')
}
'''

迭代遍历分组


import pandas as pd

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)

grouped = df.groupby('Year')

for name,group in grouped:
    print (name)
    print (group)
'''
2014
     Team  Rank  Year  Points
0  Riders     1  2014     876
2  Devils     2  2014     863
4   Kings     3  2014     741
9  Royals     4  2014     701
2015
      Team  Rank  Year  Points
1   Riders     2  2015     789
3   Devils     3  2015     673
5    kings     4  2015     812
10  Royals     1  2015     804
2016
     Team  Rank  Year  Points
6   Kings     1  2016     756
8  Riders     2  2016     694
2017
      Team  Rank  Year  Points
7    Kings     1  2017     788
11  Riders     2  2017     690
'''

选择一个分组

使用get_group()方法，可以选择一个组。

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)

grouped = df.groupby('Year')
print (grouped.get_group(2014))
'''
     Team  Rank  Year  Points
0  Riders     1  2014     876
2  Devils     2  2014     863
4   Kings     3  2014     741
9  Royals     4  2014     701
'''

聚合

聚合函数为每个组返回单个聚合值。当创建了分组(group by)对象，就可以对分组数据执行多个聚合操作。
一个比较常用的是通过聚合或等效的agg方法聚合

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)

grouped = df.groupby('Year')
print (grouped['Points'].agg(np.mean))
'''
Year
2014    795.25
2015    769.50
2016    725.00
2017    739.00
Name: Points, dtype: float64
'''

另一种查看每个分组的大小的方法是应用`size()`函数

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)
grouped = df.groupby('Team')
print (grouped.agg(np.size))
'''
        Rank  Year  Points
Team                      
Devils     2     2       2
Kings      3     3       3
Riders     4     4       4
Royals     2     2       2
kings      1     1       1
'''

一次应用多个聚合函数

通过分组系列，还可以传递函数的列表或字典来进行聚合，并生成DataFrame作为输出 -

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)

grouped = df.groupby('Team')
agg = grouped['Points'].agg([np.sum, np.mean, np.std])
print (agg)
'''
         sum        mean         std
Team                                
Devils  1536  768.000000  134.350288
Kings   2285  761.666667   24.006943
Riders  3049  762.250000   88.567771
Royals  1505  752.500000   72.831998
kings    812  812.000000         NaN
'''

转换

分组或列上的转换返回索引大小与被分组的索引相同的对象。因此，转换应该返回与组块大小相同的结果

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)

grouped = df.groupby('Team')
score = lambda x: (x - x.mean()) / x.std()*10
print (grouped.transform(score))
'''
         Rank       Year     Points
0  -15.000000 -11.618950  12.843272
1    5.000000  -3.872983   3.020286
2   -7.071068  -7.071068   7.071068
3    7.071068   7.071068  -7.071068
4   11.547005 -10.910895  -8.608621
5         NaN        NaN        NaN
6   -5.773503   2.182179  -2.360428
7   -5.773503   8.728716  10.969049
8    5.000000   3.872983  -7.705963
9    7.071068  -7.071068  -7.071068
10  -7.071068   7.071068   7.071068
11   5.000000  11.618950  -8.157595
'''

过滤

过滤根据定义的标准过滤数据并返回数据的子集。filter()函数用于过滤数据。

ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
         'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
         'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
         'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
         'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)
filter = df.groupby('Team').filter(lambda x: len(x) >= 3)

print (filter)
'''
      Team  Rank  Year  Points
0   Riders     1  2014     876
1   Riders     2  2015     789
4    Kings     3  2014     741
6    Kings     1  2016     756
7    Kings     1  2017     788
8   Riders     2  2016     694
11  Riders     2  2017     690
'''

在上述过滤条件下，要求返回三次以上参加IPL的队伍。

梦寐_

关注

1
点赞
踩
10

收藏

觉得还不错? 一键收藏
1
评论
pandas:分组（GroupBy）

andas提供了一个灵活高效的groupby功能，它使你能以一种自然的方式对数据集进行切片、切块、摘要等操作。根据一个或多个键（可以是函数、数组或DataFrame列名）拆分pandas对象。计算分组摘要统计，如计数、平均值、标准差，或用户自定义函数。对DataFrame的列应用各种各样的函数。应用组内转换或其他运算，如规格化、线性回归、排名或选取子集等。计算透视表或交叉表。执行分位数分析以及其他...
复制链接

扫一扫