数据聚合技术Aggregation

最新推荐文章于 2022-09-25 21:57:41 发布

法蒂芬

最新推荐文章于 2022-09-25 21:57:41 发布

阅读量9.8k

点赞数 1

分类专栏：实战网课 python 文章标签：大数据 python

本文链接：https://blog.csdn.net/weixin_44039183/article/details/108019912

版权

实战网课 python 专栏收录该内容

15 篇文章 0 订阅

订阅专栏

数据聚合技术Aggregation

数据聚合技术Aggregation

数据聚合技术Aggregation

引入相关库

import numpy as np
import pandas as pd
from pandas import Series,DataFrame

数据获取

df=pd.read_csv('../homework/city_weather.csv')

g=df.groupby('city')

数据聚合

通过agg函数求最大值

g.agg('max')

	date	temperature	wind
city
BJ	31/01/2016	19	5
GZ	31/07/2016	25	5
SH	27/03/2016	20	5
SZ	25/09/2016	20	4

通过agg函数求最小值

g.agg('min')

	date	temperature	wind
city
BJ	03/01/2016	-3	2
GZ	14/08/2016	-1	2
SH	03/07/2016	-10	2
SZ	11/09/2016	-10	1

自定义一个聚合函数

def foo(attr):
    print(type(attr))
    return np.nan

向agg方法传入该函数

g.agg(foo)

<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>
<class 'pandas.core.series.Series'>

	date	temperature	wind
city
BJ	NaN	NaN	NaN
GZ	NaN	NaN	NaN
SH	NaN	NaN	NaN
SZ	NaN	NaN	NaN

查看每一行结果Series里面存放的什么

def foo(attr):
    print(type(attr)),print(attr)
    return np.nan
g.agg(foo)

<class 'pandas.core.series.Series'>
Series([], Name: date, dtype: object)
<class 'pandas.core.series.Series'>
0    03/01/2016
1    17/01/2016
2    31/01/2016
3    14/02/2016
4    28/02/2016
5    13/03/2016
Name: date, dtype: object
<class 'pandas.core.series.Series'>
14    17/07/2016
15    31/07/2016
16    14/08/2016
17    28/08/2016
Name: date, dtype: object
<class 'pandas.core.series.Series'>
6     27/03/2016
7     10/04/2016
8     24/04/2016
9     08/05/2016
10    22/05/2016
11    05/06/2016
12    19/06/2016
13    03/07/2016
Name: date, dtype: object
<class 'pandas.core.series.Series'>
18    11/09/2016
19    25/09/2016
Name: date, dtype: object
<class 'pandas.core.series.Series'>
Series([], Name: temperature, dtype: int64)
<class 'pandas.core.series.Series'>
0     8
1    12
2    19
3    -3
4    19
5     5
Name: temperature, dtype: int64
<class 'pandas.core.series.Series'>
14    10
15    -1
16     1
17    25
Name: temperature, dtype: int64
<class 'pandas.core.series.Series'>
6     -4
7     19
8     20
9     17
10     4
11   -10
12     0
13    -9
Name: temperature, dtype: int64
<class 'pandas.core.series.Series'>
18    20
19   -10
Name: temperature, dtype: int64
<class 'pandas.core.series.Series'>
Series([], Name: wind, dtype: int64)
<class 'pandas.core.series.Series'>
0    5
1    2
2    2
3    3
4    2
5    3
Name: wind, dtype: int64
<class 'pandas.core.series.Series'>
14    2
15    5
16    5
17    4
Name: wind, dtype: int64
<class 'pandas.core.series.Series'>
6     4
7     3
8     3
9     3
10    2
11    4
12    5
13    5
Name: wind, dtype: int64
<class 'pandas.core.series.Series'>
18    1
19    4
Name: wind, dtype: int64

	date	temperature	wind
city
BJ	NaN	NaN	NaN
GZ	NaN	NaN	NaN
SH	NaN	NaN	NaN
SZ	NaN	NaN	NaN

把函数改成attr的最大值减去最小值，例如北京的wind为max-min=5-2=3

def foo(attr):
    return attr.max()-attr.min()
g.agg(foo)

	temperature	wind
city
BJ	22	3
GZ	26	3
SH	30	3
SZ	30	3

查看原始的DataFrame

df

	date	city	temperature	wind
0	03/01/2016	BJ	8	5
1	17/01/2016	BJ	12	2
2	31/01/2016	BJ	19	2
3	14/02/2016	BJ	-3	3
4	28/02/2016	BJ	19	2
5	13/03/2016	BJ	5	3
6	27/03/2016	SH	-4	4
7	10/04/2016	SH	19	3
8	24/04/2016	SH	20	3
9	08/05/2016	SH	17	3
10	22/05/2016	SH	4	2
11	05/06/2016	SH	-10	4
12	19/06/2016	SH	0	5
13	03/07/2016	SH	-9	5
14	17/07/2016	GZ	10	2
15	31/07/2016	GZ	-1	5
16	14/08/2016	GZ	1	5
17	28/08/2016	GZ	25	4
18	11/09/2016	SZ	20	1
19	25/09/2016	SZ	-10	4

对df做groupby，传入两个columns，‘city’和‘wind’，产生一个groupby对象

g_new=df.groupby(['city','wind'])
g_new

<pandas.core.groupby.generic.DataFrameGroupBy object at 0x0000017B3ADF1E08>

对新产生的groupby对象做group，比原始的g的groups要多，因为加上了wind,对于每个城市不同的wind又会分group

g_new.groups

{('BJ', 2): Int64Index([1, 2, 4], dtype='int64'),
 ('BJ', 3): Int64Index([3, 5], dtype='int64'),
 ('BJ', 5): Int64Index([0], dtype='int64'),
 ('GZ', 2): Int64Index([14], dtype='int64'),
 ('GZ', 4): Int64Index([17], dtype='int64'),
 ('GZ', 5): Int64Index([15, 16], dtype='int64'),
 ('SH', 2): Int64Index([10], dtype='int64'),
 ('SH', 3): Int64Index([7, 8, 9], dtype='int64'),
 ('SH', 4): Int64Index([6, 11], dtype='int64'),
 ('SH', 5): Int64Index([12, 13], dtype='int64'),
 ('SZ', 1): Int64Index([18], dtype='int64'),
 ('SZ', 4): Int64Index([19], dtype='int64')}

g.groups

{('BJ': Int64Index([0,1,2,3,4,5], dtype='int64'),
  'GZ': Int64Index([14,15,16,17], dtype='int64'),
  'SH': Int64Index([6, 7, 8, 9, 10, 11, 12, 13], dtype='int64'),
  'SZ': Int64Index([18, 19], dtype='int64'),

在g获取BJ这一group

g.get_group('BJ')

	date	temperature	wind
0	03/01/2016	8	5
1	17/01/2016	12	2
2	31/01/2016	19	2
3	14/02/2016	-3	3
4	28/02/2016	19	2
5	13/03/2016	5	3

在g_new获取BJ，wind为2这一group

g_new.get_group(('BJ',2))

	date	city	temperature	wind
1	17/01/2016	BJ	12	2
2	31/01/2016	BJ	19	2
4	28/02/2016	BJ	19	2

在g_new获取BJ，wind为3这一group

g_new.get_group(('BJ',3))

	date	city	temperature	wind
3	14/02/2016	BJ	-3	3
5	13/03/2016	BJ	5	3

用for循环访问的按多个columns做group的所有group

for (name_1,name_2),group in g_new:
    print(name_1,name_2)
    print(group)
BJ 2
         date city  temperature  wind
1  17/01/2016   BJ           12     2
2  31/01/2016   BJ           19     2
4  28/02/2016   BJ           19     2
BJ 3
         date city  temperature  wind
3  14/02/2016   BJ           -3     3
5  13/03/2016   BJ            5     3
BJ 5
         date city  temperature  wind
0  03/01/2016   BJ            8     5
GZ 2
          date city  temperature  wind
14  17/07/2016   GZ           10     2
GZ 4
          date city  temperature  wind
17  28/08/2016   GZ           25     4
GZ 5
          date city  temperature  wind
15  31/07/2016   GZ           -1     5
16  14/08/2016   GZ            1     5
SH 2
          date city  temperature  wind
10  22/05/2016   SH            4     2
SH 3
         date city  temperature  wind
7  10/04/2016   SH           19     3
8  24/04/2016   SH           20     3
9  08/05/2016   SH           17     3
SH 4
          date city  temperature  wind
6   27/03/2016   SH           -4     4
11  05/06/2016   SH          -10     4
SH 5
          date city  temperature  wind
12  19/06/2016   SH            0     5
13  03/07/2016   SH           -9     5
SZ 1
          date city  temperature  wind
18  11/09/2016   SZ           20     1
SZ 4
          date city  temperature  wind
19  25/09/2016   SZ          -10     4

法蒂芬

关注

1
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
数据聚合技术Aggregation

数据聚合技术Aggregation数据聚合技术Aggregation引入相关库数据获取数据聚合数据聚合技术Aggregation引入相关库import numpy as npimport pandas as pdfrom pandas import Series,DataFrame数据获取df=pd.read_csv('../homework/city_weather.csv')g=df.groupby('city')数据聚合通过agg函数求最大值g.agg('max')
复制链接

扫一扫