Python数据分析实战【第三章】2.20-分组转换及一般性“拆分-应用-合并”【python】

【课程2.20 分组转换及一般性“拆分-应用-合并”】 课程作业

1.数据分组转换,transform

df = pd.DataFrame({'data1':np.random.rand(5),
                  'data2':np.random.rand(5),
                  'key1':list('aabba'),
                  'key2':['one','two','one','two','one']})
k_mean = df.groupby('key1').mean()
print(df)
print(k_mean)
print(pd.merge(df,k_mean,left_on='key1',right_index=True).add_prefix('mean_'))  # .add_prefix('mean_')：添加前缀

# 通过分组、合并，得到一个包含均值的Dataframe

print(df.groupby('key2').mean()) # 按照key2分组求均值
print(df.groupby('key2').transform(np.mean))
# data1、data2每个位置元素取对应分组列的均值
# 字符串不能进行计算
----------------------------------------------------------------------
      data1     data2 key1 key2
0  0.003727  0.390301    a  one
1  0.744777  0.130300    a  two
2  0.887207  0.679309    b  one
3  0.448585  0.169208    b  two
4  0.448045  0.993775    a  one

         data1     data2
key1                    
a     0.398850  0.504792
b     0.667896  0.424258

   mean_data1_x  mean_data2_x mean_key1 mean_key2  mean_data1_y  mean_data2_y
0      0.003727      0.390301         a       one      0.398850      0.504792
1      0.744777      0.130300         a       two      0.398850      0.504792
4      0.448045      0.993775         a       one      0.398850      0.504792
2      0.887207      0.679309         b       one      0.667896      0.424258
3      0.448585      0.169208         b       two      0.667896      0.424258
----------------------------------------------------------------------
         data1     data2
key2                    
one   0.446326  0.687795
two   0.596681  0.149754

      data1     data2
0  0.446326  0.687795
1  0.596681  0.149754
2  0.446326  0.687795
3  0.596681  0.149754
4  0.446326  0.687795

一般化Groupby方法：apply

df = pd.DataFrame({'data1':np.random.rand(5),
                  'data2':np.random.rand(5),
                  'key1':list('aabba'),
                  'key2':['one','two','one','two','one']})

print(df.groupby('key1').apply(lambda x: x.describe()))
# apply直接运行其中的函数
# 这里为匿名函数，直接描述分组后的统计量

def f_df1(d,n):
    return(d.sort_index()[:n])
def f_df2(d,k1):
    return(d[k1])
print(df.groupby('key1').apply(f_df1,2))
print(df.groupby('key1').apply(f_df2,'data2'))
# f_df1函数：返回排序后的前n行数据
# f_df2函数：返回分组后表的k1列，结果为Series，层次化索引
# 直接运行f_df函数
# 参数直接写在后面，也可以为.apply(f_df,n = 2))
----------------------------------------------------------------------
               data1     data2
key1                          
a    count  3.000000  3.000000
     mean   0.561754  0.233470
     std    0.313439  0.337209
     min    0.325604  0.026906
     25%    0.383953  0.038906
     50%    0.442303  0.050906
     75%    0.679829  0.336753
     max    0.917355  0.622599
b    count  2.000000  2.000000
     mean   0.881906  0.547206
     std    0.079357  0.254051
     min    0.825791  0.367564
     25%    0.853849  0.457385
     50%    0.881906  0.547206
     75%    0.909963  0.637026
     max    0.938020  0.726847

           data1     data2 key1 key2
key1                                
a    0  0.325604  0.050906    a  one
     1  0.917355  0.622599    a  two
b    2  0.825791  0.726847    b  one
     3  0.938020  0.367564    b  two 

key1   
a     0    0.050906
      1    0.622599
      4    0.026906
b     2    0.726847
      3    0.367564

df = pd.DataFrame({'data1':np.random.rand(8),
                  'data2':np.random.rand(8),
                  'key':list('aabbabab')})

print(df)
df_ = df.groupby('key').transform(np.mean)
print(df1)
print(df.join(df_,rsuffix='_mean'))
----------------------------------------------------------------------
创建df为：
       data1     data2 key
0  0.298507  0.481976   a
1  0.761940  0.472252   a
2  0.639491  0.964618   b
3  0.238919  0.756300   b
4  0.231337  0.617190   a
5  0.681743  0.906671   b
6  0.601592  0.798819   a
7  0.071703  0.109292   b 
------
  data1     data2
0  0.560657  0.575840
1  0.560657  0.575840
2  0.497225  0.201286
3  0.497225  0.201286
4  0.560657  0.575840
5  0.497225  0.201286
6  0.560657  0.575840
7  0.497225  0.201286

求和且合并之后结果为：
       data1     data2 key  data1_mean  data2_mean
0  0.298507  0.481976   a    0.473344    0.592559
1  0.761940  0.472252   a    0.473344    0.592559
2  0.639491  0.964618   b    0.407964    0.684220
3  0.238919  0.756300   b    0.407964    0.684220
4  0.231337  0.617190   a    0.473344    0.592559
5  0.681743  0.906671   b    0.407964    0.684220
6  0.601592  0.798819   a    0.473344    0.592559
7  0.071703  0.109292   b    0.407964    0.684220