Pandas索引，Pandas分组计算，聚合函数，数据IO，时间序列，重采样，数据可视化

Pandas索引

Series索引

In [379]: a = pd.Series(np.random.rand(5),index=list('abcde'))

In [380]: a
Out[380]:
a    0.716350
b    0.922494
c    0.739640
d    0.757864
e    0.364554
dtype: float64

In [381]: a.index.name='alpha'

In [383]: s
Out[383]:
0     a
1     b
2     c
3     d
4     a
5     b
6     c
7     d
8     d
9     b
10    c
dtype: object

In [384]: a
Out[384]:
alpha
a    0.716350
b    0.922494
c    0.739640
d    0.757864
e    0.364554
dtype: float64

DataFrame索引

（1）行索引

 df = pd.DataFrame(np.random.randn(4,3),columns=['one','two','three'])

In [386]: df
Out[386]:
        one       two     three
0  1.284715 -0.320224  1.636582
1 -0.620074 -0.257786 -0.913668
2 -1.291568  0.933435 -0.755850
3  0.517897  1.550158  0.342784

In [387]: df.index
Out[387]: RangeIndex(start=0, stop=4, step=1)

（2）列索引

 df.columns
Out[388]: Index(['one', 'two', 'three'], dtype='object')

（3）给索引取别名

In [389]: df.index.name='row'

In [390]: df.columns.name='col'

In [391]: df
Out[391]:
col       one       two     three
row
0    1.284715 -0.320224  1.636582
1   -0.620074 -0.257786 -0.913668
2   -1.291568  0.933435 -0.755850
3    0.517897  1.550158  0.342784

（4）查看索引的类别

In [392]: pd.*Index?
pd.CategoricalIndex
pd.DatetimeIndex
pd.Float64Index
pd.Index
pd.Int64Index
pd.IntervalIndex
pd.MultiIndex
pd.PeriodIndex
pd.RangeIndex
pd.TimedeltaIndex
pd.UInt64Index

（5）重复索引：索引值有重复的项，对重复的值返回Series，对没有重复的值返回一个数据

In [395]: s['a']
Out[395]:
a    0
a    5
dtype: int32

In [396]: s['c']
Out[396]: 2

（6）判断是否为重复索引，若为False则为重复索引

In [397]: s.index.is_unique
Out[397]: False

In [398]: a.index.unique()
Out[398]: Index(['a', 'b', 'c', 'd', 'e'], dtype='object', name='alpha')

（7）根据具体要求对重复索引进行清洗：groupby分组处理

In [400]: s.groupby(s.index).sum()
Out[400]:
a    5
b    5
c    2
d    3
dtype: int32

In [401]: s.groupby(s.index).mean()
Out[401]:
a    2.5
b    2.5
c    2.0
d    3.0
dtype: float64

In [402]: s.groupby(s.index).first()
Out[402]:
a    0
b    1
c    2
d    3
dtype: int32

多级索引

把更高维度的数据用二维数据表示，可以增强可读性

Series多级索引

• Series多级索引的创建

In [404]: a = [['a','a','b','b','b','c','c'],[1,2,3,1,2,3,3]]

In [405]: t=list(zip(*a))

In [406]: t
Out[406]: [('a', 1), ('a', 2), ('b', 3), ('b', 1), ('b', 2), ('c', 3), ('c', 3)]

In [407]: index = pd.MultiIndex.from_tuples(t,names=['level1','level2'])

In [408]: index
Out[408]:
MultiIndex(levels=[['a', 'b', 'c'], [1, 2, 3]],
           codes=[[0, 0, 1, 1, 1, 2, 2], [0, 1, 2, 0, 1, 2, 2]],
           names=['level1', 'level2'])

In [409]: s=pd.Series(np.random.rand(7),index=index)

In [410]: s
Out[410]:
level1  level2
a       1         0.001792
        2         0.147195
b       3         0.340437
        1         0.933020
        2         0.310989
c       3         0.056918
        3         0.430210
dtype: float64

• Series多级索引的调用

In [411]: s['b']
Out[411]:
level2
3    0.340437
1    0.933020
2    0.310989
dtype: float64

In [412]: s['b':'c']
Out[412]:
level1  level2
b       3         0.340437
        1         0.933020
        2         0.310989
c       3         0.056918
        3         0.430210
dtype: float64

In [413]: s[['a','c']]
Out[413]:
level1  level2
a       1         0.001792
        2         0.147195
c       3         0.056918
        3         0.430210
dtype: float64

In [414]: s[:,2]
Out[414]:
level1
a    0.147195
b    0.310989
dtype: float64

DataFrame多级索引

• DataFrame多级索引的创建

 df = pd.DataFrame(np.random.randint(1,10,(4,3)),index = [['a','a','b','b'],[1,2,1,2]],columns = [['one','one','two'],[
     ...: 'blue','red','blue']])

In [416]: df.index.names = ['row-1','row-2']

In [417]: df.columns.name = ['col-1','col-2']

In [418]: df
Out[418]:
             one      two
            blue red blue
row-1 row-2
a     1        5   3    9
      2        1   4    1
b     1        8   8    1
      2        2   8    9

• DataFrame多级索引的调用

 df.loc['a']
Out[419]:
       one      two
      blue red blue
row-2
1        5   3    9
2        1   4    1

In [420]: type(df.loc['a'])
Out[420]: pandas.core.frame.DataFrame

In [421]: df.loc['a',1]
Out[421]:
one  blue    5
     red     3
two  blue    9
Name: (a, 1), dtype: int32

In [422]: df.loc['a',1].index
Out[422]:
MultiIndex(levels=[['one', 'two'], ['blue', 'red']],
           codes=[[0, 0, 1], [0, 1, 0]])

• 索引的交换

In [423]: df2 = df.swaplevel('row-1','row-2')

In [424]: df2
Out[424]:
             one      two
            blue red blue
row-2 row-1
1     a        5   3    9
2     a        1   4    1
1     b        8   8    1
2     b        2   8    9

多级索引的统计

In [430]: df.sum(level=1)
Out[430]:
       one      two
      blue red blue
row-2
1       13  11   10
2        3  12   10

In [431]: df.sum(level=0)
Out[431]:
       one      two
      blue red blue
row-1
a        6   7   10
b       10  16   10

列数据和索引数据的转换

（1）将列数据转换成索引数据

 df = pd.DataFrame({
   'a':range(7),'b':range(7,0,-1),'c':['one','one','one','two','two','two','two'],'d':[0,1,2,0,1,2,3]}
     ...: )

In [433]: df
Out[433]:
   a  b    c  d
0  0  7  one  0
1  1  6  one  1
2  2  5  one  2
3  3  4  two  0
4  4  3  two  1
5  5  2  two  2
6  6  1  two  3

In [434]: df.set_index('c')
Out[434]:
     a  b  d
c
one  0  7  0
one  1  6  1
one  2  5  2
two  3  4  0
two  4  3  1
two  5  2  2
two  6  1  3

In [435]: df.set_index(['c','d'])
Out[435]:
       a  b
c   d
one 0  0  7
    1  1  6
    2  2  5
two 0  3  4
    1  4  3
    2  5  2
    3  6  1

（2）将索引转化回列数据

In [436]: df2 = df.set_index(['c','d'])

In [437]: df2
Out[437]:
       a  b
c   d
one 0  0  7
    1  1  6
    2  2  5
two 0  3  4
    1  4  3
    2  5  2
    3  6  1

In [438]: df2.reset_index()
Out[438]:
     c  d  a  b
0  one  0  0  7
1  one  1  1  6
2  one  2  2  5
3  two  0  3  4
4  two  1  4  3
5  two  2  5  2
6  two  3  6  1

In [440]: df.reset_index().sort_index('columns')
Out[440]:
   a  b    c  d  index
0  0  7  one  0      0
1  1  6  one  1      1
2  2  5  one  2      2
3  3  4  two  0      3
4  4  3  two  1      4
5  5  2  two  2      5
6  6  1  two  3      6

In [441]: df2.reset_index().sort_index('columns') == df
Out[441]:
      a     b     c     d
0  True  True  True  True
1  True  True  True  True
2  True  True  True  True
3  True  True  True  True
4  True  True  True  True
5  True  True  True  True
6  True  True  True  True

Pandas分组计算

分组计算三步曲：拆分→应用→合并
拆分：根据什么进行分组？
应用：每个分组进行什么样的计算？
合并：把每个分组的计算结果合并

通过列表进行分组

按行进行分组

（1）常规分组计算

df = pd.DataFrame({
   'key1':['a','a','b','b','a'],'key2':['one','two','one','two','one'],'data1':np.random.randint(1,10,
     ...: 5),'data2':np.random.randint(1,10,5)})

In [443]: df
Out[443]:
  key1 key2  data1  data2
0    a  one      4      6
1    a  two      7      2
2    b  one      1      3
3    b  two      4      1
4    a  one      6      6

In [444]: df['data1'].groupby(df['key1']).mean()
Out[444]:
key1
a    5.666667
b    2.500000
Name: data1, dtype: float64

In [449]: df.groupby('key1').sum()
Out[449]:
      data1  data2
key1
a        17     14
b         5      4
# 把不是数字的‘key2’组直接丢掉

In [450]: df.groupby('key1').sum()['data1']
Out[450]:
key1
a    17
b     5
Name: data1, dtype: int32

In [453]: mean = df.groupby(['key1','key2']).sum()['data1']

In [454]: mean
Out[454]:
key1  key2
a     one     10
      two      7
b     one      1
      two      4
Name: data1, dtype: int32

# 通过unstack转化成DafaFrame
In [455]: mean.unstack()
Out[455]:
key2  one  two
key1
a      10    7
b       1    4

（2）自定义分组键

In [445]: key = [1,2,1,1,2]

In [446]: df['data1'].groupby(key).mean()
Out[446]:
1    3.0
2    6.5
Name: data1, dtype: float64

（3）多层索引分组

In [447]: df['data1'].groupby([df['key1'],df['key2']]).sum()
Out[447]:
key1  key2
a     one     10
      two      7
b     one      1
      two      4
Name: data1, dtype: int32

In [448]: df['data1'].groupby([df['key1'],df['key2']]).size()
Out[448]:
key1  key2
a     one     2
      two     1
b     one     1
      two     1
Name: data1, dtype: int64

（4）使用groupby的迭代器协议

In [457]: for name, group in df.groupby('key1'):
     ...:     print (name)
     ...:     print (group)
     ...:
a
  key1 key2  data1  data2
0    a  one      4      6
1    a  two      7      2<