本文是对pandas的一个入门介绍,仅仅针对初学者。如果需要更详细的内容,请移步[Cookbook](http://pandas.pydata.org/pandas-docs/stable/cookbook.html#cookbook). 首先,导入所需要的python包:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
创建对象 ———– pandas中的数据结构包括Series、DataFrame、Panel、Pannel4D等,详细介绍移步[数据结构介绍](http://pandas.pydata.org/pandas-docs/stable/dsintro.html#dsintro). 常用的数据结构是前两个:Series和DataFrame。 通过传入一个已有的python列表(list)对象来创建一个Series对象。
s = pd.Series([1,3,4,np.nan,6,8])
s
0 1.0 1 3.0 2 4.0 3 NaN 4 6.0 5 8.0 dtype: float64 通过传入一个numpy数组来构建一个DataFrame对象。使用时间序列作为每行的索引,并为每列数据分配一个列名。
dates = pd.date_range('20130101', periods=6)
dates
DatetimeIndex([‘2013-01-01’, ‘2013-01-02’, ‘2013-01-03’, ‘2013-01-04’, ‘2013-01-05’, ‘2013-01-06’], dtype=’datetime64[ns]’, freq=’D’)
# 创建DataFrame对象,并指定索引index和列名columns
df = pd.DataFrame(np.random.randn(6,4), index=dates, columns=list('ABCD'))
df
A | B | C | D | |
---|---|---|---|---|
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
2013-01-04 | 1.447812 | -1.862121 | 0.115950 | -0.664134 |
2013-01-05 | 0.520409 | -1.402740 | -0.356049 | 0.460950 |
2013-01-06 | -0.404900 | 0.585420 | -0.073923 | -0.501197 |
通过传入一个python字典对象来创建一个DataFrame对象。
df2 = pd.DataFrame({
'A': 1.,
'B': pd.Timestamp('20160102'),
'C': pd.Series(1,index=list(range(4)),dtype='float32'),
'D': np.array([3]*4, dtype='int32'),
'E': pd.Categorical(['test','train','test','train']),
'F': 'foo'})
df2
A | B | C | D | E | F | |
---|---|---|---|---|---|---|
0 | 1.0 | 2016-01-02 | 1.0 | 3 | test | foo |
1 | 1.0 | 2016-01-02 | 1.0 | 3 | train | foo |
2 | 1.0 | 2016-01-02 | 1.0 | 3 | test | foo |
3 | 1.0 | 2016-01-02 | 1.0 | 3 | train | foo |
# DataFrame 中每列的数据类型可以不同
df2.dtypes
A float64
B datetime64[ns]
C float32
D int32
E category
F object
dtype: object
在ipython中可以使用“Tab”键对DataFrame的列名和公共属性进行自动补全。
查看对象中的数据
查看DataFrame的前几行或最后几行
df.head()
A | B | C | D | |
---|---|---|---|---|
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
2013-01-04 | 1.447812 | -1.862121 | 0.115950 | -0.664134 |
2013-01-05 | 0.520409 | -1.402740 | -0.356049 | 0.460950 |
df.tail()
A | B | C | D | |
---|---|---|---|---|
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
2013-01-04 | 1.447812 | -1.862121 | 0.115950 | -0.664134 |
2013-01-05 | 0.520409 | -1.402740 | -0.356049 | 0.460950 |
2013-01-06 | -0.404900 | 0.585420 | -0.073923 | -0.501197 |
df.head(3)
A | B | C | D | |
---|---|---|---|---|
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
获取DataFrame的索引、列名、数据(值)。
df.index
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')
df.columns
Index([u'A', u'B', u'C', u'D'], dtype='object')
df.values
array([[-0.28589413, 0.49001051, 0.17112101, -1.54980655],
[-0.06837701, -0.45280422, -0.39189213, -0.85252018],
[ 1.30438846, -1.80848416, -0.28648908, -0.43745725],
[ 1.44781215, -1.86212061, 0.11594994, -0.66413402],
[ 0.5204089 , -1.4027399 , -0.35604882, 0.4609499 ],
[-0.40489995, 0.58541997, -0.07392295, -0.5011969 ]])
使用“describe”获取数据的统计信息。
df.describe()
A | B | C | D | |
---|---|---|---|---|
count | 6.000000 | 6.000000 | 6.000000 | 6.000000 |
mean | 0.418906 | -0.741786 | -0.136880 | -0.590694 |
std | 0.808192 | 1.112849 | 0.244213 | 0.652884 |
min | -0.404900 | -1.862121 | -0.391892 | -1.549807 |
25% | -0.231515 | -1.707048 | -0.338659 | -0.805424 |
50% | 0.226016 | -0.927772 | -0.180206 | -0.582665 |
75% | 1.108394 | 0.254307 | 0.068482 | -0.453392 |
max | 1.447812 | 0.585420 | 0.171121 | 0.460950 |
将DataFrame进行转置。
df.T
2013-01-01 00:00:00 | 2013-01-02 00:00:00 | 2013-01-03 00:00:00 | 2013-01-04 00:00:00 | 2013-01-05 00:00:00 | 2013-01-06 00:00:00 | |
---|---|---|---|---|---|---|
A | -0.285894 | -0.068377 | 1.304388 | 1.447812 | 0.520409 | -0.404900 |
B | 0.490011 | -0.452804 | -1.808484 | -1.862121 | -1.402740 | 0.585420 |
C | 0.171121 | -0.391892 | -0.286489 | 0.115950 | -0.356049 | -0.073923 |
D | -1.549807 | -0.852520 | -0.437457 | -0.664134 | 0.460950 | -0.501197 |
df
A | B | C | D | |
---|---|---|---|---|
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
2013-01-04 | 1.447812 | -1.862121 | 0.115950 | -0.664134 |
2013-01-05 | 0.520409 | -1.402740 | -0.356049 | 0.460950 |
2013-01-06 | -0.404900 | 0.585420 | -0.073923 | -0.501197 |
对坐标轴进行排序。
df.sort_index(axis=1, ascending=False)
D | C | B | A | |
---|---|---|---|---|
2013-01-01 | -1.549807 | 0.171121 | 0.490011 | -0.285894 |
2013-01-02 | -0.852520 | -0.391892 | -0.452804 | -0.068377 |
2013-01-03 | -0.437457 | -0.286489 | -1.808484 | 1.304388 |
2013-01-04 | -0.664134 | 0.115950 | -1.862121 | 1.447812 |
2013-01-05 | 0.460950 | -0.356049 | -1.402740 | 0.520409 |
2013-01-06 | -0.501197 | -0.073923 | 0.585420 | -0.404900 |
df.sort_index(axis=0, ascending=False)
A | B | C | D | |
---|---|---|---|---|
2013-01-06 | -0.404900 | 0.585420 | -0.073923 | -0.501197 |
2013-01-05 | 0.520409 | -1.402740 | -0.356049 | 0.460950 |
2013-01-04 | 1.447812 | -1.862121 | 0.115950 | -0.664134 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
对值进行排序。
df.sort_values(by='B')
A | B | C | D | |
---|---|---|---|---|
2013-01-04 | 1.447812 | -1.862121 | 0.115950 | -0.664134 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
2013-01-05 | 0.520409 | -1.402740 | -0.356049 | 0.460950 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
2013-01-06 | -0.404900 | 0.585420 | -0.073923 | -0.501197 |
df.sort_values(by='B',ascending=False)
A | B | C | D | |
---|---|---|---|---|
2013-01-06 | -0.404900 | 0.585420 | -0.073923 | -0.501197 |
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-05 | 0.520409 | -1.402740 | -0.356049 | 0.460950 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
2013-01-04 | 1.447812 | -1.862121 | 0.115950 | -0.664134 |
选择数据
pandas中对数据的选择可以使用标准的python/numpy方式。
df['A']
2013-01-01 -0.285894
2013-01-02 -0.068377
2013-01-03 1.304388
2013-01-04 1.447812
2013-01-05 0.520409
2013-01-06 -0.404900
Freq: D, Name: A, dtype: float64
# 选择一个单独的列,将产生一个Series,此时df['A']等价于df.A
df.A
2013-01-01 -0.285894
2013-01-02 -0.068377
2013-01-03 1.304388
2013-01-04 1.447812
2013-01-05 0.520409
2013-01-06 -0.404900
Freq: D, Name: A, dtype: float64
对行进行切片操作。
df[0:3]
A | B | C | D | |
---|---|---|---|---|
2013-01-01 | -0.285894 | 0.490011 | 0.171121 | -1.549807 |
2013-01-02 | -0.068377 | -0.452804 | -0.391892 | -0.852520 |
2013-01-03 | 1.304388 | -1.808484 | -0.286489 | -0.437457 |
df['20130103':