python_时间序列
Time Series
import numpy as np
import pandas as pd
np.random.seed(12345)
import matplotlib.pyplot as plt
plt.rc('figure', figsize=(10, 6))
PREVIOUS_MAX_ROWS = pd.options.display.max_rows
pd.options.display.max_rows = 20
np.set_printoptions(precision=4, suppress=True)
Date and Time Data Types and Tools
from datetime import datetime
now = datetime.now()
now
now.year, now.month, now.day
(2019, 12, 1)
delta = datetime(2011, 1, 7) - datetime(2008, 6, 24, 8, 15)
delta
datetime.timedelta(926, 56700)
from datetime import timedelta
start = datetime(2011, 1, 7)
start + timedelta(12)
start - 2 * timedelta(12)
datetime.datetime(2010, 12, 14, 0, 0)
Converting Between String and Datetime
from datetime import datetime
stamp = datetime(2011, 1, 3)
str(stamp)
stamp.strftime('%Y-%m-%d')
'2011-01-03'
value = '2011-01-03'
datetime.strptime(value, '%Y-%m-%d')
datestrs = ['7/6/2011', '8/6/2011']
[datetime.strptime(x, '%m/%d/%Y') for x in datestrs]
[datetime.datetime(2011, 7, 6, 0, 0), datetime.datetime(2011, 8, 6, 0, 0)]
日期
from dateutil.parser import parse
parse('2011-01-03')
datetime.datetime(2011, 1, 3, 0, 0)
parse('Jan 31, 1997 10:45 PM')
datetime.datetime(1997, 1, 31, 22, 45)
parse('6/12/2011', dayfirst=True)
datetime.datetime(2011, 12, 6, 0, 0)
imort pandas as pd
import numpy as np
import pandas as pd
datestrs = ['2011-07-06 12:00:00', '2011-08-06 00:00:00']
pd.to_datetime(datestrs)
DatetimeIndex(['2011-07-06 12:00:00', '2011-08-06 00:00:00'], dtype='datetime64[ns]', freq=None)
idx = pd.to_datetime(datestrs + [None])
idx
idx[2]
pd.isnull(idx)
array([False, False, True])
Time Series Basics
11.2 时间序列基础
from datetime import datetime
dates = [datetime(2011, 1, 2), datetime(2011, 1, 5),
datetime(2011, 1, 7), datetime(2011, 1, 8),
datetime(2011, 1, 10), datetime(2011, 1, 12)]
ts = pd.Series(np.random.randn(6), index=dates)
ts
2011-01-02 -0.293891
2011-01-05 -1.792573
2011-01-07 0.598355
2011-01-08 -0.288192
2011-01-10 1.116760
2011-01-12 -0.155656
dtype: float64
ts.index
DatetimeIndex(['2011-01-02', '2011-01-05', '2011-01-07', '2011-01-08',
'2011-01-10', '2011-01-12'],
dtype='datetime64[ns]', freq=None)
ts + ts[::2]
2011-01-02 -0.587782
2011-01-05 NaN
2011-01-07 1.196709
2011-01-08 NaN
2011-01-10 2.233519
2011-01-12 NaN
dtype: float64
ts.index.dtype
dtype('<M8[ns]')
DatetimeIndex中的各个标量值是pandas的Timestamp对象:
stamp = ts.index[0]
stamp
Timestamp('2011-01-02 00:00:00')
Indexing, Selection, Subsetting
索引、选取、⼦集构造
stamp = ts.index[2]
ts[stamp]
0.5983546154952606
ts['1/10/2011']
ts['20110110']
1.1167596220313787
对于较⻓的时间序列,只需传⼊“年”或“年⽉”即可轻松选取数据
longer_ts = pd.Series(np.random.randn(1000),
index=pd.date_range('1/1/2000', periods=1000))
longer_ts
longer_ts['2001']
2001-01-01 0.409091
2001-01-02 0.960074
2001-01-03 -0.060883
2001-01-04 0.058963
2001-01-05 -0.468906
2001-01-06 -2.020290
2001-01-07 -1.545546
2001-01-08 -0.426660
2001-01-09 1.070679
2001-01-10 1.636438
2001-01-11 0.141087
2001-01-12 0.550284
2001-01-13 1.840379
2001-01-14 -1.453970
2001-01-15 0.650821
2001-01-16 -3.095478
2001-01-17 -1.446392
2001-01-18 1.351560
2001-01-19 -0.994964
2001-01-20 -0.458826
2001-01-21 -0.282168
2001-01-22 -0.210463
2001-01-23 0.139130
2001-01-24 -0.252216
2001-01-25 1.083073
2001-01-26 -0.447663
2001-01-27 0.012877
2001-01-28 1.472457
2001-01-29 0.901276
2001-01-30 0.567316
...
2001-12-02 -1.029440
2001-12-03 -1.196075
2001-12-04 -0.942923
2001-12-05 0.071254
2001-12-06 0.147439
2001-12-07 0.037061
2001-12-08 0.365074
2001-12-09 -1.503204
2001-12-10 0.106575
2001-12-11 -0.514702
2001-12-12 0.449840
2001-12-13 0.495410
2001-12-14 -1.118103
2001-12-15 -0.056152
2001-12-16 0.245808
2001-12-17 0.514600
2001-12-18 0.217560
2001-12-19 -0.672829
2001-12-20 -1.431404
2001-12-21 -3.110732
2001-12-22 2.158864
2001-12-23 0.481487
2001-12-24 0.340978
2001-12-25 2.411876
2001-12-26 -0.045341
2001-12-27 0.637324
2001-12-28 -1.656429
2001-12-29 -3.473388
2001-12-30 -0.531520
2001-12-31 1.439394
Freq: D, Length: 365, dtype: float64
这⾥,字符串“2001”被解释成年,并根据它选取时间区间。指定
longer_ts['2001-05']
2001-05-01 1.342106
2001-05-02 0.394368
2001-05-03 1.043601
2001-05-04 -0.053014
2001-05-05 0.143684
2001-05-06 -1.640853
2001-05-07 0.032346
2001-05-08 -0.297254
2001-05-09 0.448097
2001-05-10 1.131285
2001-05-11 -0.992064
2001-05-12 -0.282382
2001-05-13 0.032537
2001-05-14 0.571400
2001-05-15 1.532062
2001-05-16 0.510977
2001-05-17 0.564124
2001-05-18 -0.646513
2001-05-19 -0.107523
2001-05-20 0.467416
2001-05-21 0.134983
2001-05-22 0.755142
2001-05-23 -0.140014
2001-05-24 0.778222
2001-05-25 -1.531328
2001-05-26 0.732237
2001-05-27 -1.407104
2001-05-28 -0.259905
2001-05-29 -1.747983
2001-05-30 -0.193310
2001-05-31 1.423779
Freq: D, dtype: float64
datetime对象也可以进⾏切⽚:
ts[datetime(2011, 1, 7):]
2011-01-07 0.598355
2011-01-08 -0.288192
2011-01-10 1.116760
2011-01-12 -0.155656
dtype: float64
由于⼤部分时间序列数据都是按照时间先后排序的,因此你也可
ts
ts['1/6/2011':'1/11/2011']
2011-01-07 0.598355
2011-01-08 -0.288192
2011-01-10 1.116760
dtype: float64
ts.truncate(after='1/9/2011')
⾯这些操作对DataFrame也有效。例如,对DataFrame的⾏进⾏
dates = pd.date_range('1/1/2000', periods=100, freq='W-WED')
long_df = pd.DataFrame(np.random.randn(100, 4),
index=dates,
columns=['Colorado', 'Texas',
'New York', 'Ohio'])
long_df.loc['5-2001']
Colorado Texas New York Ohio
2001-05-02 1.297529 0.512483 1.067099 -1.464087
2001-05-09 0.618851 0.734287 0.278167 -2.307367
2001-05-16 0.353773 2.037013 1.267919 -1.833449
2001-05-23 1.645213 -0.877974 -0.211081 0.548765
2001-05-30 -0.063467 1.429438 1.025695 0.867788
Time Series with Duplicate Indices
带有重复索引的时间序列
dates = pd.DatetimeIndex(['1/1/2000', '1/2/2000', '1/2/2000',
'1/2/2000', '1/3/2000'])
dup_ts = pd.Series(np.arange(5), index=dates)
dup_ts
2000-01-01 0
2000-01-02 1
2000-01-02 2
2000-01-02 3
2000-01-03 4
dtype: int32
通过检查索引的is_unique属性,我们就可以知道它是不是唯⼀
dup_ts.index.is_unique
False
对这个时间序列进⾏索引,要么产⽣标量值,要么产⽣切⽚,具
dup_ts['1/3/2000']
dup_ts['1/2/2000']
2000-01-02 1
2000-01-02 2
2000-01-02 3
dtype: int32
假设你想要对具有⾮唯⼀时间戳的数据进⾏聚合。⼀个办法是使
grouped = dup_ts.groupby(level=0)
grouped.mean()
grouped.count()
2000-01-01 1
2000-01-02 3
2000-01-03 1
dtype: int64
Date Ranges, Frequencies, and Shifting
ts
resampler = ts.resample('D')
Generating Date Ranges
index = pd.date_range('2012-04-01', '2012-06-01')
index
DatetimeIndex(['2012-04-01', '2012-04-02', '2012-04-03', '2012-04-04',
'2012-04-05', '2012-04-06', '2012-04-07', '2012-04-08',
'2012-04-09', '2012-04-10', '2012-04-11', '2012-04-12',
'2012-04-13', '2012-04-14', '2012-04-15', '2012-04-16',
'2012-04-17', '2012-04-18', '2012-04-19', '2012-04-20',
'2012-04-21', '2012-04-22', '2012-04-23', '2012-04-24',
'2012-04-25', '2012-04-26', '2012-04-27', '2012-04-28',
'2012-04-29', '2012-04-30', '2012-05-01', '2012-05-02',
'2012-05-03', '2012-05-04', '2012-05-05', '2012-05-06',
'2012-05-07', '2012-05-08', '2012-05-09', '2012-05-10',
'2012-05-11', '2012-05-12', '2012-05-13', '2012-05-14',
'2012-05-15', '2012-05-16', '2012-05-17', '2012-05-18',
'2012-05-19', '2012-05-20', '2012-05-21', '2012-05-22',
'2012-05-23', '2012-05-24', '2012-05-25', '2012-05-26',
'2012-05-27', '2012-05-28', '2012-05-29', '2012-05-30',
'2012-05-31', '2012-06-01'],
dtype='datetime64[ns]', freq='D')
pd.date_range(start='2012-04-01', periods=20)
pd.date_range(end='2012-06-01', periods=20)
DatetimeIndex(['2012-05-13', '2012-05-14', '2012-05-15', '2012-05-16',
'2012-05-17', '2012-05-18', '2012-05-19', '2012-05-20',
'2012-05-21', '2012-05-22', '2012-05-23', '2012-05-24',
'2012-05-25', '2012-05-26', '2012-05-27', '2012-05-28',
'2012-05-29', '2012-05-30', '2012-05-31', '2012-06-01'],
dtype='datetime64[ns]', freq='D')
pd.date_range('2000-01-01', '2000-12-01', freq='BM')
DatetimeIndex(['2000-01-31', '2000-02-29', '2000-03-31', '2000-04-28',
'2000-05-31', '2000-06-30', '2000-07-31', '2000-08-31',
'2000-09-29', '2000-10-31', '2000-11-30'],
dtype='datetime64[ns]', freq='BM')
date_range默认会保留起始和结束时间戳的时间信息(如果有的
pd.date_range('2012-05-02 12:56:31', periods=5)
DatetimeIndex(['2012-05-02 12:56:31', '2012-05-03 12:56:31',
'2012-05-04 12:56:31', '2012-05-05 12:56:31',
'2012-05-06 12:56:31'],
dtype='datetime64[ns]', freq='D')
Frequencies and Date Offsets
from pandas.tseries.offsets import Hour, Minute
hour = Hour()
hour
<Hour>
传⼊⼀个整数即可定义偏移量的倍数:
four_hours = Hour(4)
four_hours
<4 * Hours>
pd.date_range('2000-01-01', '2000-01-03 23:59', freq='4h')
DatetimeIndex(['2000-01-01 00:00:00', '2000-01-01 04:00:00',
'2000-01-01 08:00:00', '2000-01-01 12:00:00',
'2000-01-01 16:00:00', '2000-01-01 20:00:00',
'2000-01-02 00:00:00', '2000-01-02 04:00:00',
'2000-01-02 08:00:00', '2000-01-02 12:00:00',
'2000-01-02 16:00:00', '2000-01-02 20:00:00',
'2000-01-03 00:00:00', '2000-01-03 04:00:00',
'2000-01-03 08:00:00', '2000-01-03 12:00:00',
'2000-01-03 16:00:00', '2000-01-03 20:00:00'],
dtype='datetime64[ns]', freq='4H')
Hour(2) + Minute(30)
<150 * Minutes>
同理,你也可以传⼊频率字符串(如"2h30min"),这种字符串
pd.date_range('2000-01-01', periods=10, freq='1h30min')
DatetimeIndex(['2000-01-01 00:00:00', '2000-01-01 01:30:00',
'2000-01-01 03:00:00', '2000-01-01 04:30:00',
'2000-01-01 06:00:00', '2000-01-01 07:30:00',
'2000-01-01 09:00:00', '2000-01-01 10:30:00',
'2000-01-01 12:00:00', '2000-01-01 13:30:00'],
dtype='datetime64[ns]', freq='90T')
rng = pd.date_range('2012-01-01', '2012-09-01', freq='WOM-3FRI')
list(rng)
[Timestamp('2012-01-20 00:00:00', freq='WOM-3FRI'),
Timestamp('2012-02-17 00:00:00', freq='WOM-3FRI'),
Timestamp('2012-03-16 00:00:00', freq='WOM-3FRI'),
Timestamp('2012-04-20 00:00:00', freq='WOM-3FRI'),
Timestamp('2012-05-18 00:00:00', freq='WOM-3FRI'),
Timestamp('2012-06-15 00:00:00', freq='WOM-3FRI'),
Timestamp('2012-07-20 00:00:00', freq='WOM-3FRI'),
Timestamp('2012-08-17 00:00:00', freq='WOM-3FRI')]
Shifting (Leading and Lagging) Data
移动(超前和滞后)数据
ts = pd.Series(np.random.randn(4),
index=pd.date_range('1/1/2000', periods=4, freq='M'))
ts
ts.shift(2)
ts.shift(-2)
2000-01-31 0.303604
2000-02-29 -1.211751
2000-03-31 NaN
2000-04-30 NaN
Freq: M, dtype: float64
ts / ts.shift(1) - 1
ts.shift(2, freq='M')
这⾥还可以使⽤其他频率,于是你就能⾮常灵活地对数据进⾏超
ts.shift(3, freq='D')
ts.shift(1, freq='90T')
2000-01-31 01:30:00 0.721757
2000-02-29 01:30:00 -0.386397
2000-03-31 01:30:00 0.303604
2000-04-30 01:30:00 -1.211751
Freq: M, dtype: float64
Shifting dates with offsets
from pandas.tseries.offsets import Day, MonthEnd
now = datetime(2011, 11, 17)
now + 3 * Day()
Timestamp('2011-11-20 00:00:00')
如果加的是锚点偏移量(⽐如MonthEnd),第⼀次增量会将原
now + MonthEnd()
now + MonthEnd(2)
Timestamp('2011-12-31 00:00:00')
通过锚点偏移量的rollforward和rollback⽅法,可明确地将⽇期向
offset = MonthEnd()
offset.rollforward(now)
offset.rollback(now)
Timestamp('2011-10-31 00:00:00')
⽇期偏移量还有⼀个巧妙的⽤法,即结合groupby使⽤这两个“滚 动”⽅法:
ts = pd.Series(np.random.randn(20),
index=pd.date_range('1/15/2000', periods=20, freq='4d'))
ts
ts.groupby(offset.rollforward).mean()
2000-01-31 -0.116890
2000-02-29 0.065760
2000-03-31 -0.323985
dtype: float64
ts.resample('M').mean()