Structural Time Series modeling in TensorFlow Probability

最新推荐文章于 2022-11-16 15:20:22 发布

sphw

最新推荐文章于 2022-11-16 15:20:22 发布

阅读量459

点赞数

分类专栏： machine learning

本文链接：https://blog.csdn.net/zxxr123/article/details/104113364

版权

machine learning 专栏收录该内容

9 篇文章 1 订阅

订阅专栏

在邯郸学步后，想要深入用好Tensorflow中的STS model，还是要静下心来，好好阅读点材料。

$f_1(t) + f_2(t) + ... + f_n(t) + \varepsilon; \varepsilon \sim N(0, \sigma^2)$

1、时间序列模型的用处，除了实现预则立之外，还可以有以下3方面作用：

推断某一因素的因果影响，
通过容易获取的信息推断出不容易获取信息或指标当前值，
探测时间序列数据中的异常值。
Methods for forecasting time series can also be applied to infer the causal impact of a feature launch or other intervention on user engagement metrics, to infer the current value of difficult-to-observe quantities like the unemployment rate from more readily available information, as well as to detect anomalies in time series data.

2、STS的核心是Variation Inference and Hamiltonian Monte Carlo，可以实现单点和不确定性的预测，可以利用GPUs实现平行计算及与深度神经网络的整合。
TensorFlow Probability (TFP) now features built-in support for fitting and forecasting using structural time series models. This support includes Bayesian inference of model parameters using variational inference (VI) and Hamiltonian Monte Carlo (HMC), computing both point forecasts and predictive uncertainties. Because they’re built in TensorFlow, these methods naturally take advantage of vectorized hardware (GPUs and TPUs), can efficiently process many time series in parallel, and can be integrated with deep neural networks.

3、95% 预测置信区间。

We can see that the forecast uncertainty (shading ± 2 standard deviations) increases over time, as the linear trend model becomes less confident in its extrapolation of the slope. The mean forecast combines the seasonal variational with a linear extrapolation of the existing trend, which appears to slightly underestimate the accelerating growth in atmospheric CO2, but the true values are still within the 95% predictive interval.

4、分解各部分的作用
在不太理解算法内部原理的情况下，猜测从下图各部分中预测置信区间的大小看，hour-of-day 、temperature在电量预测模型中起到的比较重要的作用；day-of-week、autogressive次之，其中autogressive贡献了大部分的预测不确定性。
在这里插入图片描述
We see that the model has quite reasonably identified a large hour-of-day effect and a much smaller day-of-week effect (the lowest demand appears to occur on Saturdays and Sundays), as well as a sizable effect from temperature, and that it produces relatively confident forecasts of these effects. Most of the predictive uncertainty comes from the autoregressive process, based on its estimate of the unmodeled (residual) variation in the observed series.

5、LinearRegression除了可以处理相关时间序列数据外，也可以处理节假日和特定日期。
For time series depending on additional, time-varying covariates. Regression components can also be used to encode holiday or other date-specific effects.

As a more advanced application, we might use the design matrix to encode holiday effects. For example, suppose we are modeling data from the month of December. We can combine day-of-week seasonality with special effects for Christmas Eve (Dec 24), Christmas (Dec 25), and New Year’s Eve (Dec 31), by constructing a design matrix with indicators for those dates. Source

holiday_indicators = np.zeros([31, 3])
holiday_indicators[23, 0] = 1  # Christmas Eve
holiday_indicators[24, 1] = 1  # Christmas Day
holiday_indicators[30, 2] = 1  # New Year's Eve

holidays = tfp.sts.LinearRegression(design_matrix=holiday_indicators,
                                    name='holidays')
day_of_week = tfp.sts.Seasonal(num_seasons=7,
                               observed_time_series=observed_time_series,
                               name='day_of_week')
model = tfp.sts.Sum(components=[holidays, seasonal],
                    observed_time_series=observed_time_series)

6、Seasonal要素. For time series depending on seasonal factors, such as the hour of the day, the day of the week, or the month of the year.

day_of_week、hour_of_day的处理方式:

day_of_week = tfp.sts.Seasonal(num_seasons=7,
                               num_steps_per_season=24,
                               observed_time_series=y,
                               name='day_of_week')
hour_of_day = tfp.sts.Seasonal(num_seasons=24,
                               num_steps_per_season=1,
                               observed_time_series=y,
                               name='hour_of_day')
model = tfp.sts.Sum(components=[day_of_week, hour_of_day],
                    observed_time_series=y)

month_of_year的处理方式:

month_of_year = tfp.sts.Seasonal(
  num_seasons=12,
  num_steps_per_season=[31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31],
  drift_scale_prior=tfd.LogNormal(loc=-1., scale=0.1),
  initial_effect_prior=tfd.Normal(loc=0., scale=5.),
  name='month_of_year')

对于4年（包含闰月）的month_of_year处理方式类似如下，这一方式可用于处理农历.

num_days_per_month = np.array(
  [[31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31],
   [31, 29, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31],  # year with leap day
   [31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31],
   [31, 28, 31, 30, 30, 31, 31, 31, 30, 31, 30, 31]])

month_of_year = tfp.sts.Seasonal(
  num_seasons=12,
  num_steps_per_season=num_days_per_month,
  drift_scale_prior=tfd.LogNormal(loc=-1., scale=0.1),
  initial_effect_prior=tfd.Normal(loc=0., scale=5.),
  name='month_of_year')

7、Autoregressive

8、LocalLinearTrend

9、SemiLocalLinearTread

10、LocalLevel

1. How to make the electricity demand model better?

为什么说在观察到温度与AR的spikes基本一致，就表明addtional featues或data transformations有助于更好地捕捉到温度feature的作用？
A modeler might use this decomposition to understand how to improve the model. For example, they might notice that some spikes in temperature still seem to coincide with spikes in the AR residual, indicating that additional features or data transformations might help better capture the temperature effect.

关于这个问题，有人提出了类似的问题:
It was mentioned that since peaks in the residual correspond to peaks in temperature other features could be engineered to capture this - are you referring to say polynomial expansions of temperature? any other hints? 针对这个问题，Dave Moore的回复如下：
I think we were imagining polynomial expansions, though you could also consider other transformations like log, or one-hot vectors from quantizing temperature into bins, which at a fine enough discretization can represent arbitrary nonlinear functions. Or to be really fancy you could consider interactions between these (e.g., instead of a piecewise-constant response from quantized features, you’d use the one-hot bins as a masking pattern on the original series to get a piecewise-linear response, or on polynomial features to get a piecewise-polynomial response, etc).

1.1 How to do polynomial expansions for time series data？

模型中添加平方、立方和乘积等非线性features，往往是比较有用的。
Often it’s useful to add complexity to the model by considering nonlinear features of the input data. A simple and common method to use is polynomial features, which can get features’ high-order and interaction terms. Source

The way of creating polynominal features:

$X_1$	$X_2$
0	1
2	3
4	5

>>> poly = PolynomialFeatures(degree = 2)
>>> poly.fit_transform(X)

1	$X_1$	$X_2$	$X_1^2$	$X_1X_2$	$X_2^2$
1	0	1	$0^2$	$\times 1$	$1^2$
1	2	3	$2^2$	$\times 3$	$3^2$
1	4	5	$4^2$	$\times 5$	$5^2$

>>> X = np.arange(6).reshape(3, 2)
>>> X
array([[0, 1],
       [2, 3],
       [4, 5]])
>>> poly = PolynomialFeatures(2)
>>> poly.fit_transform(X)
array([[ 1.,  0.,  1.,  0.,  0.,  1.],
       [ 1.,  2.,  3.,  4.,  6.,  9.],
       [ 1.,  4.,  5., 16., 20., 25.]])

Polynomial expansions for time series:
Refer to the Polynomial Regression and Generating polynomial features, the polynomial expansions for time series maybe like the following code:

>>> from sklearn.preprocessing import PolynomialFeatures
>>> import numpy as np

>>> x = np.arange(5)
>>> x = x[:, np.newaxis]
>>> x
array([[0],
       [1],
       [2],
       [3],
       [4]])
>>> polynomial_features= PolynomialFeatures(degree=2)
>>> x_poly = polynomial_features.fit_transform(x)
>>> x_poly
array([[ 1.,  0.,  0.],
       [ 1.,  1.,  1.],
       [ 1.,  2.,  4.],
       [ 1.,  3.,  9.],
       [ 1.,  4., 16.]])
>>> polynomial_features= PolynomialFeatures(degree=3)
>>> x_poly = polynomial_features.fit_transform(x)
>>> x_poly
array([[ 1.,  0.,  0.,  0.],
       [ 1.,  1.,  1.,  1.],
       [ 1.,  2.,  4.,  8.],
       [ 1.,  3.,  9., 27.],
       [ 1.,  4., 16., 64.]])

1.2 Try to make the model better

Add $l o g (t e m p e r a t u r e)$ transformation to build_model.

temperature_log_effect = sts.LinearRegression(
      design_matrix=tf.reshape(np.log(temperature),
                               (-1, 1)), name='temperature_log_effect')

def build_model(observed_time_series):
  hour_of_day_effect = sts.Seasonal(
      num_seasons=24,
      observed_time_series=observed_time_series,
      name='hour_of_day_effect')
  day_of_week_effect = sts.Seasonal(
      num_seasons=7, num_steps_per_season=24,
      observed_time_series=observed_time_series,
      name='day_of_week_effect')
  temperature_effect = sts.LinearRegression(
      design_matrix=tf.reshape(temperature - np.mean(temperature),
                               (-1, 1)), name='temperature_effect')
  # added feature
  temperature_log_effect = sts.LinearRegression(
      design_matrix=tf.reshape(np.log(temperature),
                               (-1, 1)), name='temperature_log_effect')

  autoregressive = sts.Autoregressive(
      order=1,
      observed_time_series=observed_time_series,
      name='autoregressive')
  model = sts.Sum([hour_of_day_effect,
                   day_of_week_effect,
                   temperature_effect,
                   # added feature
                   temperature_log_effect,
                   autoregressive],
                   observed_time_series=observed_time_series)
  return model

模型预测精度有了较大改善，如下图所示：
官网model的预测结果：
only temperature

添加了 $l o g (t e m p e r a t u r e)$ 后model的预测结果：
temperature + log(temperature)

sphw

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
Structural Time Series modeling in TensorFlow Probability

在邯郸学步后，想要深入用好Tensorflow中的STS model，还是要静下心来，好好阅读点材料。f(t)=f1(t)+f2(t)+...+fn(t)+ε;ε∼N(0,σ2)f(t) = f_1(t) + f_2(t) + ... + f_n(t) + \varepsilon; \varepsilon \sim N(0, \sigma^2)f(t)=f1(t)+f2(t)+...+f...
复制链接

扫一扫

专栏目录