【论文阅读】HyperST-Net:HypernetworksforSpatio-TemporalForecasting

摘要

HyperST-Net包含三个主要模块，空间模块、时间模块和推导模块deduction module

空间模块可以得到空间特征，推导模块通过空间特征得到时间模块的权重。

原理

强调考虑和时间模块和空间模块之间的关系

1  spatial characteristics of an ST object are determined by its spatial attributes, like POIs and road networks.

2  considering the above causality, we deduce temporal characteristics from spatial characteristics, as shown in Figure 3.

3  the predicted values are obtained from the temporal characteristics, based on temporal information (e.g., previous readings and weather).
 

模型架构

Spatial module

1  spatial attributes are embedded into a low dimensional vector

2  it generates a series of factors (parallelograms in the green rectangle) independently, and then use them to model parameter weights of the corresponding neural network (NN) layer in the temporal module by the deduction module

Temporal module

the parameter weights of HyperST layers are computed by the deduction module

can be regarded as the temporal characteristics of the objects, which determine the future values based on temporal information.

Deduction module

 bridges the spatial and temporal modules by applying a deduction process for the parameter weights, such that the intrinsic causality between spatial and temporal characteristics is well considered

总的来说spatial attributes导致temporal modules有不同的权重值；Hyperst网络相当于根据相应对象的空间属性自动设计n个时间模型

 it can be seen as a relaxed-form of weight sharing across objects

 

Methodologies

General HyperSTLayer

fk：the k-th layer in the temporal network

θk：a set of parameters , can be modeled by the spatial network gk by using a set of parameters ωk

ωk:  be trained by back-propagation and gradient descent

spatial module first embeds spatial attributes into a vector with the dimension equals to the number of parameters in θk

deduction module reshapes the vector to a tensor with the shape specified by θk

the tensor, i.e., the output of HyperST layer, is used as the parameters of the NN layer in the temporal module

 

 the loss function l(f(si,Ti;θ),Li) =

the temporal information (e.g., historical readings and weather) of the i-th object in a period of M timestamps,where each row isa DT-dimensional vector

Predicted values. where row j is a DL dimensional vector, which indicates the readings of object i at time slot j

 

HyperST-Dense（用于减少参数数量）

input and output of the dense layer , W is  parameters

we employ the spatial module to generate a weight scaling vector ,and the deduction module to scale rows of a weight matrix W' W' is learnable （in this way the number of parameters can be easily controlled）

We extend the dense layer in the RNN cell to the HyperST-Dense layer

HyperST-Conv(可能相当于多维扩展)

the input of the convolution operator is a tensor Xin with Nin channels, while the output is Xout with Nout channels

the spatial module first embeds the spatial attributes into a weight vector z ∈ RNout

the deduction module, the kernel of the convolution operator can be expressed as W = diag(z)·W‘  W‘ is a learnable tensor

 the convolution operation can be expressed as

结果