Dynamic Memory based Attention Network for Sequential Recommendation

将用户的历史行为序列分割成长度为T的连续的连续的若干个子序列：

S = { x 1 , x 2 , . . . , x ∣ S ∣ } = { S n } n = 1 N S = \{x_1,x_2,...,x_{|S|}\} = \{S_n\}_{n=1}^N S={x1​,x2​,...,x∣S∣​}={Sn​}n=1N​

其中 S n = { x n , 1 , x n , 2 , . . . , x x , T } S_n = \{x_{n,1},x_{n,2},...,x_{x,T}\} Sn​={xn,1​,xn,2​,...,xx,T​} 表示第 n 个序列，T表示子序列的长度。

短期兴趣生成网络

${\tilde{H}}_{n-1}^l\in R^{T\times D}$ 表示序列$S_{n-1}$的第$l$层隐状态，网络结构为：

$${\tilde{H}}_n^l=Atte{n}_{rec}^l({\tilde{Q}}_n^l,{\tilde{K}}_n^l,{\tilde{V}}_n^l)=softmax({\tilde{Q}}_n^l({\tilde{K}}_n^l{)}^T){\tilde{V}}_n^l$$

$${\tilde{Q}}_n^l={\tilde{H}}_n^{l-1}{\tilde{W}}_Q^T$$

$${\tilde{K}}_n^l=H_n^{l-1}{\tilde{W}}_K^T$$

$${\tilde{V}}_n^l=H_n^{l-1}{\tilde{W}}_V^T$$

$$H_n^{l-1}={\tilde{H}}_n^{l-1}||StopGradient({\tilde{H}}_{n-1}^{l-1})$$

H ~ n 0 = X n = { x n , 1 , x n , 2 , . . . , x x , T } ∈ R T × D \tilde{H}_n^0 = X_n = \{x_{n,1},x_{n,2},...,x_{x,T}\}\in R^{T\times D} H~n0​=Xn​={xn,1​,xn,2​,...,xx,T​}∈RT×D

$${\tilde{H}}_n={\tilde{H}}_n^L$$

其中$||$表示concat操作，${\tilde{W}}_Q,{\tilde{W}}_K,{\tilde{W}}_V\in R^{D\times D}$ 表示模型参数。

长期兴趣生成网络

$$M^l\in R^{m\times D}$$

m表示memory slot的个数，$M^l$表示$l$层memory 矩阵。

$${\hat{H}}_n^l=Atte{n}^l({\hat{Q}}_n^l,{\hat{K}}_n^l,{\hat{V}}_n^l)$$

$${\hat{Q}}_n^l,{\hat{K}}_n^l,{\hat{V}}_n^l={\hat{H}}_n^{l-1}{\hat{W}}_Q^T,M^{l-1}{\hat{W}}_K^T,M^{l-1}{\hat{W}}_Q^T$$

$${\hat{H}}_n={\hat{H}}_n^L$$

${\hat{W}}_Q,{\hat{W}}_K,{\hat{W}}_V$表示模型参数。

兴趣融合网络

$$V_n=G_n\odot {\tilde{H}}_n+(1-G_n)\odot {\hat{H}}_n$$

$$G_n=\sigma ({\tilde{H}}_n{W}_{short}+{\hat{H}}_n{W}_{long})\in R^{T\times D}$$

其中$\odot$表示逐位乘法，$W_{short},W_{long}\in R^{D\times D}$表示模型参数.

长期兴趣更新网络

$$M^l\leftarrow f_{abs}^l(M^l,{\tilde{H}}_{n-1}^l)$$

$f_{abs}^l:R^{(m+T)\times D}\to R^{m\times D}$

$f_{abs}^l$的通过胶囊网络实现：

$$b_{ij}={\overline{x}}_j{W}_{ij}{x}_i$$

$${\alpha }_{ij}=exp(b_{ij})/\sum _{j^{\prime }=1}^{m+T}exp(b_{i{j}^{\prime }})$$

$$s_j=\sum _{i=1}^{m+T}{\alpha }_{ij}{W}_{ij}{x}_i$$

$${\overline{x}}_j=squash(s_j)=\frac{||s_j|{|}^2}{1+||s_j|{|}^2}\frac{s_j}{||s_j||}$$

$$M=[{\overline{x}}_1,{\overline{x}}_2,...,{\overline{x}}_m]$$

$f_{abs}^l$辅助训练loss：

$${\mathcal{L}}_{ae}=\sum _{l=1}^L||atten{t}_{rec}^l({\tilde{Q}}^l,{\tilde{K}}^l,\tilde{V}{)}^l-atten{t}_{rec}^l({\tilde{Q}}^l,{\hat{K}}^l,{\hat{V}}^l)|{|}_F^2$$

$${\tilde{Q}}^l={\tilde{H}}_n^l,{\tilde{K}}_n^l={\tilde{V}}_n^l=M^l||{\tilde{H}}_{n-1}^l,{\hat{K}}^l={\hat{V}}^l=M^l$$

loss 函数

$${\mathcal{L}}_{like}=-\sum _{u\in U}\sum _{t\in S_n}log\frac{exp(x_t^T{V}_{}n,t)}{{\sum }_{j\in V}exp(x_j^T{V}_{n,t})}$$

$$\mathcal{L}={\mathcal{L}}_{like}+{\mathcal{L}}_{ae}$$