【论文笔记】KGAT：Knowledge Graph Attention Network for Recommendation

KGAT：Knowledge Graph Attention Network for Recommendation

Paper address：Knowledge Graph Attention Network for Recommendation

Contribution

提出了将具有辅助信息的实体融入图谱的CKG，引入注意力机制完成预测

Method

分为Embedding layer，Attentive Embedding layer和Prediction layer三个主要结构

• Embedding layer
  利用TransR，目标是给定三元组$(h,r,t)$满足${\mathbf{e}}_h^r+{\mathbf{e}}_r\approx {\mathbf{e}}_t^r$的假设，具体表达为${\mathcal{L}}_{\mathrm{KG}}={\sum }_{(h,r,t,t^{\prime })\in \mathcal{T}}-\ln \sigma (g(h,r,t^{\prime })-g(h,r,t))$其中$\sigma$为sigmoid函数，$g(h,r,t)=\Vert {\mathbf{W}}_r{\mathbf{e}}_h+{\mathbf{e}}_r-{\mathbf{W}}_r{\mathbf{e}}_t{\Vert }_2^2$,$g(h,r,t)$越小则越符合上述假设即更真实，反之亦然；$t^{\prime }$为随机任取实体

• Attentive layer
  包含三个组件information propagation, knowledge-aware attention和information aggregation

  • information propagation

    ${\mathbf{e}}_{{\mathcal{N}}_h}={\sum }_{(h,r,t)\in {\mathcal{N}}_h}\pi (h,r,t){\mathbf{e}}_t$表示所有以$h$为起始的三元组传播表示，$\pi (h,r,t)$表示信息传播系数

  • knowledge-aware attention

    $\pi (h,r,t)=({\mathbf{W}}_r{\mathbf{e}}_t{)}^{\mathsf{T}}\tanh (({\mathbf{W}}_r{\mathbf{e}}_h+{\mathbf{e}}_r))$，直观上，相似的节点之间信息交换更多，之后在以$h$为首的组内做归一化，表示为$\pi (h,r,t)=\frac{\exp (\pi (h,r,t))}{{\sum }_{(h,r^{\prime },t^{\prime })\in {\mathcal{N}}_h}\exp (\pi (h,r^{\prime },t^{\prime }))}$

  • information aggregation

    信息聚合表示为${\mathbf{e}}_h^{(1)}=f({\mathbf{e}}_h,{\mathbf{e}}_{{\mathcal{N}}_h})$

    分为三种聚合方法：GCN,GraphSAGE和Hybrid

    GCN aggregate：$f_{\mathrm{GCN}}=\text{LeakyReLU}\left(\mathbf{W}({\mathbf{e}}_h+{\mathbf{e}}_{{\mathcal{N}}_h})\right)$

    GraphSAGE aggregate：$f_{\text{GraphSage }}=\text{LeakyReLU}\left(\mathbf{W}({\mathbf{e}}_h||{\mathbf{e}}_{{\mathcal{N}}_h})\right)$

    Hybrid aggregate：$\begin{array}{rl}{f}_{\text{Bi-Interaction }}=& \text{LeakyReLU}({\mathbf{W}}_1({\mathbf{e}}_h+{\mathbf{e}}_{{\mathcal{N}}_h}))+\text{LeakyReLU}({\mathbf{W}}_2({\mathbf{e}}_h\odot {\mathbf{e}}_{{\mathcal{N}}_h}))\end{array}$

    其中$W(\cdot )$为点乘可学习矩阵，$\mathbf{W}\in {\mathbb{R}}^{d^{\prime }\times d}$，$d^{\prime }$为变换维度，$\odot$为哈达玛积

• Prediction layer

  经过L层Attentive layer获得User Embedding和Item Embedding各L+1个，评分$\hat{y}(u,i)={\mathbf{e}}_u^{\ast \top }{\mathbf{e}}_i^{\ast }$其中

  ||为concat操作

  ${\mathbf{e}}_u^{\ast }={\mathbf{e}}_u^{(0)}\Vert \cdots \Vert {\mathbf{e}}_u^{(L)},{\mathbf{e}}_i^{\ast }={\mathbf{e}}_i^{(0)}\Vert \cdots \Vert {\mathbf{e}}_i^{(L)}$

optimization

${\mathcal{L}}_{\mathrm{CF}}={\sum }_{(u,i,j)\in O}-\ln \sigma \left(\hat{y}(u,i)-\hat{y}(u,j)\right)$其中$O=\{(u,i,j)|(u,i)\in {\mathbb{R}}^{+},(u,j)\in {\mathbb{R}}^{-}\}$分别对应已交互和未交互集

总损失为${\mathcal{L}}_{\mathrm{KGAT}}={\mathcal{L}}_{\mathrm{KG}}+{\mathcal{L}}_{\mathrm{CF}}+\lambda {\Vert \Theta \Vert }_2^2$其中 Θ = { E , W r , ∀ l ∈ R , W 1 ( l ) , W 2 ( l ) , ∀ l ∈ { 1 , ⋯   , L } } \begin{aligned}\Theta=\{\mathrm{E},\mathbf{W}_r,\forall l\in\mathcal{R},\mathbf{W}_1^{(l)},\mathbf{W}_2^{(l)},\forall l\in\{1,\cdots,L\}\}\end{aligned} Θ={E,Wr​,∀l∈R,W1(l)​,W2(l)​,∀l∈{1,⋯,L}}​目的是防止过拟合

Dataset

Experiment