GNN论文阅读笔记

General Idea

GNN, as one of the Message Passing Neural Networks(MPNN), generally aggregates the features from the neighborhood of nodes in each layer.

1. Structure-awareness GNN

因为GNN的aggregator通常是permutation-invariant的，比如sum、mean、max等，这导致了GNN无法区分某些non-isomorphic graph。而structure-awareness类方法，就是通过强调structure的概念，

1.1 GEOM-GCN — GEOMETRIC GRAPH CONVOLUTIONAL NETWORKS(ICLR 2020)

3 modules:

• Node Embedding: $v\to z_v$
  • Implementation: Isomap \ Poincare \ Stru2vec
• Structural Neighborhood：
  • $N(v)$ = {neighbordhood in graph $N_g(v)$, neighbordhood in latent space $N_s(v)$, relational op $\tau$}
  • $N_s(v)$: $d(z_v,z_u)<\rho$
  • $\tau$: function，输入($z_v$,$z_u$)，输出他们的geometric relationship r，每个node在latent space中相对于target node会有一一对应的geometric relationship
• Bi-level Aggregation:
  • Low-level Aggregation:
    • 把neighborhood(graph/latent space)和geo relationship都相同的node通过类似GCN的计算aggregate到virtual node e
  • High-level Aggregation:
    • 把各个virtual node的feature concat起来

2. Entity Alignment(KG) + GNN

• https://paperswithcode.com/task/entity-alignment
• Idea: Match the same entity in different knowledge graph
• Challenge: Different KG often have different neighbor structure for the same entity
  

2.1 AliNet — Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation

• 由于不同KG对应entity的local structure不一样，所以导致不同KG同样的entity的representation会有差异
• 但是，就算KG不同，跟同一entity有关的东西终究就是那些，只不过有的会从1-hop neighbor变成 2-hop甚至k-hop
• 因此，AliNet在对1-hop neighbor运用GCN的基础上，对2-hop运用attention机制，如下图
  

2.2 MuGNN — Multi-Channel Graph Neural Network for Entity Alignment

• 不同KG有2种结构差异：
  • the missing relations due to the incompleteness nature of KG
  • the exclusive entities caused by different construction demands of applications or languages.
    
• KG Completion（把KG漏掉的link给补上）
  • rule inference: 先通过AMIE+去分别mine rule
  • rule transfer: 对于图G得到的rules set，如果某个rule k中的每个relation都和G’中的某个relation能够align，就把k中的每个relation都进行替换构成k’加入到G’的rules set中
  • rule grounding: 根据之前得到的每个图自己的rules set，然后对每个rule，看看自己图中有哪些premise在图中，而conclusion不在的情况，就对应的把那些relation(triplets)补上
• MuGNN
  • Relation weighting
    • 2 adjacency matrices for each channel
    • KG self-attention（看neighborhood里面哪个比较重要） -> A1
    • KG cross-attention（把多余relation的weight置零） -> A2
  • Multi-channel GNN encoder
    • 这里channel应该理解为branch——每个branch有一层shared weight的GCN，不同branch功能不同
    • 本文channel=2——A1、A2
  • Align model
    • 在2个图直接根据seed alignment为entity和relation分别构造pair
    • 让positive pair的距离更小，negative pair的距离更大
    • 每5epoch重新构造negative pair，选最难的（最相似的）
    • 对triplets和rule grounds也构造类似的pair和loss