基于Tensorflow2.0实现图神经网络中Message Passing

基于Tensorflow2.0实现图神经网络中Message Passing

近年来图神经网络越来越受到科研领域的关注，而众多的图圣经网络都是通过message passing方法来更新网络中节点的状态。Message Passing的流程可以分为以下两个步骤：从邻居节点收集消息message，然后利用神经网络来更新节点表示。pytorch_geometric是目前最完善的用来实现GNN相关工作的代码库了，在该库中基本实现了目前你所知道的大多已经发表的GNN模型，并提供了统一的接口方便研究者在此基础上做新算法的设计。

Message Passing便是其中一个重要接口。实际上Tensorflow2.0同样能做到这些。

在fennlp中我通过tensorflow2.0实现了message passing并基于此实现了GAN,GCN,和GIN，并且达到了相关论文的精度。相关结果以及代码已经开源在github上。更多模型实现正在努力中…

由于笔者研究方向是自然语言处理，所以该Package有一定的倾向性，更多的模型是用来做NLP相关任务，如果你对NLP相关任务并没兴趣，可以直接看Package中的GNN模块，以及案例。

---

Package description

The field of natural language processing is currently undergoing tremendous changes, and many excellent models have been proposed in recent years, including BERT, GPT, etc.
At the same time, graph neural network as an exquisite design is constantly being used in the field of natural language processing, such as TextGCN and Tensor-TextGCN.
This toolbox is dedicated to natural language processing and expects to implement models in the simplest way.
Keywords: NLP; GNN

Models:

• BERT
• ALBERT
• GPT2
• TextCNN
• Bilstm+Attention
• GCN, GAN, GIN

Examples (See tests for more details):

• BERT-NER (Chinese and English Version)
• BERT-CRF-NER (Chinese and English Version)
• BERT-CLS (Chinese and English Version)
• ALBERT-NER (Chinese and English Version)
• ALBERT-CLS (Chinese and English Version)
• GPT2-generation (English Version)
• Bilstm+Attention (Chinese and English Version)
• TextCNN(Chinese and English Version)
• GCN, GAN, GIN (Base on message passing)

Requirement

tensorflow-gpu>=2.0

GNN

1、GCN, GAN, GIN (Based on message passing)

Same data split and parameters setting as proposed in the paper

model	Cora	Pubmed	Citeseer
GCN	0.8180	0.7950	0.7120
GAN	0.8300	0.7900	0.7230

---

以上是该Package的一些基本信息，以及GNN在其中的表现。

更多细节参考：https://github.com/kyzhouhzau/NLPGNN

参考:
【1】Neural Message Passing for Quantum Chemistry