A Comprehensive Survey on Graph Neural Networks
-
LINK: https://arxiv.org/abs/1901.00596
-
CLASSIFICATION: GNN, SURVEY
-
YEAR: Submitted on 3 Jan 2019 (v1), last revised 4 Dec 2019 (this version, v4)
-
FROM: ArXiv 2019
-
WHAT PROBLEM TO SOLVE: Existing surveys only include some of the GNNs and examine a limited number of works, thereby missing the most recent development of GNNs.
-
SOLUTION:
This paper makes notable contributions summarized as follows:
-
New taxonomy
Recurrent graph neural networks, Convolutional graph neural networks, Graph autoencoders, and Spatial-temporal graph neural networks.
-
Comprehensive review
Provide detailed descriptions on representative models, make the necessary comparison, and summarise the corresponding algorithms.
-
Abundant resources
Including state-of-the-art models, benchmark data sets, open-source codes, and practical applications.
-
Future directions
Model depth, scalability trade-off, heterogeneity, and dynamicity.
-
-
CORE POINT:
-
Taxonomy of GNNs
-
RecGNNs: Aim to learn node representations with recurrent neural architectures. They assume a node in a graph constantly exchanges information/message with its neighbors until a stable equilibrium is reached.
-
ConvGNNs: Generate a node v’s representation by aggregating its own features xv and neighbors’ features xu, where u ∈ N(v). Different from RecGNNs, ConvGNNs stack multiple graph convolutional layers to extract high-level node representations.
-
GAEs: Unsupervised learning frameworks which encode nodes/graphs into a latent vector space and reconstruct graph data from the encoded information. GAEs are used to learn network embeddings and graph generative distributions.
-
STGNNs: Consider spatial dependency and temporal dependency at the same time. Many current approaches integrate graph convolutions to capture spatial dependency with RNNs or CNNs to model the temporal dependency.
-
-
Level Tasks
- Node-level: Outputs relate to node regression and node classification tasks with a multi-perceptron or a softmax layer as the output layer. RecGNNs and ConvGNNs can extract high-level node representations by information propagation/graph convolution.
- Edge-level: With two nodes’ hidden representations from GNNs as inputs, a similarity function or a neural network can be utilized to predict the label/connection strength of an edge.
- Graph-level: Outputs relate to the graph classification task. To obtain a compact representation on the graph level, GNNs are often combined with pooling and readout operations.
-
Training Framework
- Semi-supervised learning for node-level classification: Given a single network with partial nodes being labeled and others remaining unlabeled, ConvGNNs can learn a robust model that effectively identifies the class labels for the unlabeled nodes.
- Supervised learning for graph-level classification: Graph-level classification aims to predict the class label(s) for an entire graph. The end-to-end learning for this task can be realized with a combination of graph convolutional layers, graph pooling layers, and/or readout layers.
- Unsupervised learning for graph embedding: When no class labels are available in graphs, we can learn the graph embedding in a purely unsupervised way in an end-to-end framework.
-
Representative RecGNNs and ConvGNNs
O(m) if the graph adjacency matrix is sparse and is O(n^2) otherwise, O(n^3) due to some other operations.
-
RECURRENT GRAPH NEURAL NETWORKS
Apply the same set of parameters recurrently over nodes in a graph to extract high-level node representations.
- Graph Neural Network (GNN*)
- Graph Echo State Network (GraphESN)
- Gated Graph Neural Network (GGNN)
- Stochastic Steady-
-