Graph Neural Networks: A Review of Methods and Applications (1812.08434v6)

Abstract: Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling physics systems, learning molecular fingerprints, predicting protein interface, and classifying diseases demand a model to learn from graph inputs. In other domains such as learning from non-structural data like texts and images, reasoning on extracted structures (like the dependency trees of sentences and the scene graphs of images) is an important research topic which also needs graph reasoning models. Graph neural networks (GNNs) are neural models that capture the dependence of graphs via message passing between the nodes of graphs. In recent years, variants of GNNs such as graph convolutional network (GCN), graph attention network (GAT), graph recurrent network (GRN) have demonstrated ground-breaking performances on many deep learning tasks. In this survey, we propose a general design pipeline for GNN models and discuss the variants of each component, systematically categorize the applications, and propose four open problems for future research.

Overview of "Graph Neural Networks: A Review of Methods and Applications"

The paper "Graph Neural Networks: A Review of Methods and Applications" by Jie Zhou et al. provides a thorough examination of graph neural networks (GNNs), exploring their underlying methods and extensive real-world applications. The authors delineate a systematic taxonomy of various GNN models, proffering a holistic design pipeline and addressing open problems in the field.

Introduction

Graph neural networks (GNNs) are pivotal in domains requiring the analysis of graph-structured data. These data structures encapsulate rich relational information among elements, necessitating sophisticated models adept at capturing such dependencies. GNNs have demonstrated exceptional utility in numerous areas, ranging from physical system modeling to image reasoning.

Methodological Insight

The general design pipeline for GNNs encompasses four integral steps: identifying the graph structure, specifying graph type and scale, designing loss functions, and constructing the model using computational modules.

1. Graph Structure: In structural scenarios, graphs are explicitly defined, while in non-structural scenarios, they must be inferred.
2. Graph Type and Scale: Graph types can range from directed and undirected to homogeneous and heterogeneous, with additional considerations for large-scale graphs necessitating sampling techniques.
3. Loss Function Design: Depending on the task type (node-level, edge-level, or graph-level) and the training setting (supervised, semi-supervised, or unsupervised), appropriate loss functions are crafted.
4. Model Construction: The construction involves essential computational modules such as propagation, sampling, and pooling modules, each with varied implementations.

Computational Modules

The paper discusses several key computational modules:

• Propagation Modules: These facilitate the propagation of information across graph nodes and include convolution operators, recurrent operators, and skip connections. Convolution operators are further categorized into spectral and spatial approaches, with attention-based methods like Graph Attention Network (GAT) gaining prominence for their flexibility in handling varying node degrees and facilitating inductive learning.
• Sampling Modules: These modules, crucial for handling large graphs, range from node and layer sampling to subgraph sampling techniques, each optimizing the propagation process.
• Pooling Modules: These are employed to glean high-level representations from nodes, helpful in tasks requiring graph-level representations. Methods vary from direct pooling like node selection strategies to hierarchical approaches involving graph coarsening techniques.

Variants Considering Graph Type and Scale

The paper highlights various advanced modifications catering to specific graph types:

• Directed Graphs: Methods like DGP (Kampffmeyer et al.) leverage different weight matrices for forward and reverse edges.
• Heterogeneous Graphs: Approaches such as HAN, MAGNN, and R-GCN handle multi-typed nodes and edges, improving representations by incorporating meta-path based methods.
• Dynamic Graphs: Models like DCRNN and EvolveGCN address temporally evolving structures, encoding graphs with dynamic variables.
• Large Graphs: Techniques like FastGCN, ClusterGCN, and GraphSAINT focus on scaling up GNNs using efficient sampling and approximation strategies.

Training Approaches

For unsupervised training, the paper presents methods like Graph Auto-Encoders (GAE, VGAE) and adversarially regularized autoencoders (ARGA). Additionally, contrastive learning methods like Deep Graph Infomax (DGI) and Infograph optimize mutual information to enhance representation learning.

Application Spectrum

GNNs excel in diverse applications within structured and non-structured domains:

• Graph Mining: Tasks such as graph matching and clustering.
• Physical Systems and Biology: Modeling molecular fingerprints and protein interfaces.
• Knowledge Graphs: Enhancing knowledge base completion and alignment.
• Generative Models and Combinatorial Optimization: Applications in traffic networks, recommendation systems, and optimization problems like TSP.

Implications and Future Developments

The implications of GNN research span theoretical and practical horizons:

• Theoretical: Current trends focus on the generalization, expressivity, and invariance properties of GNNs, exploring their transferability and stability.
• Practical: Applications continue to expand into domains previously dominated by traditional machine learning, emphasizing the robustness and interpretability of models.

Concluding Remarks and Open Problems

Despite the impressive strides, the field grapples with challenges:

• Robustness: Addressing adversarial attacks to bolster GNN defenses.
• Interpretability: Enhancing the transparency of model predictions.
• Pretraining: Developing self-supervised methods for graph pretraining.
• Complex Structures: Accommodating the intricacies of dynamic, heterogeneous, and other complex graphs.

Conclusively, as GNNs evolve, they promise to redefine data representations across various domains, spearheading advancements in both AI research and practical applications.