Graph Random Neural Network for Semi-Supervised Learning on Graphs (2005.11079v4)

Abstract: We study the problem of semi-supervised learning on graphs, for which graph neural networks (GNNs) have been extensively explored. However, most existing GNNs inherently suffer from the limitations of over-smoothing, non-robustness, and weak-generalization when labeled nodes are scarce. In this paper, we propose a simple yet effective framework -- GRAPH RANDOM NEURAL NETWORKS (GRAND) -- to address these issues. In GRAND, we first design a random propagation strategy to perform graph data augmentation. Then we leverage consistency regularization to optimize the prediction consistency of unlabeled nodes across different data augmentations. Extensive experiments on graph benchmark datasets suggest that GRAND significantly outperforms state-of-the-art GNN baselines on semi-supervised node classification. Finally, we show that GRAND mitigates the issues of over-smoothing and non-robustness, exhibiting better generalization behavior than existing GNNs. The source code of GRAND is publicly available at https://github.com/Grand20/grand.

• The paper introduces a novel random propagation mechanism that mitigates over-smoothing and boosts robustness in graph learning using a structured DropNode approach.
• It leverages consistency regularization to enforce similarity across random augmentations, leading to improved generalization on semi-supervised graphs.
• Empirical results show that Grand outperforms standard models like GCN and GAT on benchmarks while resisting adversarial perturbations.

Overview of Graph Random Neural Networks for Semi-Supervised Learning on Graphs

The paper "Graph Random Neural Networks for Semi-Supervised Learning on Graphs" introduces a new framework named Graph Random Neural Network (Grand). This work addresses critical issues prevalent in current Graph Neural Networks (GNNs) such as over-smoothing, non-robustness, and reduced generalization power in scenarios with limited labeled data. To mitigate these issues, the authors propose a simple yet effective model architecture that leverages a random propagation strategy coupled with consistency regularization.

Problem Statement and Context

Semi-supervised learning on graphs is a task wherein the objective is to predict the labels of a large number of unlabeled nodes using a small subset of labeled nodes. The inherent challenges in this task when using GNNs include their susceptibility to non-robustness and over-smoothing. A primary reason for these issues is the deterministic feature propagation scheme common in existing models, which can render node features indistinguishable and vulnerable to adversarial attacks.

Contributions and Methodology

Graph Random Neural Network (Grand): The key contribution of Grand is its innovative use of random propagation for graph data augmentation. Traditional GNNs rely on deterministic propagation, while Grand integrates a random propagation strategy, where each node can randomly drop along its neighborhood interactions during training. This stochasticity enhances the model's robustness and enables the separation of feature propagation from transformation, which reduces over-smoothing and overfitting.

1. Random Propagation: Grand's novel propagation mechanism randomly perturbs node features. Unlike traditional dropout methods used in deep learning, Grand employs "DropNode," which involves dropping entire node features rather than single elements. This introduces a more structured noise and significant stochastic data augmentations. Nodes thereby generate approximate representations through their neighborhoods, leading to more robust and diverse feature representations.
2. Consistency Regularization: Building upon the augmented features, Grand employs consistency regularization that leverages variance among multiple data augmentations to enforce similarities in model predictions across augmentations. This approach promotes better generalization by leveraging the consistency of unlabeled data.
3. Theoretical Insights: The paper provides a theoretical underpinning showing how the variabilities induced by random propagation contribute to a regularization effect, encouraging smoothness in the feature space over the graph's structured data.

Experimental Evaluation

Empirical evaluations reveal Grand's robustness and versatility, outperforming other state-of-the-art GNN architectures across standard benchmark datasets. Results indicate substantial improvements in accuracy over baseline models like GCN and GAT. Further experiments highlight Grand's capability to maintain model performance under different adversarial perturbations and conditions known to cause over-smoothing. The research shows that as the complexity typically associated with GNN architectures is reduced, performance gains are still achieved.

Implications and Future Directions

The introduction of a graph data augmentation strategy paired with consistency regularization presents an impactful advancement in semi-supervised graph learning. By demonstrating improved performance and robustness without expending computational complexity, Grand offers a potential pivot from the typical trajectory focussed on complex GNN architectures and opens avenues for more focus on regularization-centric research.

Future developments could investigate scaling these approaches to even larger graphs and more complex datasets. Additionally, integrating such randomized approaches with other advances like graph attention mechanisms could further bolster model performance.

In summary, Grand exemplifies a significant stride in the pursuit of robust and effective GNNs for semi-supervised learning, showcasing that simpler models when empowered with innovative data handling and regularization strategies can offer comparable or superior results compared to more complex architectural designs.