Adversarial Graph Augmentation to Improve Graph Contrastive Learning (2106.05819v4)

Abstract: Self-supervised learning of graph neural networks (GNN) is in great need because of the widespread label scarcity issue in real-world graph/network data. Graph contrastive learning (GCL), by training GNNs to maximize the correspondence between the representations of the same graph in its different augmented forms, may yield robust and transferable GNNs even without using labels. However, GNNs trained by traditional GCL often risk capturing redundant graph features and thus may be brittle and provide sub-par performance in downstream tasks. Here, we propose a novel principle, termed adversarial-GCL (AD-GCL), which enables GNNs to avoid capturing redundant information during the training by optimizing adversarial graph augmentation strategies used in GCL. We pair AD-GCL with theoretical explanations and design a practical instantiation based on trainable edge-dropping graph augmentation. We experimentally validate AD-GCL by comparing with the state-of-the-art GCL methods and achieve performance gains of up-to $14\%$ in unsupervised, $6\%$ in transfer, and $3\%$ in semi-supervised learning settings overall with 18 different benchmark datasets for the tasks of molecule property regression and classification, and social network classification.

• The paper introduces AD-GCL, which employs adversarial training to optimize trainable edge-dropping and reduce redundant graph features.
• It achieves up to 14% improvements in unsupervised settings, 6% in transfer learning, and 3% in semi-supervised tasks across diverse benchmarks.
• The method generalizes graph augmentation by learning non-uniform edge drop probabilities, thereby enhancing robustness without reliance on annotated data.

Adversarial Graph Augmentation to Improve Graph Contrastive Learning

The paper by Susheel Suresh et al. introduces a novel approach to enhance the performance of Graph Contrastive Learning (GCL) through a method called Adversarial Graph Contrastive Learning (AD-GCL). This work focuses on addressing the limitations of traditional GCL methods, which can capture redundant graph features and impair the robustness and transferability of graph neural networks (GNNs) in various tasks. The research proposes an adversarial framework to optimize graph data augmentations, specifically through trainable edge-dropping strategies, with the objective of retaining only the minimal yet sufficient information for downstream graph-level tasks.

The AD-GCL Principle and Implementation

The core principle of AD-GCL is to pair the GCL objective, which maximizes the correspondence between different augmented views of a graph, with adversarial training that minimizes the mutual information captured from potentially redundant features. The paper introduces a two-component model: a GNN encoder that follows the InfoMax principle by maximizing mutual information, and a GNN-based augmenter that uses adversarial training to optimize augmentation strategies that reduce redundancy.

A notable theoretical insight provided by the authors is the ability of AD-GCL to maintain an upper bound on redundant information while ensuring a lower bound on task-relevant information. This is significant because it aligns with the aims of the Information Bottleneck (IB) principle without the need for downstream task labels—a common challenge in self-supervised learning settings. The researchers demonstrate that by learning non-uniform edge drop probabilities, the AD-GCL can effectively discern and discard less informative parts of the graph, thereby improving the learnt representations' robustness and utility.

Empirical Evaluation

The paper validates the efficacy of AD-GCL through extensive experiments on large-scale benchmarks, including both chemical molecular property datasets and social network graphs. The experimental results consistently exhibit performance improvements over state-of-the-art GCL methods, with up to 14% gains in unsupervised settings, 6% improvements in transfer learning, and 3% gains in semi-supervised learning across diverse tasks such as molecule property regression, classification, and social network classification. These results underscore the practical viability of AD-GCL in enhancing GNNs' adaptability and accuracy without reliance on annotated data.

Implications and Future Directions

The introduction of a learnable augmentation strategy represents a critical shift in the design of graph representation learning models. The AD-GCL framework not only offers a robust solution to reduce the dependencies on manual or domain-specific augmentation selection but also facilitates the development of more generalizable models across different graph-level applications.

Future developments in AI could see this approach extended to more complex graph structures and diverse domains, potentially exploring additional augmentation techniques beyond edge-dropping. Moreover, the satisfactory trade-off between augmentation aggressiveness and informativeness seen in AD-GCL could inspire new methodologies for example-driven augmentation in other domains like natural language processing or computer vision, wherein similar challenges of redundancy and information bottleneck must be addressed.

In conclusion, the contribution of the AD-GCL method lies in its theoretically-backed, adversarial enhancement of graph contrastive learning. This paper sets a foundation that researchers can build upon to further improve self-supervised learning techniques by reducing redundant feature capture and promoting effective information extraction from graph data.