Equivariant Subgraph Aggregation Networks (2110.02910v3)

Abstract: Message-passing neural networks (MPNNs) are the leading architecture for deep learning on graph-structured data, in large part due to their simplicity and scalability. Unfortunately, it was shown that these architectures are limited in their expressive power. This paper proposes a novel framework called Equivariant Subgraph Aggregation Networks (ESAN) to address this issue. Our main observation is that while two graphs may not be distinguishable by an MPNN, they often contain distinguishable subgraphs. Thus, we propose to represent each graph as a set of subgraphs derived by some predefined policy, and to process it using a suitable equivariant architecture. We develop novel variants of the 1-dimensional Weisfeiler-Leman (1-WL) test for graph isomorphism, and prove lower bounds on the expressiveness of ESAN in terms of these new WL variants. We further prove that our approach increases the expressive power of both MPNNs and more expressive architectures. Moreover, we provide theoretical results that describe how design choices such as the subgraph selection policy and equivariant neural architecture affect our architecture's expressive power. To deal with the increased computational cost, we propose a subgraph sampling scheme, which can be viewed as a stochastic version of our framework. A comprehensive set of experiments on real and synthetic datasets demonstrates that our framework improves the expressive power and overall performance of popular GNN architectures.

• The paper introduces novel WL test variants and theoretical insights that enhance graph isomorphism discrimination.
• The paper proposes two architecture variants, DSS-GNN and DS-GNN, that demonstrate improved performance on graph classification tasks.
• The paper validates ESAN’s practical benefits in domains like cheminformatics and social network analysis through rigorous experiments.

Equivariant Subgraph Aggregation Networks: Enhancing Expressive Power in Graph Neural Networks

The paper "Equivariant Subgraph Aggregation Networks" (ESAN) addresses a fundamental issue in the field of graph neural networks (GNNs): the expressive limitations of traditional message-passing neural networks (MPNNs). Such limitations are particularly evident when these models attempt to distinguish between non-isomorphic graphs that, intuitively, seem different yet remain indistinguishable by these architectures. The proposed ESAN framework seeks to enhance the expressive power of GNNs by leveraging the equivariance properties of subgraphs derived from original graph structures.

Overview of the Proposed Approach

At the heart of the ESAN framework lies the hypothesis that, although MPNNs struggle to differentiate between entire graphs, focusing on subgraphs may reveal distinguishing characteristics. Thus, ESAN represents each input graph as a collection of subgraphs according to a predefined policy—such as deleting nodes or edges—and processes them through a carefully designed equivariant architecture. The novelty of ESAN includes developing new variants of the 1-dimensional Weisfeiler-Leman (1-WL) test for graph isomorphism, which serves as a benchmark for the expressiveness of graph machine learning models.

Main Contributions

1. Theoretical Advancements:
   • The paper introduces variants of the existing WL test, namely DSS-WL and DS-WL, enhancing separation capabilities for non-isomorphic graphs. ESAN proves strictly more powerful than the 1-WL test, particularly with suitably chosen subgraph selection policies such as node-deleted or edge-deleted subgraphs.
   • Theoretically, ESAN further shows that it can extend the expressivity of both MPNNs and more sophisticated architectures, such as those using higher-dimensional WL tests, by incorporating suitable subgraph selection mechanisms.
2. Design and Architecture:
   • ESAN proposes two main architecture variants, DSS-GNN and DS-GNN, with the latter being a simpler implementation where subgraphs are processed independently (a Siamese network).
3. Experimental Verification:
   • Experimental setups on synthetic and real-world datasets substantiate ESAN's claims, noting improvement in graph classification benchmarks and exhibiting perfect performance on synthetic benchmark graphs designed to test expressiveness.

Practical Implications

The implications of ESAN's enhanced expressivity are significant for domains reliant on graph data representation, such as cheminformatics and social network analysis, where traditional GNN approaches might underperform. By efficiently processing bags of subgraphs, ESAN maintains scalability while expanding the modelling capacity, adept at revealing intricate patterns that GNNs might overlook.

Future Directions

The authors imply potential future investigations in:

• Learned Subgraph Selection: Designing methodologies to autonomously determine optimal subgraph policies suited for specific tasks.
• Complex Subgraph Relationships: Examining whether construction of higher-order structures within subgraphs could lead to even more powerful discriminative capabilities.
• Stochastic Version Analysis: Providing a theoretical foundation for the stochastic variant, which promises computational efficiency gains without markedly sacrificing performance.

In summary, the ESAN framework provides a robust enhancement to the representation and expressivity of graph neural networks. It intelligently utilizes the structural information available in graphs through subgraph decomposition, ensuring that both the expressive power and scalability of GNNs are bolstered. The paper contributes a substantive theoretical and practical framework, likely to inspire further research and application across diverse fields reliant on graph representations.