How Powerful are Graph Neural Networks? (1810.00826v3)

Abstract: Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks. However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations. Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures. We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance.

Analyzing the Expressive Power of Graph Neural Networks

Introduction

The paper of Graph Neural Networks (GNNs) has unveiled their potency in the domain of representation learning for graph-structured data, allowing the encapsulation of nodes' local structure and feature information. These models, following a recursive neighborhood aggregation scheme, have achieved notable success across tasks such as node classification, link prediction, and graph classification. However, the theoretical foundation underpinning the representational capacity of GNNs remains relatively uncharted.

Theoretical Framework for GNN Expressive Power

This paper presents a comprehensive theoretical framework to examine the expressive power of GNNs, delineating how different GNN variants perform in distinguishing various graph structures. A significant contribution is the establishment of a connection between GNNs and the Weisfeiler-Lehman (WL) graph isomorphism test. This test iteratively refines node labels based on neighborhood structures and serves as a benchmark for GNN discriminative power.

Insights and Main Contributions

The paper delivers several key insights:

1. Upper Bound Expressiveness: It proves that GNNs are at most as powerful as the WL test in distinguishing different graph structures.
2. Conditions for Maximal Expressiveness: The authors delineate specific conditions under which a GNN can mirror the maximal discriminative power of the WL test. This requires the aggregation and readout functions to be injective.
3. Limitations of Existing GNN Variants: Popular GNN architectures such as Graph Convolutional Networks (GCNs) and GraphSAGE are shown to be inherently less powerful due to the non-injective nature of their aggregation schemes.
4. Graph Isomorphism Network (GIN): The authors propose the Graph Isomorphism Network (GIN) which achieves maximal expressiveness akin to the WL test by leveraging sum aggregation schemes. This model outperforms other GNN variants empirically on various graph classification benchmarks.

Empirical Validation

To corroborate their theoretical findings, the authors conduct extensive experiments on nine graph classification datasets spanning bioinformatics and social networks. The results validate the superior training and test performance of GIN compared to other GNN variants. Specifically:

• Training Performance: GIN nearly perfectly fits the training data, underscoring its strong representational power.
• Test Performance: GIN achieves state-of-the-art performance in graph classification tasks, outperforming less expressive GNN variants and matching or exceeding the performance of the WL subtree kernel.

Discussion on Aggregation Strategies

The analysis encompasses the limitations of mean and max-pooling schemes employed by typical GNN variants. Mean aggregators, while useful for capturing the distribution of features, fall short in distinguishing graphs where node features repeat. Max-pooling, on the other hand, effectively captures the skeleton of graphs but similarly fails to distinguish more nuanced structures.

Implications and Future Directions

Theoretical insights into the representational constraints and capabilities of GNNs can guide the design of more robust and expressive models. The conditions laid out for achieving maximal expressiveness highlight the importance of injective aggregation functions. Future research can explore beyond neighborhood aggregation frameworks to derive more powerful graph learning architectures.

The findings prompt the refinement of existing GNN architectures and suggest potential pathways for achieving better generalization on graph-based learning tasks. Moreover, understanding the optimization landscape and generalization dynamics of GNNs can complement these theoretical advancements, leading to more reliable and high-performing models in practice.

Conclusion

This paper establishes a theoretical foundation for analyzing and enhancing the expressive power of GNNs. By connecting GNN expressiveness to the WL graph isomorphism test and introducing the powerful GIN model, it paves the way for future research and development in graph representation learning. The theoretical insights and empirical validations underscore the potential for constructing more discriminative and application-effective GNNs.