On the Expressive Power of Geometric Graph Neural Networks (2301.09308v3)

Abstract: The expressive power of Graph Neural Networks (GNNs) has been studied extensively through the Weisfeiler-Leman (WL) graph isomorphism test. However, standard GNNs and the WL framework are inapplicable for geometric graphs embedded in Euclidean space, such as biomolecules, materials, and other physical systems. In this work, we propose a geometric version of the WL test (GWL) for discriminating geometric graphs while respecting the underlying physical symmetries: permutations, rotation, reflection, and translation. We use GWL to characterise the expressive power of geometric GNNs that are invariant or equivariant to physical symmetries in terms of distinguishing geometric graphs. GWL unpacks how key design choices influence geometric GNN expressivity: (1) Invariant layers have limited expressivity as they cannot distinguish one-hop identical geometric graphs; (2) Equivariant layers distinguish a larger class of graphs by propagating geometric information beyond local neighbourhoods; (3) Higher order tensors and scalarisation enable maximally powerful geometric GNNs; and (4) GWL's discrimination-based perspective is equivalent to universal approximation. Synthetic experiments supplementing our results are available at \url{https://github.com/chaitjo/geometric-gnn-dojo}

• The paper introduces the Geometric Weisfeiler-Leman (GWL) test, extending the WL framework to distinguish geometric graphs while respecting physical symmetries.
• It categorizes geometric GNNs into invariant and equivariant models, emphasizing higher-order tensors and scalarization to boost expressivity.
• The study demonstrates that the universality of geometric GNNs enables complex spatial approximations, guiding the design of more effective network architectures.

On the Expressive Power of Geometric Graph Neural Networks

This paper addresses the expressive capabilities of Geometric Graph Neural Networks (GNNs), focusing on their ability to handle geometric graphs embedded in Euclidean spaces. The paper introduces a geometric adaptation of the Weisfeiler-Leman (WL) graph isomorphism test, termed the Geometric Weisfeiler-Leman (GWL) test. This new framework aims to distinguish geometric graphs while respecting intrinsic physical symmetries such as permutation, rotation, reflection, and translation.

Key Contributions

1. Geometric Weisfeiler-Leman (GWL) Test: The GWL test extends the classical WL framework to geometric graphs. It distinguishes graphs based on $t$-hop subgraph neighborhoods, considering geometric symmetries to enhance graph discriminability. The introduction of the GWL test provides a theoretical upper bound for the expressive power of geometric GNNs.
2. Characterization of Geometric GNNs:

The paper categorizes geometric GNNs into: - Invariant GNNs with limited expressivity constrained by one-hop identical graphs. - Equivariant GNNs that capture extensive geometric details through deep equivariant layers. - The utilization of higher-order tensors and scalarization for maximum expressive power in geometric GNNs.

1. Universality and Practical Implications: The research establishes that the ability of geometric GNNs to discriminate graphs equates to universal approximation. This universality is linked with practical challenges in constructing injective and expressive GNN architectures.

Implications for Theory and Practice

The GWL test offers a structured approach to analyzing the expressive power of geometric GNNs, providing insights into network design decisions, such as the implications of network depth and the use of invariant versus equivariant layers. The paper also highlights the significance of higher body-order scalarizations in enhancing expressivity, which could inform the design of future GNN architectures to effectively model complex geometric systems in materials science, biochemistry, and robotics.

Speculations on Future AI Developments

As methodologies such as GWL influence GNN design, a natural evolution toward incorporating more sophisticated geometric features and enhancing scalability is expected. The development of architectures that can balance computational efficiency with expressivity through hierarchical and modular design patterns is a potential direction. Moreover, the quest (avoiding fanciful language) to expand GNNs to more general geometric domains opens novel avenues for research in artificial intelligence, moving towards a comprehensive understanding of spatial and structural data in various scientific disciplines.

In conclusion, the paper rigorously advances the theoretical understanding of GNNs, providing a robust framework for researchers to assess and improve the expressivity of geometric network architectures.