Message passing all the way up (2202.11097v1)

Abstract: The message passing framework is the foundation of the immense success enjoyed by graph neural networks (GNNs) in recent years. In spite of its elegance, there exist many problems it provably cannot solve over given input graphs. This has led to a surge of research on going "beyond message passing", building GNNs which do not suffer from those limitations -- a term which has become ubiquitous in regular discourse. However, have those methods truly moved beyond message passing? In this position paper, I argue about the dangers of using this term -- especially when teaching graph representation learning to newcomers. I show that any function of interest we want to compute over graphs can, in all likelihood, be expressed using pairwise message passing -- just over a potentially modified graph, and argue how most practical implementations subtly do this kind of trick anyway. Hoping to initiate a productive discussion, I propose replacing "beyond message passing" with a more tame term, "augmented message passing".

• The paper demonstrates that augmenting traditional message passing with strategic graph modifications effectively addresses complex graph functions.
• The paper reinterprets methods labeled 'beyond message passing' as enhanced forms utilizing rewiring and feature augmentation.
• The paper highlights that a unified augmented message passing framework simplifies GNN design and guides future research on graph transformations.

Analyzing "Message Passing All the Way Up"

Introduction to the Argument

The paper "Message Passing All the Way Up" by Petar Veličković provides a critical examination of the prevalent terminology and assumptions in graph neural networks (GNNs), specifically regarding the phrase "beyond message passing." The author challenges the notion that to overcome the limitations of conventional message passing frameworks, wholly new paradigms must be developed. Instead, Veličković argues that augmenting conventional message passing by modifying graph structures can extend their utility and expressiveness effectively while still operating within the known framework.

Core Thesis

Veličković's central thesis is that the expression "beyond message passing" misleads newcomers into thinking that the message-passing mechanism is fundamentally insufficient. Instead, the author posits that by augmenting the existing message-passing frameworks, almost any graph computational problem can be addressed. The author suggests adopting the term "augmented message passing" to more accurately describe this evolution and acknowledge the continued relevance of the message-passing paradigm.

Key Arguments and Evidence

The paper provides several lines of argumentation to support its thesis:

1. Universality of Augmented Message Passing: The author suggests that any desired graph function can likely be realized through pairwise message passing in a modified graph structure. He demonstrates how practical implementations often rely on implicit graph modifications such as the addition of nodes or edges.
2. Analysis of Existing Methods: Veličković reviews current methodologies, often classified as going "beyond message passing," showing how they can be reinterpreted as message-passing variants that utilize enhanced graph models. Examples include:
   • Feature Augmentation: Precomputing subgraph properties and including them as node features.
   • Graph Rewiring: Modifying the adjacency structure to facilitate interaction across graph elements.
   • Subgraph Aggregation: Processing multiple overlapping subgraphs within a graph.
3. Substructure-Based Methods: These seek to adaptively recognize complex substructure interactions. Despite their complexity, they can be seen as leverages on message passing through strategic graph alterations.
4. Equivariant GNNs: Utilizing linear permutation-equivariant layers shows how theoretical analysis leads back to graph transformations expressible through message passing over edges rather than nodes.

Implications and Future Directions

The reframing proposed by Veličković could have notable theoretical and practical implications:

• It encourages a unified view of the landscape of graph neural networks, thereby simplifying the conceptual model for newcomers and easing the path for academic discourse.
• The proposed framework may stimulate future research to focus on optimizing graph structure transformations and augmentations rather than developing completely new paradigms.
• This approach could encourage more efficient implementations, leveraging current highly optimized message-passing operations, reducing the need for wholly new frameworks that may not yet enjoy similar levels of hardware and software support.

Conclusion

By arguing for "augmented message passing" instead of interpretations that suggest moving beyond it, Petar Veličković's paper offers a thought-provoking perspective that could steer the field toward a more cohesive understanding of graph neural network expressivity. This refined understanding not only clarifies the message-passing debate but also potentially broadens the applicability of GNNs across more complex graph problems by focusing on augmenting core methodologies with graph modifications. The paper lays a foundation for more rigorous exploration of these augmentations, deepening the field's appreciation of message passing's inherent power and versatility.