Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks (2203.00199v5)

Abstract: Graph neural networks (GNN) have shown great advantages in many graph-based learning tasks but often fail to predict accurately for a task-based on sets of nodes such as link/motif prediction and so on. Many works have recently proposed to address this problem by using random node features or node distance features. However, they suffer from either slow convergence, inaccurate prediction, or high complexity. In this work, we revisit GNNs that allow using positional features of nodes given by positional encoding (PE) techniques such as Laplacian Eigenmap, Deepwalk, etc. GNNs with PE often get criticized because they are not generalizable to unseen graphs (inductive) or stable. Here, we study these issues in a principled way and propose a provable solution, a class of GNN layers termed PEG with rigorous mathematical analysis. PEG uses separate channels to update the original node features and positional features. PEG imposes permutation equivariance w.r.t. the original node features and imposes $O(p)$ (orthogonal group) equivariance w.r.t. the positional features simultaneously, where $p$ is the dimension of used positional features. Extensive link prediction experiments over 8 real-world networks demonstrate the advantages of PEG in generalization and scalability.

Equivariant and Stable Positional Encoding for Graph Neural Networks

Graph Neural Networks (GNNs) exhibit proficient capabilities on graph-based learning tasks, notably single-node tasks and whole-graph classifications. However, they encounter challenges when predicting sets of nodes like link prediction or motif prediction. The efficient encoding of node positions, known as positional encoding (PE), presents a promising approach to overcome these challenges. The paper entitled "Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks" provides a thorough investigation into PE techniques, proposing a novel solution termed PEG, backed by mathematical rigor to enhance GNN performance.

Revisiting Graph Neural Network Shortcomings

Traditional methods such as GCN and variants of GAE often struggle with node-set tasks due to inherent node ambiguity where nodes that map under graph automorphism become indistinguishable. Attempts to resolve this through random features (RF) inject variability, resulting in convergence issues and inaccuracies, while deterministic distance encoding (DE) introduces computational overhead and complexities. The exploration of PE arises from the need to distinguish nodes without compromising scalability and generalization.

The Proposed Solution: PEG Layer

The authors introduce PEG, a class of GNN layers designed to handle PE in a mathematically sound manner. PEG employs separate channels for original node features and positional features, imposing permutation equivariance for node features and orthogonal group (O(p)) equivariance for positional features. The mathematical framework supporting PEG offers stability, crucial for expected perturbations in graph structures. Key proofs demonstrate PEG's invariance to coordinate rotations and reflections as pivotal to maintaining stability and generalization across unseen graphs.

Experimental Analysis: Practical Implications

The performance of PEG was evaluated on extensive link prediction experiments across eight real-world networks, showcasing superior accuracy and scalability, especially when tasked with domain-shift link prediction scenarios. In traditional settings, PEG demonstrated comparable performance with strong DE-based baselines and notably enhanced generalization in cross-domain predictions over unseen graphs.

1. Link prediction tasks: PEG achieved compelling results, comparable to standard DE techniques like SEAL, with efficient computation and reduced computational costs.
2. Domain generalization: PEG's stability and equivariance led to significant performance gains in cross-domain link prediction tasks, demonstrating its robustness and applicability in real-world applications.
3. Computational efficiency: Compared to DE methods, PEG maintains lower training and testing complexity, optimizing the model's usability in diverse operational environments.

Theoretical and Future Directions

The rigorous mathematical treatment of PE stability underpins the contributions of this work, offering a robust foundation for future research developments. The success of PEG encourages further exploration into PE forms such as Deepwalk and LINE, expanding applicability beyond Laplacian Eigenmap. Additionally, the conventional neural message-passing framework could be further enhanced through incorporation of this technique, facilitating broader impacts across various graph-based tasks beyond link prediction.

In conclusion, the insights from this paper establish PEG as a valuable tool for improving GNN expressiveness in graph learning tasks, with extensions potentially extending to temporal networks and complex pattern predictions. Most notably, the established theoretical understanding opens pathways for optimized implementations tailored to emerging challenges in graph data analysis.