Learnable Laplacian Positional Encodings

• LLPE are learnable positional encoding techniques that utilize the graph Laplacian’s spectral properties to generate robust and expressive node embeddings.
• They employ learnable spectral filters and message-passing GNNs to capture both global and local graph structures while avoiding the computational bottlenecks of traditional eigenvector methods.
• These methods deliver state-of-the-art performance on various benchmarks, offering scalable, adaptive, and theoretically grounded alternatives for graph representation learning.

Learnable Laplacian Positional Encodings (LLPE) are a class of techniques that leverage the spectral properties of a graph’s Laplacian operator to construct node representations optimized for use in graph neural networks (GNNs) and related models. These methods address the core challenges of capturing global and local topological information, efficiently scaling to large graphs, and ensuring robustness under graph perturbations or task shifts.

LLPE integrates principles from spectral graph theory, deep learning, and positional encoding, permitting learnable, often end-to-end differentiable, mappings from graph structure to node embeddings. These encodings have demonstrated both theoretical and empirical advantages over fixed Laplacian eigenvector–based methods, especially regarding stability, expressivity, scalability, and adaptivity to homophilous and heterophilous regimes.

1. Spectral Foundations and Laplacian Eigenvector Encodings

Let $G = (V, E)$, with adjacency matrix $A \in \mathbb{R}^{n \times n}$ and degree matrix $D = \mathrm{diag}(d_1,\ldots,d_n)$. The (combinatorial) Laplacian is $L = D - A$. Its eigendecomposition is $L = U \Lambda U^\top$ with orthonormal $U$ and diagonal $$\Lambda = \operatorname{diag}(\lambda_1,\ldots,\lambda_n)$$, $$0 = \lambda_1 \leq \lambda_2 \leq \cdots \leq \lambda_n$$.

Classical Laplacian eigenvector positional encodings (PEs) use either the full matrix $U$ or the truncated $U_k$ (the first $A \in \mathbb{R}^{n \times n}$0 columns) as node features. These spectral encodings provide permutation equivariance and, in principle, can distinguish non-isomorphic graphs (greater expressivity than 1-WL). However, they have several key limitations:

• Computational cost: Explicit eigendecomposition is $A \in \mathbb{R}^{n \times n}$1 in time, $A \in \mathbb{R}^{n \times n}$2 in memory, and $A \in \mathbb{R}^{n \times n}$3 for truncated $A \in \mathbb{R}^{n \times n}$4-eigenvector use—a severe bottleneck for large graphs.
• Instability: As established by Davis-Kahan and related analyses, eigenvectors of $A \in \mathbb{R}^{n \times n}$5 are highly sensitive to small perturbations if the spectral gap $A \in \mathbb{R}^{n \times n}$6 is small, leading to non-robustness under even minor noise or structure changes.
• Non-uniqueness: Eigenvectors are defined up to sign and rotation within degenerate eigenspaces; care is required to maintain permutation equivariance and invariance to basis choices (Huang et al., 2023).

2. Core Formulations of Learnable Laplacian Positional Encodings

LLPE frameworks overcome the above limitations by replacing or augmenting explicit eigenvector use with learnable, permutation-equivariant mappings. The following paradigms now anchor the field:

2.1 Filtered Spectral LLPE

LLPE can be parameterized by learnable spectral filters $A \in \mathbb{R}^{n \times n}$7, operating over the entire Laplacian spectrum. For each output dimension $A \in \mathbb{R}^{n \times n}$8, a scalar filter $A \in \mathbb{R}^{n \times n}$9 is constructed as a Chebyshev polynomial of order $D = \mathrm{diag}(d_1,\ldots,d_n)$0:

$D = \mathrm{diag}(d_1,\ldots,d_n)$1

where $D = \mathrm{diag}(d_1,\ldots,d_n)$2 is the normalized eigenvalue and $D = \mathrm{diag}(d_1,\ldots,d_n)$3 is the $D = \mathrm{diag}(d_1,\ldots,d_n)$4-th Chebyshev polynomial. Node encodings are then propagated as $D = \mathrm{diag}(d_1,\ldots,d_n)$5. This enables the model to control the frequency content explicitly—learning to emphasize low, mid, or high-frequency modes, crucial for capturing both homophilous and heterophilous graph structures (Ito et al., 29 Apr 2025).

2.2 GNN-based LLPE via Message Passing

An alternative, scalable approach instantiates LLPE via message-passing GNNs (e.g., GIN), viewing GNN layers as nonlinear, permutation-equivariant spectral filters. In PEARL (2502.01122), two cases are distinguished:

• R-PEARL (Random Initialization): Each node is seeded with $D = \mathrm{diag}(d_1,\ldots,d_n)$6 i.i.d. random vectors. The GNN processes each sampled input, pooling the $D = \mathrm{diag}(d_1,\ldots,d_n)$7 outputs (typically via mean) to produce an equivariant, spectrally expressive node embedding. This approximates general basis-invariant mappings without explicit eigendecomposition and operates in near-linear time for large graphs.
• B-PEARL (Basis Initialization): For small graphs, the GNN is initialized with standard basis vectors (one per node) and output features are pooled via summation. With polynomial GNN filters, this approach can recover expressive basis-invariant encodings, with cost $D = \mathrm{diag}(d_1,\ldots,d_n)$8.

Both methods enable LLPE to bypass the cubic complexity and stability pitfalls of direct spectral encodings, achieving high expressivity by leveraging the universality of message-passing networks.

2.3 Hybrid Approaches and Integration with Standard GNNs

Some architectures, such as LSPE (Dwivedi et al., 2021), maintain explicit positional streams initialized via a Laplacian embedding (e.g., $D = \mathrm{diag}(d_1,\ldots,d_n)$9 with learned $L = D - A$0), updating these streams alongside structural ones through GNN layers. This decoupling sharpens both local and global representational power and allows for interactively regularizing the positional component (e.g., via a Laplacian eigenvector loss).

Other works integrate a small learnable GNN or MLP to refine raw Laplacian eigenvector features (Wijesinghe et al., 22 Feb 2025), particularly in self-supervised contrastive learning, or combine them with higher-order, multi-hop, or walk-based features.

3. Theoretical Properties: Stability, Expressivity, and Universality

LLPE methods bring forth the following provable properties:

• Stability: Learnable GNN LLPEs with Lipschitz-continuous nonlinearities and bounded filters inherit Lipschitz continuity with respect to graph perturbations. For R-/B-PEARL (as formalized in (2502.01122)), node encodings obey

$L = D - A$1

where $L = D - A$2 is the number of GNN layers and $L = D - A$3 is mild in $L = D - A$4—independent of spectral gap.

• Expressivity: LLPEs using GNNs or Chebyshev spectral filtering are universal for continuous, basis-invariant functions over the Laplacian. This means that for any such $L = D - A$5, there exists a parameterization of LLPE that can approximate it to arbitrary precision (2502.01122, Huang et al., 2023). LLPEs can count cycles of length up to 7 (R-PEARL, via nonlinearities) and generalize the power of 1-WL and higher.
• Adaptivity to Homophily/Heterophily: Filter-based LLPE methods adaptively learn which parts of the spectrum—corresponding to different structural scales—are selected for the task. On stochastic block models (SBMs), LLPEs can exactly recover communities regardless of whether structure lies in the low- or high-frequency spectrum (Ito et al., 29 Apr 2025).
• Generalization: Rademacher complexity bounds for Chebyshev LLPE show generalization error does not grow with spectrum truncation order, contrasting with MLP-based approaches that scale poorly with $L = D - A$6 (Ito et al., 29 Apr 2025).

4. Empirical Performance and Benchmarking

Multiple LLPE instantiations have demonstrated state-of-the-art results and superior scaling across synthetic, molecular, and large-scale real-world graphs:

Method	Complexity	Expressivity	Stability	Example Benchmarks
R-PEARL (2502.01122)	$L = D - A$7	$L = D - A$81-WL	Yes	REDDIT-B, REDDIT-M, OGB, DrugOOD
B-PEARL (2502.01122)	$L = D - A$9	Full spectral	Yes	ZINC (logP), DrugOOD, RelBench
Chebyshev LLPE (Ito et al., 29 Apr 2025)	$L = U \Lambda U^\top$0 ($L = U \Lambda U^\top$1 approx.)	Task-adaptive	Yes	SBMs, Power-law, TEXAS, Penn94
SPE (Huang et al., 2023)	$L = U \Lambda U^\top$2	Universal	Provable	ZINC, Alchemy, DrugOOD, Cycle counting
LSPE (Dwivedi et al., 2021)	Linear (sparse GNNs)	Task-adaptive	Optional†	ZINC, OGBG-MOLTOX21, OGBG-MOLPCBA

†With Laplacian eigenvector regularization.

Key results:

• R-PEARL achieves comparable or superior performance to SignNet and other previous eigenvector-based methods at far lower computational cost; mean accuracy 94.5% on REDDIT-B with $L = U \Lambda U^\top$3 complexity (2502.01122).
• Chebyshev LLPE (with order $L = U \Lambda U^\top$4) yields up to 35% relative accuracy gain on heterophilous SBMs, and up to 14% absolute gain on real heterophilous benchmarks (e.g., Penn94) (Ito et al., 29 Apr 2025).
• SPE provides provable robustness to OOD shifts with optimal performance in stability–expressivity tradeoff regimes, outperforming earlier basis-invariant networks (Huang et al., 2023).
• LSPE and message-passing LLPEs close the gap between fixed PE (often overfitting on small data) and flexible but data-driven representation, achieving up to 64.1% error reduction on ZINC (Dwivedi et al., 2021).

5. Implementation, Complexity, and Practical Guidance

Implementation proceeds via:

1. Spectral Filtering Approaches:
   • Compute the Laplacian eigendecomposition ($L = U \Lambda U^\top$5), or—on large graphs—the first and last $L = U \Lambda U^\top$6 eigenpairs using Arnoldi-type sparse solvers ($L = U \Lambda U^\top$7).
   • Normalize eigenvalues to $L = U \Lambda U^\top$8.
   • Choose Chebyshev order $L = U \Lambda U^\top$9, PE dimension $U$0, and initialize $U$1 coefficients.
   • Synthesize $U$2, multiply with eigenvectors to form node PEs, and concatenate with node features.
   • Optimize end-to-end with the graph model using cross-entropy or task-specific losses, along with regularization (e.g., $U$3, $U$4 on $U$5).
2. GNN-based LLPE:
   • Seed each node with $U$6 random scalars (R-PEARL) or standard basis vectors (B-PEARL) as initial features.
   • Run a shared $U$7-layer GNN for each sample/input, apply statistical pooling (mean or sum) to enforce permutation equivariance.
   • For large graphs, select $U$8 and GNN filter order $U$9 according to the desired spectral complexity.
   • Normalize learned embeddings for tighter control of stability constants.
   • Feed output positional features to downstream GNNs or graph transformers for training.

Chebyshev LLPE methods scale efficiently up to $$\Lambda = \operatorname{diag}(\lambda_1,\ldots,\lambda_n)$$0, with sparse methods enabling partial spectrum approximation for $$\Lambda = \operatorname{diag}(\lambda_1,\ldots,\lambda_n)$$1. GNN-based LLPE attains near-linear scaling for sparse graphs and can be deployed under strict resource constraints.

6. Extensions and Open Directions

Significant extensions of LLPE frameworks include:

• Learning with generalized graph shift operators—normalized Laplacian, random-walk matrices, or edge-weighted variants—in the same spectral or message-passing framework (2502.01122).
• Combining absolute positional encodings (LLPE) with relative or pairwise-structural embeddings, enabling richer graph representations.
• Task-dependent initializations or anchor-based distributions for random or basis GNN initialization, focusing on motifs or local substructures.
• Development of scalable, incremental, or streaming variations for dynamic graphs.
• Extending LLPE to directed, heterogeneous, or hypergraph settings where traditional Laplacian theory does not readily apply (Huang et al., 2023).
• Investigation of optimal spectral truncation (choice of $$\Lambda = \operatorname{diag}(\lambda_1,\ldots,\lambda_n)$$2) and filter order $$\Lambda = \operatorname{diag}(\lambda_1,\ldots,\lambda_n)$$3 to balance computational tractability with information richness.
• Theoretical characterization of LLPE under non-uniform graph size distributions and in the presence of high-multiplicity eigenvalues.

A plausible implication is that further integration of spectral and structure-aware encoding, coupled with adaptivity to both global and local graph variation, will underpin advances in generalizable, scalable, and robust graph learning.

7. LLPE in Coordinate-MLPs and Continuous Domains

Adaptations of LLPE beyond standard graphs have been proposed, most notably for coordinate-MLPs. Here, LLPE regularizes instance-specific embedding parameters (e.g., RBF widths $$\Lambda = \operatorname{diag}(\lambda_1,\ldots,\lambda_n)$$4), via graph-Laplacian penalties on a similarity graph over embedded coordinates (Ramasinghe et al., 2021). This approach:

• Uses a super-Gaussian radial basis embedding with coordinate-dependent $$\Lambda = \operatorname{diag}(\lambda_1,\ldots,\lambda_n)$$5 fit via a low-degree polynomial in the local Jacobian norm of the target function.
• Incorporates a Laplacian penalty to enforce embedding smoothness adaptively, balancing memorization and generalization.
• Results in more stable, generalizable, and plug-and-play intermediate positional embeddings for deep coordinate-based models, outperforming both hand-tuned and end-to-end randomly initialized Fourier features.

• (2502.01122): Learning Efficient Positional Encodings with Graph Neural Networks (PEARL)
• (Ito et al., 29 Apr 2025): Learning Laplacian Positional Encodings for Heterophilous Graphs
• (Huang et al., 2023): On the Stability of Expressive Positional Encodings for Graphs (SPE)
• (Dwivedi et al., 2021): Graph Neural Networks with Learnable Structural and Positional Representations (LSPE)
• (Wijesinghe et al., 22 Feb 2025): Graph Self-Supervised Learning with Learnable Structural and Positional Encodings (GenHopNet/StructPosGSSL)
• (Ramasinghe et al., 2021): Learning Positional Embeddings for Coordinate-MLPs

LLPE synthesizes advances from spectral graph theory, nonlinear permutation-equivariant learning, and positional embedding design, yielding state-of-the-art, theoretically grounded, and scalable encodings for a broad set of modern graph learning problems.