Laplacian Positional Encoding

• Laplacian positional encoding is a method that uses eigenfunctions of the Laplacian to embed geometric, boundary, and connectivity information in continuous domains and graphs.
• It employs classical eigendecomposition and finite element discretization to construct robust embeddings for PDE solvers, GNNs, and graph transformers.
• Advanced variants, like learnable Laplacian encodings and Magnetic Laplacians, enhance expressivity and stability, improving performance in heterogeneous and temporal graph tasks.

Laplacian positional encoding encompasses a family of methods that encode geometric, structural, or topological information for continuous domains and discrete graphs, leveraging the spectral properties of the Laplacian operator. These encodings have become foundational in geometrically informed neural architectures applied to partial differential equation (PDE) solvers, graph neural networks (GNNs), graph transformers, and temporal graph learning, due to their intrinsic capacity to represent connectivity, boundary conditions, and multi-scale structures.

1. Laplacian Eigenvalue Problems and Classical Formulations

The foundational principle of Laplacian positional encoding is the association of positions with the eigenfunctions or eigenvectors of a Laplacian operator, enforcing intrinsic geometry and boundary constraints. On Euclidean or manifold domains, the Laplace–Beltrami operator is central:

$$- \nabla^2 \phi(x) = \lambda \phi(x)\,, \quad \text{for}\ x \in \Omega_x,$$

subject to boundary conditions (Dirichlet, Neumann, or periodic):

• Dirichlet: $\phi(x) = 0$ on $\partial\Omega_x^{(DC)}$,
• Neumann: $\partial\phi/\partial n(x) = 0$ on $\partial\Omega_x^{(NM)}$,
• Periodic: $\phi$ periodic across domain faces.

For undirected graphs with adjacency $A$ and degree $D$, the graph Laplacian is $L = D - A$ (unnormalized) or $L_n = I - D^{-1/2} A D^{-1/2}$ (symmetric normalized). Its eigendecomposition

$L = U \Lambda U^\top$

yields eigenvalues $0 = \lambda_1 \le ... \le \lambda_n$ and orthonormal eigenvectors $U \in \mathbb{R}^{n \times n}$.

Classical Laplacian positional encoding for graphs assigns node $i$ the vector of its entries in the first $k$ nontrivial eigenvectors, i.e., $(u_2[i], ..., u_{k+1}[i])$, encoding global spatial or connectivity information. In the continuous case, the eigenfunctions $\phi_i(x)$ likewise provide variational harmonic features that encode geometry (Kast et al., 2023).

2. Numerical Discretization and Embedding Construction

For PDE domains, a finite-element discretization is applied:

• Select a high-order FE space $V_h$ (e.g., $P_3$ elements on a fine mesh) with basis $\{\varphi_i(x)\}$,
• Approximate eigenfunctions: $\phi(x) \approx \sum_i c_i \varphi_i(x)$,
• Formulate the weak problem and reduce to a generalized eigenproblem:

$K c = \lambda M c,$

where $$K_{ij} = \int \nabla \varphi_i \cdot \nabla \varphi_j$$, $M_{ij} = \int \varphi_i \varphi_j$.

Entries are ordered to retain the $K$ lowest eigenmodes; typically, K is selected based on a trade-off between expressivity and numerical cost (e.g., $K=10$ or $15$) (Kast et al., 2023).

For graphs, obtaining the first $k$ eigenvectors involves partial eigensolvers (e.g., Lanczos) for scalability, especially for large $n$ (2502.01122, Galron et al., 2 Jun 2025).

The positional embedding is defined as

$\Phi(x) = [\phi_1(x), ..., \phi_K(x)]^\top$

or for graphs

$$P_{\mathrm{LPE}} = U_{1:k} \in \mathbb{R}^{n \times k}$$

where each node or spatial coordinate receives a $k$-dimensional encoding.

3. Extensions: Spectral Filtering, p-Norm and Learnable Laplacian Encodings

Spectral Filtering and Generalization

Classical encodings use the lowest-frequency eigenfunctions, capturing "smooth" structure (community, geometry). For graphs with heterophily, this approach is suboptimal; often, task-relevant signals live in higher-frequency (large-eigenvalue) components (Ito et al., 29 Apr 2025).

To address this, learnable Laplacian positional encodings (LLPE) apply a trainable spectral filter to all eigenvectors:

$P_{\mathrm{LLPE}} = U W(\Lambda),$

with $$W(\Lambda) = \mathrm{diag}(h(\lambda_1 ; \theta), ..., h(\lambda_n ; \theta))$$ and $h$ parameterized (e.g., Chebyshev polynomials). This enables the model to adaptively weight spectral content. LLPEs have been shown to provably approximate a wide class of spectral distances between nodes, including commute-time and diffusion distances. Empirically, LLPEs provide up to 35% accuracy gains under strong heterophily and average 14% gains on real-world benchmarks compared to fixed LPEs (Ito et al., 29 Apr 2025).

A related generalization uses p-norms in the Laplacian embedding objective:

$$X^*_p = \arg\min_{X \in \mathbb{R}^{n \times k}} \sum_{i,j} A_{ij} \|X_{i,:} - X_{j,:}\|_p^p, \quad X^\top X = I_k$$

for various $p$. $p=2$ yields classical Laplacian eigenmaps; $p \to 1$ connects to minimal ratio cuts and Cheeger partitioning (Maskey et al., 2022). The embedding transitions from smooth (cluster-like) to piecewise-constant as $p$ decreases.

Efficient and Stable Alternatives

Direct eigendecomposition is computationally expensive, particularly as $n$ grows. The PEARL framework generates high-quality positional encodings using GNNs as nonlinear mappings from the graph shift operator's eigenvectors, initialized with random or standard basis node features and equilibrated over samples (2502.01122). PEARL achieves performance matching or surpassing full-spectrum Laplacian encodings, at one or two orders of magnitude lower cost; its stability is theoretically independent of eigengaps, avoiding the Davis–Kahan instability issues of raw eigenvectors.

4. Integration into Learning Architectures

In continuum PDE solvers, such as Evolutional Deep Neural Networks (EDNNs), variational harmonic features $\Phi(x)$ are concatenated (with any parametric dependence $\alpha$) to form the network input. Because the features inherit the same boundary conditions as the PDE, any resulting function $\hat{u}(\Phi(x))$ is guaranteed to obey those constraints, with homogeneous Dirichlet or Neumann boundary conditions transferring automatically through the embedding. For strict enforcement, subtraction at a boundary point can be used (Kast et al., 2023).

In GNNs and graph transformers, Laplacian positional encodings or their generalizations are concatenated to each node's features. Classical eigenvectors have sign ambiguity and instability under small graph perturbations, addressed by random sign-flipping during training or sign-invariant small networks (e.g., SignNet), and by stable neural encoders that process entire eigenspaces (Maskey et al., 2022, Huang et al., 2024).

Temporal graph learning extends Laplacian encodings to dynamic graphs via the supra-Laplacian, which builds a block-tridiagonal operator over stacked snapshot graphs (Galron et al., 2 Jun 2025). The supra-Laplacian's $k$ smallest eigenvectors simultaneously encode per-slice spatial arrangement and cross-slice temporal consistency, yielding strong improvements (up to 65% of cases, +1.1pp AUC on average) in dynamic link prediction tasks.

5. Empirical Performance, Expressivity, and Limitations

Empirical results consistently indicate:

• PDEs on complex domains: Variational harmonic features (first $K$ Laplacian modes) enable error reduction by 1-2 orders of magnitude versus random or Fourier features (Kast et al., 2023).
• Node classification and regression: Classical Laplacian PEs yield 5-10% accuracy improvements for tasks with strong global structure; LLPEs extending to the full spectrum achieve up to 35% gains in synthetic heterophilous graphs and 14% on real-world benchmarks (Ito et al., 29 Apr 2025).
• For graph regression (e.g., ZINC molecules): incorporating $p\approx1.2$ PEs sharpens substructure boundaries, improving mean absolute error for certain GNNs, though training becomes more sensitive (Maskey et al., 2022).
• Scalability: Classical LPEs, requiring $O(n^3)$ time and $O(n^2)$ memory, are limited to small graphs; PEARL and randomized or block Krylov eigensolvers (e.g., LOBPCG) achieve efficient encodings for $n$ up to $5 \times 10^4$, with up to $56\times$ runtime reduction without significant loss of accuracy (2502.01122, Galron et al., 2 Jun 2025).

However, raw truncated eigenvector encodings may poorly capture fine structure in highly heterogeneous or heterophilous regions; further, eigenvector instability (highlighted by the Davis–Kahan theorem) undermines robustness unless mitigated by aggregation or filtering (2502.01122).

6. Variants for Directed and Temporal Graphs

For directed graphs, the classical Laplacian lacks sensitivity to directional structure. The Magnetic Laplacian introduces a complex Hermitian operator $L_q$ parameterized by a "magnetic potential" $q$, where Multiple-q Magnetic Laplacian encodings concatenate the spectral information from multiple $q$ settings. This yields provable expressivity for bidirectional walk profiles: with $Q=L+1$ magnetic Laplacians, all bidirectional walk counts up to length $L$ are uniquely recoverable (Huang et al., 2024). Processing these embeddings requires stable, basis-invariant neural layers to resolve ambiguities.

In temporal graphs, the supra-Laplacian paradigm encodes both spatial and temporal consistency in a unified spectrum, outperforming naïve per-slice strategies. Scalable solvers (Krylov subspace, LOBPCG, or trajectory-concatenation) are essential for practical deployment on large-scale graphs (Galron et al., 2 Jun 2025).

7. Summary and Applications

Laplacian positional encoding provides a mathematically rigorous, geometry-aware mechanism for injecting spatial or relational structure into neural models for PDEs, graphs, and temporal networks. Core strengths include boundary condition enforcement, spectral (multi-scale) representation, and, in graph domains, provable gains in expressivity when compared to message-passing-only architectures—a benefit amplified by recent learnable and stable extensions. Limitations persist in scalability for high-order decompositions and robustness under perturbations when using raw eigenvectors, but innovations such as PEARL, LLPE, and Krylov-based methods provide effective solutions. Observed empirical gains, especially for tasks on complex domains or heterophilous graphs, underscore the central role of Laplacian-based encodings in contemporary geometric deep learning frameworks (Kast et al., 2023, Ito et al., 29 Apr 2025, Maskey et al., 2022, 2502.01122, Huang et al., 2024, Galron et al., 2 Jun 2025).