Classical Laplacian Position Encoding

• Classical Laplacian Positional Encoding is a method that assigns node embeddings via the spectral decomposition of the graph Laplacian, capturing both local and global graph structure.
• The approach formulates an optimization problem that minimizes the trace of the Laplacian quadratic form, extracting the smallest non-trivial eigenvectors to serve as positional encodings.
• Its integration in GNNs enhances expressive power and permutation equivariance, although challenges like sign and ordering ambiguities remain.

Classical Laplacian positional encoding is a method for assigning structural representations to nodes in a graph by computing node features from the spectral decomposition of the graph Laplacian. Widely used as an input to graph neural networks (GNNs), this scheme leverages the eigenvectors of the Laplacian to provide positional encodings that capture global and local connectivity properties of the graph, resulting in increased expressive power relative to message-passing architectures that lack such embeddings (Maskey et al., 2022).

1. Optimization Problem and Variational Formulation

Given an undirected, possibly weighted, graph $G = (V, E)$ with $n = |V|$ nodes, weighted adjacency matrix $A \in \mathbb{R}^{n \times n}$, and degree matrix $D = \mathrm{diag}(d_1,\dots,d_n)$, the unnormalized Laplacian is defined as

$L = D - A.$

Classical Laplacian positional encoding seeks an embedding $X \in \mathbb{R}^{n \times k}$, where each node $i$ is assigned the $k$-dimensional row vector $X_{i,:}$. The embedding is defined via the minimization problem:

$$\min_{X \in \mathbb{R}^{n \times k}} \sum_{i,j} a_{ij} \|X_{i,:} - X_{j,:}\|_2^2 \quad \text{s.t.} \quad X^\top D X = I_k.$$

This penalty enforces that adjacent nodes remain close in the embedded space while the constraint removes trivial constant solutions and fixes a canonical scale. The sum can be rewritten as a matrix trace,

$n = |V|$0

yielding the equivalent constrained optimization:

$n = |V|$1

2. Spectral Decomposition and Eigenvector Selection

The minimization is equivalent to a generalized eigenvalue problem. In the scalar case ($n = |V|$2), the task reduces to minimizing the generalized Rayleigh quotient,

$n = |V|$3

Stationarity conditions obtained via Lagrange multipliers yield

$n = |V|$4

the generalized eigenproblem. For invertible $n = |V|$5, this can be recast as a standard eigenproblem for the normalized Laplacians:

• Random-walk Laplacian: $n = |V|$6 so that $n = |V|$7,
• Symmetric normalized Laplacian: $n = |V|$8, using $n = |V|$9, yielding $A \in \mathbb{R}^{n \times n}$0.

For $A \in \mathbb{R}^{n \times n}$1, the minimizers are the $A \in \mathbb{R}^{n \times n}$2 smallest non-trivial generalized eigenvectors $A \in \mathbb{R}^{n \times n}$3, corresponding to the second through the $A \in \mathbb{R}^{n \times n}$4-th smallest eigenvalues.

3. Construction of Node Positional Encodings

After computing the eigenbasis $A \in \mathbb{R}^{n \times n}$5 of the generalized eigenproblem $A \in \mathbb{R}^{n \times n}$6, the classical Laplacian positional encoding $A \in \mathbb{R}^{n \times n}$7 is constructed by stacking the first $A \in \mathbb{R}^{n \times n}$8 non-trivial eigenvectors:

$A \in \mathbb{R}^{n \times n}$9

The constant eigenvector $D = \mathrm{diag}(d_1,\dots,d_n)$0 for $D = \mathrm{diag}(d_1,\dots,d_n)$1 is omitted, as it fails to distinguish nodes (being proportional to $D = \mathrm{diag}(d_1,\dots,d_n)$2). Lower-indexed (smaller eigenvalue) eigenvectors vary more smoothly over the graph and thus capture larger-scale connectivity patterns due to the spectral gap ordering $D = \mathrm{diag}(d_1,\dots,d_n)$3.

4. Theoretical Properties and Inherent Limitations

Classical Laplacian positional encoding exhibits several advantageous properties:

• Permutation equivariance: Under a node permutation $D = \mathrm{diag}(d_1,\dots,d_n)$4, the Laplacian and its eigenvectors are permuted likewise ($D = \mathrm{diag}(d_1,\dots,d_n)$5, $D = \mathrm{diag}(d_1,\dots,d_n)$6), ensuring that $D = \mathrm{diag}(d_1,\dots,d_n)$7 is equivariant (or, as a sorted multiset up to sign, invariant) under relabeling.
• Sign ambiguity: Each eigenvector is determined only up to sign ($D = \mathrm{diag}(d_1,\dots,d_n)$8), so downstream models must be either sign-invariant or resolve this ambiguity explicitly, for example with absolute values or purpose-built architectures (e.g., SignNet).
• Ordering ambiguity: When eigenvalues have multiplicity greater than one, the choice of orthonormal eigenbasis within each eigenspace is non-unique, resulting in potential basis rotations within repeated blocks.
• Expressivity enhancement: Incorporating at least two non-trivial eigenvectors into standard message-passing GNNs (1-WL-equivalent) provably increases their expressive power; specifically, such augmented GNNs can distinguish graph pairs (including regular graphs) that are indistinguishable by the 1-WL test.

5. Computational Considerations for Large Graphs

Direct eigendecomposition of the $D = \mathrm{diag}(d_1,\dots,d_n)$9 Laplacian requires $L = D - A.$0 time, rendering dense computations infeasible for large-scale graphs. Practical large-graph approaches rely on sparse iterative methods such as Lanczos or Arnoldi algorithms to extract the first few eigenpairs. Typical implementations utilize:

Framework	Functionality	Applicability
Python/SciPy	scipy.sparse.linalg.eigsh(L, k=5, which='SM') computes smallest eigenvalues	Sparse, medium-sized graphs
PyTorch	torch.linalg.eig / torch.linalg.eigh for dense; or custom Lanczos routines for sparse	Dense/small, or via SciPy
PyTorch-Geometric	get_laplacian_pe(edge_index, edge_weight, num_pe) extracts first num_pe eigenvectors	GNN pipelines

For massive graphs, approximation via Nystrom methods, Chebyshev polynomial filters, or graph coarsening with interpolation is common.

6. Applications and Impact on Graph Representation Learning

Classical Laplacian positional encoding forms a core architectural component in enhancing GNN expressive power, overcoming several inherent limitations of message passing by providing richer, globally aware node features. The permutation-equivariant property ensures representations are meaningful irrespective of node labeling. When incorporated as node embeddings, these Laplacian features allow GNNs to distinguish structures beyond the reach of the Weisfeiler–Lehman (1-WL) isomorphism test, thus expanding the range of graph-structured data they can model and learn effectively (Maskey et al., 2022). Sign and ordering ambiguities, however, remain considerations for practical deployment and network architecture design.