Laplacian Eigenvector Gradient Orthogonalization (LEGO)

• Laplacian Eigenvector Gradient Orthogonalization (LEGO) is a spectral method that estimates tangent spaces by orthogonalizing gradients of low-frequency graph Laplacian eigenvectors.
• It employs a local SVD-based orthogonalization of the gradient matrix, using global spectral geometry to overcome the scale trade-offs inherent in local PCA.
• LEGO shows improved performance in manifold learning, boundary detection, and intrinsic dimension estimation compared to traditional methods.

Laplacian Eigenvector Gradient Orthogonalization (LEGO) is a spectral method for tangent-space estimation on noisy point clouds and sampled manifolds. It was introduced as an alternative to Local PCA (LPCA), with the central premise that tangent directions can be recovered more robustly by orthogonalizing the gradients of low-frequency graph Laplacian eigenvectors than by relying solely on local covariance structure. In this formulation, global spectral geometry is used to guide local differential estimation, and the resulting tangent spaces are reported to improve downstream tasks including manifold learning, boundary detection, and local intrinsic dimension estimation (Kohli et al., 2 Oct 2025).

1. Problem setting and motivation

The target problem is the estimation of tangent spaces from noisy samples of an underlying manifold. The standard baseline is LPCA: for each data point, one finds its $k$-nearest neighbors, computes the local covariance, and uses the top $d$ eigenvectors as a tangent basis. The difficulty is the classical scale trade-off: small neighborhoods reduce curvature bias but are noisy, whereas large neighborhoods suppress noise but incur bias from curvature, reach, and nonlinearity. The optimal neighborhood size depends on geometric quantities and noise level that are typically unknown (Kohli et al., 2 Oct 2025).

LEGO replaces this purely local strategy with a spectral one. Its key observation is that graph Laplacian eigenvectors encode global structure of the data, and that the gradients of low-frequency eigenvectors can be used as tangent features. The paper identifies a specific spectral phenomenon: low-frequency Laplacian eigenvectors tend to have gradients aligned with the manifold tangent bundle, while eigenvectors deeper in the spectrum are more likely to vary in normal or noise directions. On that basis, LEGO estimates the tangent space at each point by computing gradients of low-frequency eigenvectors and orthogonalizing them (Kohli et al., 2 Oct 2025).

This changes the locus of regularization. LPCA regularizes by neighborhood size; LEGO regularizes by spectral truncation, through the choice of how many low-frequency eigenvectors are retained. A plausible implication is that LEGO trades local scale sensitivity for spectral model selection.

2. Graph Laplacian construction and spectral coordinates

Let

$X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$

be the noisy point cloud. The method uses kernel-based graph Laplacians, including the random walk kernel, self-tuned kernel, and doubly stochastic kernel. In the random-walk construction used in the theory section, the Gaussian-kernel adjacency is

$$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$

with clean-data analogue

$A_{ij} = o_s(Y_i - Y_j).$

The degree variables are

$$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$

with corresponding degree matrices $D$ and $\hat D$. The normalized adjacency matrices are

$$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$

and the random-walk graph Laplacians are

$$L = I_n - D^{-1}K,\qquad \hat L = I_n - \hat D^{-1}\hat K.$$

The method then computes the first $d$0 eigenvectors of the graph Laplacian and retains the first $d$1 low-frequency eigenvectors $d$2 for tangent estimation (Kohli et al., 2 Oct 2025).

The role of these eigenvectors is not to provide a final embedding directly, but to furnish scalar fields whose gradients reflect local geometric directions. In that sense, LEGO uses spectral coordinates as differential probes rather than as clustering coordinates or as a basis for graph signal decomposition.

3. Gradient estimation and orthogonalization

For each eigenvector $d$3, the objective is to estimate the gradient at each data point: $d$4 Using the $d$5-nearest neighbors $d$6 of $d$7, LEGO fits a local linear model based on centered neighbors $d$8 and centered eigenvector values $d$9: $X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$0

The paper represents the gradient estimate as

$X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$1

where $X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$2 is an orthonormal basis for the span of the first $X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$3 Laplacian eigenvectors, and $X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$4 is found by least squares. The optimization problem is

$X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$5

subject to

$X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$6

Using orthonormality $X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$7, the least-squares solution is

$X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$8

so the gradient estimate becomes $X = [X_1,\dots,X_n] \in \mathbb{R}^{p\times n}$9 (Kohli et al., 2 Oct 2025).

At each point $$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$0, the estimated gradients of the retained eigenvectors are stacked into

$$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$1

LEGO then computes the singular value decomposition

$$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$2

and defines the tangent estimator as the matrix of the top $$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$3 left singular vectors,

$$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$4

If $$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$5 is unknown, the method estimates it through an explained-variance threshold: $$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$6 Algorithmically, the procedure is: compute the first $$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$7 Laplacian eigenvectors; estimate gradients of the first $$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$8 low-frequency eigenvectors; stack these gradients into $$A_{ij} = o_s(X_i - X_j) = \exp\!\left(-\frac{\|X_i - X_j\|^2}{s^2}\right),$$9; and take the top $A_{ij} = o_s(Y_i - Y_j).$0 left singular vectors as the tangent basis (Kohli et al., 2 Oct 2025).

The term “orthogonalization” in LEGO refers to this SVD-based extraction of an orthonormal basis from a matrix of estimated gradients. It is therefore a local orthogonalization of gradient information, not an additional orthogonality condition imposed on the Laplacian eigenvectors themselves.

4. Theoretical basis

The first theoretical justification is differential geometric and is formulated on a tubular neighborhood $A_{ij} = o_s(Y_i - Y_j).$1 of a manifold $A_{ij} = o_s(Y_i - Y_j).$2. The noisy observations are modeled as lying in the tube, with perturbations in normal directions. The analysis distinguishes the horizontal space, which approximates the true tangent space of the clean manifold, from the vertical space, which corresponds to normal or noise directions. For $A_{ij} = o_s(Y_i - Y_j).$3, with $A_{ij} = o_s(Y_i - Y_j).$4 denoting derivatives along manifold coordinates and $A_{ij} = o_s(Y_i - Y_j).$5 derivatives along normal coordinates, the horizontal and vertical energies are

$A_{ij} = o_s(Y_i - Y_j).$6

The normalized versions are

$A_{ij} = o_s(Y_i - Y_j).$7

Theorem 3 is summarized as saying that the Laplacian eigenvalue $A_{ij} = o_s(Y_i - Y_j).$8 of an eigenfunction on the tube is controlled by horizontal and vertical energies, with the vertical term weighted more heavily when the tube is thin. After simplification, the paper states that if $A_{ij} = o_s(Y_i - Y_j).$9 and suitable bounds on $$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$0 hold, then

$$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$1

The stated consequence is that if an eigenfunction has large variation in normal directions, its eigenvalue must be large; therefore, low eigenvalues correspond to eigenfunctions whose gradients are mostly horizontal, hence aligned with the manifold tangent bundle. A converse-type result extends eigenfunctions of the base manifold to the tube with controlled Dirichlet energy, and Corollary 7 ensures the existence of tube eigenfunctions with small vertical energy when the manifold has small eigenvalues (Kohli et al., 2 Oct 2025).

The second justification is random-matrix theoretic and addresses robustness to sub-Gaussian noise. The observed data satisfy

$$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$2

where $$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$3 are clean points and $$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$4 are i.i.d. centered sub-Gaussian noise vectors with variance proxy $$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$5, satisfying

$$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$6

for all $$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$7. The paper states that this generalizes tubular normal noise when $$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$8 is supported in the normal directions. Under the condition

$$d_i = \sum_{j=1}^n A_{ij}, \qquad \hat d_i = \sum_{j=1}^n \hat A_{ij},$$9

Theorem 10 proves, for large $D$0, that

$D$1

with high probability. Lemma 11 shows that under bounded support of clean points,

$D$2

with high probability. Theorem 12 gives

$D$3

and Theorem 13 yields

$D$4

with high probability. By Davis–Kahan, if the eigengaps of the clean Laplacian do not shrink too fast, then the eigenvectors of $D$5 and $D$6 remain close; Corollary 14 states that for fixed $D$7, if

$D$8

then

$D$9

up to a sign choice, with high probability (Kohli et al., 2 Oct 2025).

Together, these arguments support the paper’s central claim: low-frequency eigenvectors are geometrically tangent-aligned and statistically stable, while deeper eigenvectors are more sensitive to normal variation and spectral instability.

5. Empirical behavior and downstream tasks

The empirical evaluation compares LEGO against LPCA on noisy synthetic and real datasets. Tangent-space accuracy is measured using principal angles between estimated and true tangent subspaces: $\hat D$0 The experiments also assess manifold learning or intrinsic parameterization, boundary detection, and local intrinsic dimension estimation (Kohli et al., 2 Oct 2025).

On a high-aspect-ratio Swiss roll, the dataset has $\hat D$1 points in $\hat D$2 with additive uniform noise in the normal direction, and LEGO uses $\hat D$3, $\hat D$4, and $\hat D$5. The reported result is that LEGO yields much lower tangent-space discrepancy than LPCA. LPCA degrades rapidly with noise, whereas LEGO remains stable. The downstream effects include better 2D parameterization, better boundary detection, and better local intrinsic dimension estimates. The paper further states that LEGO concentrates variance in the first two principal directions, while LPCA spreads variance into spurious normal directions (Kohli et al., 2 Oct 2025).

On a truncated torus with $\hat D$6 and heteroskedastic normal noise, LEGO again significantly outperforms LPCA in tangent-space accuracy and in downstream tasks. On the Puppets dataset, a real image dataset with two rotating objects, the data are first reduced by PCA to $\hat D$7, and tangent spaces are estimated in the 2D torus setting. The reported result is that LEGO produces a coherent torn 2D embedding with correct gluing structure, whereas LPCA yields a less interpretable embedding. LEGO also better concentrates functional variance in the first two directions (Kohli et al., 2 Oct 2025).

The paper also reports a noise-ablation and hyperparameter-robustness study. Tangent-space error versus noise level shows LPCA deteriorating quickly, while LEGO remains robust. LEGO is reported to be stable over a broad range of $\hat D$8 and $\hat D$9. A stated limitation is that if $$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$0 is too small on a high-aspect-ratio manifold, the gradients may span only a lower-dimensional subspace; this point is highlighted in the Swiss roll example (Kohli et al., 2 Oct 2025).

6. Relation to adjacent Laplacian-eigenvector methods

LEGO belongs to a broader literature that uses graph Laplacian eigenvectors for geometric inference, but it is methodologically distinct from earlier approaches. One nearby line of work addresses the organization of graph Laplacian eigenvectors themselves. The paper "How can we naturally order and organize graph Laplacian eigenvectors?" argues that graph Laplacian eigenvalues should not generally be interpreted as “frequencies” except on very special graphs such as paths and cycles. It proposes organizing eigenvectors by a transport-based notion of similarity: convert each eigenvector $$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$1 into a probability mass function

$$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$2

define pairwise distances using Ramified Optimal Transport, and embed the resulting distance matrix into a low-dimensional Euclidean space using classical MDS. That paper is explicit that it does not develop a separate theory of gradient orthogonalization; the orthogonality present there is only the standard $$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$3-orthogonality of Laplacian eigenvectors, and the organizational principle is transport-based geometry rather than orthogonality (Saito, 2018).

A second adjacent line of work concerns scalable computation of Laplacian eigenvectors for spectral embedding. "Stochastic Gradient Descent for Spectral Embedding with Implicit Orthogonality Constraint" reformulates the standard problem

$$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$4

by writing $$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$5 and replacing the constrained formulation with

$$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$6

Here orthogonality is handled implicitly through the Cholesky factor $$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$7 of $$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$8, which makes mini-batch stochastic optimization over edges possible. Its objective is scalable spectral embedding and graph clustering, with the rows of the recovered $$K = D^{-1} A D^{-1}, \qquad \hat K = \hat D^{-1} \hat A \hat D^{-1},$$9 passed to $$L = I_n - D^{-1}K,\qquad \hat L = I_n - \hat D^{-1}\hat K.$$0-means; it is not a tangent-space estimator based on gradient orthogonalization at each data point (Gheche et al., 2018).

These comparisons clarify two common points of confusion. First, LEGO should not be identified with any use of Laplacian eigenvectors plus an orthogonality operation: in LEGO, the crucial object is the local gradient matrix $$L = I_n - D^{-1}K,\qquad \hat L = I_n - \hat D^{-1}\hat K.$$1, and the orthogonalization step is its SVD. Second, “low-frequency” in LEGO is tied to the low end of the graph-Laplacian spectrum used for tangent recovery, but adjacent literature shows that graph Laplacian eigenvalues are not universally interpretable as physical frequencies on arbitrary graphs. This suggests that LEGO’s spectral truncation is best understood as a data-geometric selection principle rather than as a direct analogue of Euclidean Fourier frequency ordering.