Functional Latent Alignment (FuLA)

• Functional Latent Alignment is a framework that defines alignment via functional maps and spectral decompositions to robustly match latent representations across domains.
• It leverages methodologies such as regularized least-squares, end-to-end model stitching, dual-pass spectral encoding, and GP-LVM for optimizing alignment.
• FuLA achieves state-of-the-art performance in applications like graph node matching, bilingual retrieval, and sequence alignment, ensuring consistency and interpretability.

Functional Latent Alignment (FuLA) is a unifying conceptual and algorithmic framework for aligning high-dimensional latent representations across disparate data domains, neural architectures, graphs, and sequential observations. Its central insight is to formulate alignment as a problem in functional, rather than pointwise or coordinate, correspondence—leveraging the geometry of function spaces, spectral decompositions, and, where appropriate, probabilistic generative models. FuLA has emerged in several independent but convergent research lines, encompassing spectral geometry for neural representations, end-to-end functional consistency in model stitching, dual-pass spectral encoding for graph (and cross-modal) alignment, and nonparametric Bayesian inference of warping functions in sequential data.

1. Mathematical Foundations of Functional Latent Alignment

The theoretical core of FuLA is the functional map formalism, initially developed within spectral geometry for shape analysis and subsequently transferred to neural latent spaces. For two compact metric spaces (e.g., latent manifolds) $\mathcal{M}$ and $\mathcal{N}$, and their $L^2$ function spaces, a (pointwise) correspondence $T:\mathcal{M}\to\mathcal{N}$ induces a pullback linear operator $T_F: L^2(\mathcal{M})\to L^2(\mathcal{N})$ defined as $T_F[f] = f \circ T^{-1}$. The functional map $C$ is the matrix representation of $T_F$ in appropriate orthonormal bases.

Spectral decomposition—using the Laplace–Beltrami operators $\Delta_\mathcal{M}$ and $\Delta_\mathcal{N}$ and their respective eigenbases—enables representing functions (and thus correspondences) compactly. Truncation to the first $\mathcal{N}$0 basis functions yields a low-rank approximation and reduces alignment to inferring a $\mathcal{N}$1 linear operator $\mathcal{N}$2, with structure governed by the geometry of $\mathcal{N}$3 and their spectra. This philosophy informs both the algorithmic designs in spectral neural alignment (Fumero et al., 2024, Behmanesh et al., 11 Sep 2025) and the probablistic generative model in time-series alignment (Kazlauskaite et al., 2018).

2. Methodological Instantiations: From Spectral Maps to Probabilistic Models

2.1 Spectral Geometry and Regularized Least-Squares

The spectral FuLA pipeline (Fumero et al., 2024) proceeds by constructing $\mathcal{N}$4-nearest neighbor graphs over latent samples from two spaces, computing normalized graph Laplacians, and extracting the leading $\mathcal{N}$5 eigenpairs. Descriptor functions—either geometric (e.g., geodesic distance) or semantic (e.g., label indicators)—are projected into these spectral bases. The alignment objective is to find $\mathcal{N}$6 minimizing:

$\mathcal{N}$7

where $\mathcal{N}$8 are spectral descriptor matrices, the commutativity regularizers enforce geometric consistency, and $\mathcal{N}$9 are descriptor-induced multiplication operators. Closed-form or iterative solvers recover $L^2$0; an optional ZoomOut refinement iteratively sharpens alignment by increasing $L^2$1 (Fumero et al., 2024).

2.2 End-to-End Functional Consistency in Model Stitching

In model stitching (Athanasiadis et al., 26 May 2025), FuLA defines the optimality of an affine stitching transform $L^2$2 not simply by matching activations at the stitch layer (as in Direct Matching), but by enforcing that for all subsequent layers $L^2$3 in the receiving network $L^2$4, the transformed activations $L^2$5 are close to $L^2$6. The total loss is a weighted sum over these layer-wise Frobenius distances, normalized to account for scale invariance. This multi-level hinting extends knowledge distillation ideas to a fully unsupervised, task-agnostic, and propagation-consistent setting.

2.3 Dual-Pass Spectral Encoders and Latent Communication in Graphs

For unsupervised graph alignment (Behmanesh et al., 11 Sep 2025), FuLA is realized as a two-stage process: a dual-pass spectral encoder applies separate low- and high-frequency filters, yielding embeddings $L^2$7 that capture both smooth (structural) and discriminative (detailed) information. A geometry-aware functional map module projects these embeddings into spectral bases and learns a pair of approximately bijective, orthogonal maps $L^2$8 between them using regularized least-squares, bijectivity, and (partial) isometry losses. This jointly addresses oversmoothing and latent-space collapse in GNNs, and supports alignment across graphs and modalities.

2.4 Gaussian Process Latent Variable Alignment

In time-series and sequence modeling (Kazlauskaite et al., 2018), FuLA is formalized as a hierarchical probabilistic model. Each sequence is generated by an unknown latent function evaluated on a warped input domain; the unwarped "aligned" pseudo-observations are governed by a latent manifold via a GP-LVM. The model infers both the amplitude functions $L^2$9 and monotonic warps $T:\mathcal{M}\to\mathcal{N}$0, with monotonicity strictly enforced through softmax–cumsum parameterization. Variational inference or MAP optimization integrates over generating functions, warps, and latent locations, achieving joint alignment and clustering of heterogeneous sequences.

3. Optimization Algorithms and Practical Pipelines

Algorithmic details across FuLA variants share reliance on spectral (or probabilistic) bases, compact (often closed-form) optimization, and domain-aware regularization.

• Spectral Regularized Least-Squares: In (Fumero et al., 2024), the constrained quadratic objective in $T:\mathcal{M}\to\mathcal{N}$1 leads to efficient solvers. Empirically, hyperparameter grids for regularization terms are tractable due to modest matrix dimensions ($T:\mathcal{M}\to\mathcal{N}$2).
• Layer-Wise Affine Map Estimation: In model stitching (Athanasiadis et al., 26 May 2025), $T:\mathcal{M}\to\mathcal{N}$3 is first initialized by solving the DM objective via Moore-Penrose pseudoinverse on a sample batch, then refined with stochastic optimization (Adam) solely on the affine parameters and batch norm statistics, avoiding updates to the underlying models.
• End-to-End Training with Spectral Encoders: In graph alignment (Behmanesh et al., 11 Sep 2025), dual-pass encoder parameters and functional maps are updated simultaneously via gradient-based optimization over four loss terms—reconstruction, functional alignment, bijectivity, and orthogonality—facilitating end-to-end alignment.
• Probabilistic Inference with Constraints: The GP-based method (Kazlauskaite et al., 2018) alternates maximization steps over pseudo-observations, latent manifolds, warp parameters, and kernel hyperparameters; monotonicity is hard-constrained in the warp parametrization.

4. Theoretical Properties and Empirical Behavior

FuLA formalizations exhibit several theoretically desirable traits:

• Isometry and Orthogonality: In the spectral setting, perfect isometries induce orthogonal maps $T:\mathcal{M}\to\mathcal{N}$4 commuting with Laplacian spectra. Deviation from isometry is quantifiable via the off-diagonal mass of $T:\mathcal{M}\to\mathcal{N}$5, supporting interpretable global similarity scores $T:\mathcal{M}\to\mathcal{N}$6 that are robust to nuisances such as linear perturbations or translation (Fumero et al., 2024).
• Propagation Consistency: Multi-level hinting in neural functional alignment ensures that stitching-induced feature distortions are detected at every downstream layer, suppressing spurious matches and overfitting to decision boundaries, which affect task loss-based and soft-label-based baselines (Athanasiadis et al., 26 May 2025).
• Bijectivity and Partial Isometry in Graph Alignment: Explicit regularization for bijectivity ($T:\mathcal{M}\to\mathcal{N}$7) and orthogonality ensures that the cross-space correspondence preserves both node uniqueness and local relational structure, remedying the node indistinctiveness and oversmoothing seen in prior GNN aligners (Behmanesh et al., 11 Sep 2025).
• Probabilistic Uncertainty and Clustering: In GP-LVM based sequence alignment, the Bayesian treatment yields uncertainty quantification over warps and enables unsupervised grouping of sequences sharing aligned latent manifolds (Kazlauskaite et al., 2018).

5. Applications Across Domains and Modalities

FuLA methodologies are validated across a diverse range of tasks:

• Representation Matching: In neural networks, FuLA achieves 99.8% accuracy in matching corresponding layers across randomly initialized CNNs, outperforming both CKA and CCA (Fumero et al., 2024).
• Robustness to Perturbation: Under linear shifts and adversarial modifications, FuLA similarity scores remain stable, while those from CKA degrade (Fumero et al., 2024, Athanasiadis et al., 26 May 2025).
• Model Stitching: FuLA-based maps, often with as few as 5–50 anchors or even only label indicator descriptors, provide >60% improvement in zero-shot encoder–decoder stitching and are robust to shortcut artifacts and partial class shifts (Fumero et al., 2024, Athanasiadis et al., 26 May 2025).
• Bilingual/Multimodal Retrieval: Spectral FuLA pipelines achieve near-perfect MRR with minimal supervision in bilingual embedding alignment and vision-language graph retrieval scenarios, outperforming OT, LocalCKA, and graph matching baselines (Fumero et al., 2024, Behmanesh et al., 11 Sep 2025).
• Graph Node Alignment: Graph-based FuLA consistently outperforms NetSimile, FINAL, GAlign, and other unsupervised benchmarks under structural noise and cross-modal alignment problems (Behmanesh et al., 11 Sep 2025).
• Sequential Data Alignment: Bayesian FuLA for sequence data yields the lowest alignment and warping error on synthetic and cluster-structured datasets relative to SRVF, GTW, and other temporal warping formalisms (Kazlauskaite et al., 2018).

Application	FuLA Variant	Key Metric(s) / Effect
Layer matching	Spectral LFM, DM+Hints	99.8% pairwise diag., CKA: 99.6%
Model stitching	Multi-hint, spectral	>60% gain on stitching, robust to shifts
Graph alignment	Dual-pass + FM	Hit@1: 88.6% (ACM–DBLP), SOTA
Bilingual retrieval	Spectral FM	MRR ≈ 0.99 (5 anchors), baseline ≈ 0.85
Sequence alignment	GP-LVM-based FuLA	Best MSE vs. SRVF, CTW, GTW

6. Limitations, Interpretability, and Future Prospects

FuLA frameworks operate under the constraint that ground-truth functional similarity is unobservable; all metrics are proxies, and empirical properties may not transfer beyond tested domains. Current experiments are primarily in vision, graphs, and short sequences, with predominant architectures being CNNs and GCN variants. Extending FuLA to NLP, speech, segmentation, detection, novel architectures such as ViTs, and continual or transfer learning settings is an open goal (Athanasiadis et al., 26 May 2025).

Potential directions include learning adaptive weighting schemes for hint losses, exploring nonlinear or low-capacity stitching transforms, and integrating advanced function-consistency objectives (e.g., Jacobian or contrastive matching). For sequence alignment, incorporating contextual, piecewise, or periodic priors on warps, and employing fully Bayesian variational inference are areas of ongoing development (Kazlauskaite et al., 2018).

7. Unifying Perspective and Significance

Functional Latent Alignment offers a geometrically and probabilistically principled framework for aligning representations without reliance on task loss, manifold charts, or dense anchors, thus encompassing spectral, gradient-based, and Bayesian approaches under a common abstraction. By embedding latent structures as functions and employing geometry- or process-consistent regularization, FuLA supports robust, interpretable, and minimally supervised correspondence. This concept underlies state-of-the-art results in neural representation comparison, model merging, information retrieval, graph node mapping, and sequence alignment, with evidence of generalization to cross-modal and heterogeneous data settings (Fumero et al., 2024, Athanasiadis et al., 26 May 2025, Behmanesh et al., 11 Sep 2025, Kazlauskaite et al., 2018).