Complex Interpolation of Matrices with an application to Multi-Manifold Learning

Abstract: Given two symmetric positive-definite matrices $A, B \in \mathbb{R}^{n \times n}$, we study the spectral properties of the interpolation $A^{1-x} B^x$ for $0 \leq x \leq 1$. The presence of `common structures' in $A$ and $B$, eigenvectors pointing in a similar direction, can be investigated using this interpolation perspective. Generically, exact log-linearity of the operator norm $|A^{1-x} B^x|$ is equivalent to the existence of a shared eigenvector in the original matrices; stability bounds show that approximate log-linearity forces principal singular vectors to align with leading eigenvectors of both matrices. These results give rise to and provide theoretical justification for a multi-manifold learning framework that identifies common and distinct latent structures in multiview data.

• The paper presents identifiability and stability theorems proving that log-linear singular value evolution reveals common eigenvectors across SPD matrices.
• It employs a kernel interpolation framework to effectively differentiate shared and modality-specific latent geometries in multi-view datasets.
• The proposed SVFD method uses standard kernel construction and SVD to quantify spectral correspondences, offering practical insights for manifold learning.

Complex Interpolation of Positive Definite Matrices and its Role in Multi-Manifold Learning

Introduction

This work presents an in-depth spectral analysis of interpolations between symmetric positive-definite (SPD) matrices, focusing on the family $A^{1-x} B^x$ for $A, B \in \mathbb{R}^{n \times n}$ and $x \in [0,1]$. The central question is whether and how the spectral structures—specifically, eigenvectors—of $A$ and $B$ relate, even in cases where $A$ and $B$ arise from distinct but related data modalities. The authors establish that the evolution of the singular values of $A^{1-x} B^x$ encodes information regarding shared and distinct latent geometries, and they provide precise identifiability and stability results for detecting common eigenspaces. The paper connects these results to multi-manifold learning, introducing and justifying a kernel interpolation framework for the discovery of common and distinct latent variables in multiview datasets.

Figure 1: Two 2D projections of a 3D cow point cloud. The respective 2D projections are converted into SPD matrices; the challenge is to recover their shared geometric structure via spectral analysis.

Spectral Properties of Matrix Interpolation

Given SPD matrices $A$ and $B$, consider the analytic path $A, B \in \mathbb{R}^{n \times n}$0. For each $A, B \in \mathbb{R}^{n \times n}$1, one examines the singular values $A, B \in \mathbb{R}^{n \times n}$2 as a function of $A, B \in \mathbb{R}^{n \times n}$3. The key technical contributions are:

• Identifiability of Common Eigenvectors: Exact log-linearity in $A, B \in \mathbb{R}^{n \times n}$4 (or, more generally, any principal singular value trajectory) across $A, B \in \mathbb{R}^{n \times n}$5 is proved to be equivalent, under genericity conditions, to a shared eigenvector between $A, B \in \mathbb{R}^{n \times n}$6 and $A, B \in \mathbb{R}^{n \times n}$7.
• Stability Bounds: If the log-linearity is only approximate (i.e., the trajectory is close to a line), the associated left and right principal singular vectors at interpolation point $A, B \in \mathbb{R}^{n \times n}$8 are necessarily close—measured in inner product—to the leading eigenvectors of $A, B \in \mathbb{R}^{n \times n}$9 and $x \in [0,1]$0, respectively. The bounds obtained depend explicitly on the spectral gap and the deviation from unit norm.

These results are anchored by explicit theorems, leveraging complex interpolation, perturbation theory, and properties of holomorphic operator-valued functions.

Application to Multi-Manifold and Multimodal Learning

The theoretical contributions are connected to the analysis of datasets comprising aligned observations of related, but non-identical, manifolds—common in multimodal sensing, sensor fusion, or multi-view learning. Each observation channel produces a kernel (affinity) matrix:

$x \in [0,1]$1

where $x \in [0,1]$2 and $x \in [0,1]$3 are then normalized to be SPD. The interpolation $x \in [0,1]$4 is used to define a "singular value flow diagram" (SVFD), which visualizes the evolution of spectral components as one moves from $x \in [0,1]$5 to $x \in [0,1]$6.

Figure 2: Two point clouds sampled from 2D cylinders in $x \in [0,1]$7, where the height is a shared (common) variable while the angular variable is distinct in each view.

Evolution of singular values along this path reveals whether their corresponding eigenvectors capture shared or distinct latent structures. Shared variables generate nearly log-linear (straight) trajectories, whereas modality-specific variables result in curved, non-linear trajectories.

Figure 3: SVFD for the cylinders, highlighting the empirical singular values interpolating between the second eigenvectors. The highlighted (yellow) trajectory is nearly log-linear, confirming aligned geometry.

Figure 4: SVFD for the same data, here highlighting a trajectory linked to a modality-specific (distinct) eigenvector. The trajectory is highly non-linear, indicating non-alignment.

The SVFD not only distinguishes common and distinct latent variables but also provides a quantitative signature for the degree of alignment. Components corresponding to shared geometry maintain log-linearity, while distinct components show pronounced nonlinearity.

Theoretical Implications

On the theoretical side, the work generalizes classical results (e.g., the Heinz-Kato-Löwner inequalities) and shows that, for generic pairs of SPD matrices, the geometry of $x \in [0,1]$8 allows sharp spectral correspondence detection. The stability bounds guarantee that in the presence of approximate log-linearity, actual eigenvector alignment holds up to a quantified error. These results provide theoretical underpinnings for using such interpolations in manifold learning, kernel fusion, and geometric data analysis.

Practical Methodology

The SVFD algorithm is computationally straightforward, requiring only standard kernel construction and singular value decomposition along a discrete grid over $x \in [0,1]$9. The authors further give spectral analytic formulas for expected behavior in synthetic settings (e.g., product cylinders) and validate their method empirically.

Key algorithmic steps:

1. Construct normalized SPD kernels for each modality/view.
2. For a set of interpolation points $A$0, compute $A$1.
3. Compute leading singular values and their associated (left/right) singular vectors at each $A$2.
4. Plot the log-singular values as a function of $A$3 to form the SVFD.
5. Identify shared/distinct latent variables via the shape of trajectories (log-linearity vs. curvature).

Extensions and Future Directions

While the focus is on the geometric mean path $A$4, the paper notes that other interpolation schemes (e.g., affine-invariant Riemannian geodesics, symmetric interpolations like $A$5) are possible and may be studied via similar machinery. This opens a research direction into which interpolations optimally preserve/identify latent geometric alignment across data views or kernel-derived representations.

Potential future developments include:

• Systematic comparison of different matrix path interpolations on multimodal datasets.
• Extension to more than two views/modalities.
• Application to out-of-distribution and noisy data, including robustness analysis.
• Connections with nonparametric and kernel CCA, as well as representations in nonlinear dynamical systems.

Conclusion

The work provides both a rigorous theoretical framework and a practical methodology connecting the spectral analysis of matrix interpolations to the discovery of common and distinct structures in multiview or multimodal data. The identifiability and stability theorems for eigenvector alignment via singular value evolution support the design of principled SVFD-based manifold learning algorithms. This advances the methodology for manifold correspondence in modern data-driven settings where multiple, heterogeneous sensing modalities are prevalent.