Cross-modal Geometric Alignment

• Cross-modal Geometric Alignment is a family of methods that mitigate geometric, statistical, and relational mismatches between diverse modalities for effective information fusion.
• It employs techniques like manifold learning, optimal transport, and orthogonal mapping to align latent spaces while preserving both local and global structural features.
• Empirical evidence shows that CGA improves accuracy and efficiency in tasks such as retrieval, dense correspondence, and anomaly detection across various applications.

Cross-modal Geometric Alignment (CGA) denotes a family of methods that reduce geometric, statistical, or relational mismatch between heterogeneous modalities before or during joint inference. Across the recent literature, the aligned object may be a manifold, a token lattice, a patchwise correspondence field, a latent distribution, a category-prototype geometry, or an internal attention pattern. The common objective is not merely to place modalities in one nominally shared space, but to make cross-modal relations structurally compatible at the level required by retrieval, fusion, dense correspondence, generation, continual learning, or anomaly detection (Conjeti et al., 2016, Li et al., 2024, Shrivastava et al., 3 Jun 2025, Yu et al., 3 Jun 2026, Wang et al., 28 Jan 2026, Yu et al., 8 May 2026).

1. Scope, lineage, and problem definition

The terminology is not uniform across the literature. Closely related formulations include cross-modal manifold learning, latent geometric mapping, geometry-preserving unsupervised alignment, self-supervised spatial correspondence, structured cross-modal alignment, and cross-modal feature alignment (Conjeti et al., 2016, Zeng et al., 30 Mar 2025, Yu et al., 3 Jun 2026, Shrivastava et al., 3 Jun 2025, Wang et al., 28 Jan 2026, Bai et al., 7 May 2026). This suggests that CGA is best understood as a unifying viewpoint rather than a single algorithmic template.

An early explicit formulation appears in Cross-Modal Manifold Learning, which aligns two modality-specific manifolds into a common latent space while preserving both global manifold geometry and local neighborhood structure, using partially corresponding instances, pMST-based geometry preservation, and a joint embedding derived from a composite distance matrix (Conjeti et al., 2016). More recent work broadens the scope substantially. AlignMamba treats multimodal fusion as a problem of explicit local and global pre-alignment before Mamba-based sequence modeling (Li et al., 2024). GMapLatent replaces direct latent matching with a canonical geometric representation that preserves semantic clusters and supports constrained registration (Zeng et al., 30 Mar 2025). GPUA aligns a vision-only foundation model to a vision-language semantic space through an orthogonal map learned from unsupervised correspondence mining (Yu et al., 3 Jun 2026). Self-Supervised Spatial Correspondence Across Modalities reframes alignment as dense cross-modal space-time correspondence learning without spatially aligned multimodal pairs (Shrivastava et al., 3 Jun 2025). StructAlign introduces simplex ETF geometry as a shared prior for continual text-video retrieval (Wang et al., 28 Jan 2026). Align3D-AD maps rendering features into RGB semantic space for zero-shot 3D anomaly detection (Bai et al., 7 May 2026).

Across these formulations, the problem typically starts from a modality gap induced by heterogeneity in sampling rate, tokenization, feature statistics, semantics, or supervision. In some settings the gap is defined operationally as retrieval failure or fusion inefficiency; in others it is defined geometrically as mismatch of manifolds, distributions, or conditional neighborhoods. A recurring assumption is that semantically corresponding content exists across modalities, but is not directly recoverable by naive concatenation, standard sequential scanning, or global shared-space training alone (Li et al., 2024, Zeng et al., 30 Mar 2025, Yu et al., 8 May 2026).

2. Geometric objects and alignment regimes

CGA methods differ first by what they align. In explicit correspondence models, the central object is a transport plan or transition matrix over tokens or patches. AlignMamba formulates video-to-language alignment with a transport plan $M_{v2l} \in \mathbb{R}^{T_v \times T_l}$ under cosine cost

$$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$

and constructs aligned features by

$$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$

The same paper complements local OT with a global MMD loss

$$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$

thereby combining token-wise geometric matching and distribution-level alignment in RKHS (Li et al., 2024).

In dense visual correspondence, the aligned object is a stochastic field over spatial locations. Self-Supervised Spatial Correspondence Across Modalities defines the cross-modal transition matrix

$$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$

and trains it with cross-modal and intra-modal cycle consistency. The full objective

$$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$

treats alignment as dense soft correspondence rather than global transform estimation (Shrivastava et al., 3 Jun 2025).

Other work aligns category-level geometry rather than instancewise transport. StructAlign defines category prototypes $P=[p_1,\dots,p_C]$ arranged as a simplex ETF with

$p_c^\top p_{c'} = \begin{cases} 1, & c=c',\[4pt] -\dfrac{1}{C-1}, & c \ne c', \end{cases}$

and then aligns text and video pooled features to the same category prototype through a cosine-style loss (Wang et al., 28 Jan 2026). GPUA instead aligns whole feature spaces by solving an orthogonal Procrustes problem

$$\min_W \|ZW - PY\|_F^2,\qquad \text{s.t. } W^\top W = I,$$

with solution

$$W_0 = UV^\top,\qquad U\Sigma V^\top = \mathrm{SVD}(Z^\top P Y),$$

so that VFM geometry is preserved while mapped into a VLM semantic space (Yu et al., 3 Jun 2026).

A further regime concerns residual correction rather than explicit matching. Anisotropic Modality Align argues that image and text representations already share compatible dominant semantic geometry, while the residual modality gap is anisotropic and concentrated along a small number of dominant directions. It therefore seeks a bounded correction $$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$0 that preserves source semantics while improving target-modality compatibility, rather than a full remapping of the source manifold (Yu et al., 8 May 2026). This suggests that CGA is not always equivalent to forcing complete modal collapse; in several settings, preserving source geometry while correcting target-incompatible residual directions is the operative objective.

3. Principal methodological families

Several methodological families now recur across the literature.

Family	Representative mechanism	Representative papers
Manifold and canonical registration	joint distance embedding, OT merging, harmonic mapping, twin autoencoders	(Conjeti et al., 2016, Zeng et al., 30 Mar 2025, Rhodes et al., 26 Sep 2025)
Token/patch correspondence	OT transport, soft transition matrices, patchwise rendering-to-RGB projection	(Li et al., 2024, Shrivastava et al., 3 Jun 2025, Bai et al., 7 May 2026)
Feature-space translation	orthogonal mapping, anisotropic bounded correction	(Yu et al., 3 Jun 2026, Yu et al., 8 May 2026)
Prototype-anchored geometry	simplex ETF prototypes, dual-prompt contrastive alignment	(Wang et al., 28 Jan 2026, Bai et al., 7 May 2026)
Visualization- and interaction-driven alignment	fused projection and interactive point-set / set-set correction	(Ye et al., 2024)

The manifold-alignment family is exemplified by Cross-Modal Manifold Learning, GMapLatent, and the geometry-regularized twin-autoencoder framework. Cross-Modal Manifold Learning preserves local and global manifold structure through pMST graphs, intra- and inter-modality affinities, and a joint latent embedding (Conjeti et al., 2016). GMapLatent first transforms each latent space by barycenter translation, optimal transport merging, and graph-constrained harmonic mapping into a canonical parameter domain, then performs constrained harmonic registration to obtain a bijective cluster-level alignment under known class correspondences (Zeng et al., 30 Mar 2025). Guided Manifold Alignment with Geometry-Regularized Twin Autoencoders converts a pretrained manifold alignment into an inductive, out-of-sample-capable model through reconstruction, alignment, and anchor losses in a shared latent space (Rhodes et al., 26 Sep 2025).

Feature-space translation methods are structurally different. GPUA mines soft semantic correspondences with entropy-regularized optimal transport and then solves an orthogonal map from VFM features into a VLM semantic space, preserving neighborhood geometry and compact cluster structure without updating foundation-model weights (Yu et al., 3 Jun 2026). AnisoAlign, by contrast, explicitly rejects the view that the modality gap is a simple global shift; it decomposes the dominant subspace into blockwise radial and phase components, learns a target-modality periodic prior, and performs bounded residual correction on the source modality (Yu et al., 8 May 2026).

A separate branch uses geometry-aware visualization and interactive steering rather than fully automatic alignment. ModalChorus introduces a Modal Fusion Map that combines metric preservation and cross-modal rank-order preservation in 2D, then lets users perform point-set or set-set alignment that is mapped back to backend fine-tuning. Its contribution is therefore diagnostic and interventionist: cross-modal geometry is made visible, then semantically steered (Ye et al., 2024).

Scene-level and synthesis-oriented variants show that CGA is not limited to retrieval. CrossOver aligns RGB images, point clouds, CAD meshes, text descriptions, and floorplans into a shared scene embedding using image-centered instance contrastive learning, scene-level fusion, and unified 1D/2D/3D encoders, but does so as implicit embedding alignment rather than explicit registration (Sarkar et al., 20 Feb 2025). In aligned novel-view image and geometry synthesis, cross-modal attention instillation injects image-branch attention maps into a geometry diffusion branch so that generated novel-view RGB and pointmap outputs remain mutually aligned under a shared warped 3D scaffold (Kwak et al., 13 Jun 2025).

4. Representation learning, modality gap, and theoretical interpretation

A central theme in CGA is that shared representation spaces need not imply interchangeability. An empirical study of cross-view and cross-modal contrastive learning shows that cross-modal alignment between images and point clouds mainly captures the redundant shared geometric part of the modalities, discarding complementary visual information such as color and texture while emphasizing depth, surfaces, and spatial layout (Hehn et al., 2022). Cross-modal alignment in that setting improves downstream depth prediction, object detection, and instance segmentation, yet weakens direct recoverability of texture and can over-constrain frozen semantic representations when cross-view and cross-modal losses are forced onto the same image feature space (Hehn et al., 2022). This establishes an important trade-off: CGA can improve geometry-sensitive transfer while suppressing modality-specific information.

At the level of contrastive-learning theory, the multimodal case may be structurally different from the unimodal case. The Geometric Mechanics of Contrastive Representation Learning models learning as the evolution of representation measures on a compact embedding manifold $$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$1. In the unimodal setting, the intrinsic functional is strictly convex and admits a unique Gibbs equilibrium; in the symmetric multimodal setting, however, the intrinsic objective contains a persistent negative symmetric divergence term

$$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$2

which appears with a negative sign in the multimodal free-energy-like functional. The paper argues that this term induces barrier-driven co-adaptation and makes a population-level modality gap a structural geometric necessity rather than an initialization artifact (Cai et al., 27 Jan 2026). A direct implication is that good positive-pair alignment does not by itself guarantee full distributional alignment across modalities.

A different theoretical endpoint is the linear inverse-problem view of perfect alignment. Towards Achieving Perfect Multimodal Alignment defines perfect alignment by encoders $$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$3 and $$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$4 satisfying

$$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$5

for every paired sample, and rewrites this as

$$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$6

If the stacked multimodal matrix $$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$7 has a left null space of dimension at least $$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$8, the rows of $$C_{v2l}(i,j) = 1 - \frac{X_v^i \cdot X_l^j}{\|X_v^i\|_2 \|X_l^j\|_2},$$9 can be chosen from a basis of that left null space; otherwise, the paper uses singular vectors associated with the smallest singular values for approximate alignment (Kamboj et al., 19 Mar 2025). This is a particularly strict form of CGA: exact paired latent equivalence under a linear model.

The relation between modality gap diagnostics and operational retrieval is also nontrivial. Multimodal Representation Alignment for Cross-modal Information Retrieval reports that Wasserstein distance is informative about global modality gap, while cosine similarity consistently outperforms Euclidean, Manhattan, Chi-square, and MLP-learned scalar metrics for retrieval over CLIP and BLIP embeddings. At the same time, the paper shows that global geometry does not necessarily correlate with retrieval performance, and that shallow MLP scorers over frozen embeddings are insufficient for the complex interactions between image and text representations (Xu et al., 10 Jun 2025). This suggests that CGA is often determined less by post-hoc scalar scoring and more by the geometry already induced during representation learning.

5. Empirical evidence and application regimes

The empirical record shows that CGA is useful across markedly different tasks, but that its effects depend on the scale and object of alignment. In multimodal sequence fusion, AlignMamba reaches $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$0 Accuracy/F1 on CMU-MOSI and $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$1 on CMU-MOSEI, while ablation shows that removing local OT alignment drops MOSI performance from $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$2 to $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$3, and removing global MMD drops it to $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$4. The same paper reports reduced $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$5-distance between modality pairs and lower memory, lower inference time, and lower FLOPs than Transformer-based alternatives at long sequence lengths (Li et al., 2024). In this regime, explicit pre-alignment materially improves the downstream backbone.

In dense geometric matching, Self-Supervised Spatial Correspondence Across Modalities reports strong gains on RGB-depth and RGB-thermal correspondence. On NYU Depth V2, RGB $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$6 depth and depth $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$7 RGB reach $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$8 and $$\tilde{X}_v = M_{v2l}^{\top} X_v,\qquad \tilde{X}_a = M_{a2l}^{\top} X_a.$$9, compared with $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$0 for GMFlow and $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$1 for RAFT; on Thermal-IM, RGB $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$2 thermal and thermal $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$3 RGB reach $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$4 and $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$5, substantially above RAFT, GMFlow, DIFT, and SD-DINO. Crucially, ablations show that neither intra-modal-only nor cross-modal-only training suffices; the best results require joint intra-modal and cross-modal cycle consistency plus smoothness regularization (Shrivastava et al., 3 Jun 2025).

In latent and cross-domain generation, GMapLatent reaches FID $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$6 and accuracy $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$7 on Chinese MNIST $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$8 Arabic MNIST, and FID $$\mathcal{L}_{\text{align}} = \text{MMD}^2(\tilde{X}_v, X_l) + \text{MMD}^2(\tilde{X}_a, X_l),$$9 and accuracy $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$0 on AFHQ animal translation. Its ablations show that canonical representation helps, but full graph-constrained harmonic registration yields the largest semantic gain: on digits, accuracy rises from $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$1 to $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$2 to $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$3 as canonicalization and then graph-constrained registration are added (Zeng et al., 30 Mar 2025). Guided manifold alignment with geometry-regularized twin autoencoders reports average Mantel correlations of $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$4 with JLMA guidance and $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$5 with SPUD guidance, showing that inductive wrappers can preserve the geometry of some underlying alignment methods more faithfully than others (Rhodes et al., 26 Sep 2025).

Feature-space translation methods also show strong downstream effects. GPUA improves average zero-shot classification from $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$6 for CLIP to $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$7, and improves CLIP-based open-vocabulary segmentation frameworks such as MaskCLIP, SCLIP, and SC-CLIP on ADE20K, Pascal VOC20, and Pascal Context59 without updating either foundation model (Yu et al., 3 Jun 2026). AnisoAlign improves fully text-only MLLM training from an average score of $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$8 without alignment to $$A_{t, t+k}^{m_1, m_2} = \text{softmax}\!\left( F_t^{m_1} (F_{t+k}^{m_2})^{\top} / \tau \right),$$9, and in the mixed regime of text-only pretraining plus visual instruction tuning raises the average from $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$0 without alignment to $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$1. With $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$2M text-only samples, AnisoAlign reaches $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$3, slightly above the $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$4 reported for image-based pretraining in that setup (Yu et al., 8 May 2026).

Prototype- and prompt-based CGA is similarly effective. StructAlign improves continual text-to-video retrieval over prior methods on MSRVTT and ACTNET, with full-system ablation on MSRVTT-10 raising $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$5 from $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$6 in the baseline framework to $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$7 in the full model while reducing MeanR from $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$8 to $$\mathcal{L} = \mathcal{L}_{\mathtt{cross\text{-}crw}} + \mathcal{L}_{\mathtt{intra\text{-}crw}} + \lambda_s \mathcal{L}_{\mathtt{smooth}}$$9 (Wang et al., 28 Jan 2026). Align3D-AD reaches $P=[p_1,\dots,p_C]$0 on MVTec3D-AD and $P=[p_1,\dots,p_C]$1 on Eyecandies, outperforming prior zero-shot 3D anomaly detection baselines. Its ablations show that the fused use of RGB-aligned and raw rendering features is stronger than either alone, and that semantic consistency reweighting and dual-prompt contrastive alignment further improve localization (Bai et al., 7 May 2026). CrossOver extends the application range further to scene retrieval and localization across RGB, point clouds, CAD meshes, text referrals, and floorplans, with strong gains on scene-level recall and emergent cross-modal behaviors under missing modalities (Sarkar et al., 20 Feb 2025).

6. Limitations, misconceptions, and emerging directions

A frequent misconception is that a shared embedding space automatically yields interchangeable modalities. Several papers reject this directly. AnisoAlign argues that what prevents interchangeability is not a simple global shift but an anisotropic residual structure concentrated along a small number of dominant directions, even when modalities already share compatible dominant semantic geometry (Yu et al., 8 May 2026). The retrieval study likewise shows that low centroid gap or low Wasserstein distance does not guarantee useful cross-modal ranking, and that simple learned scorers over frozen embeddings do not recover the required structure (Xu et al., 10 Jun 2025). The theoretical contrastive analysis goes further and argues that the modality gap can be structural under symmetric multimodal objectives (Cai et al., 27 Jan 2026).

A second misconception is that more alignment is always better. The study of image–point-cloud contrastive learning shows that cross-modal alignment can discard complementary visual information such as color and texture while emphasizing redundant depth cues (Hehn et al., 2022). Align3D-AD reports that too much SCR harms performance, with the best results obtained when reweighting is limited to the final $P=[p_1,\dots,p_C]$2 epochs rather than extended through the full alignment stage (Bai et al., 7 May 2026). These results suggest that successful CGA often requires selective, task-dependent alignment rather than indiscriminate modal collapse.

Many current methods also rely on strong structural assumptions. AlignMamba is anchor-based and one-way, using language as the reference modality (Li et al., 2024). GMapLatent assumes known class-level correspondences, 2D latent visualization via t-SNE, and graph isomorphism after barycentric translation and OT merging (Zeng et al., 30 Mar 2025). Guided twin-autoencoder alignment assumes a reasonably good pretrained alignment teacher and some anchor correspondences (Rhodes et al., 26 Sep 2025). CrossOver requires a base modality during training and performs implicit scene-level alignment rather than explicit transform recovery (Sarkar et al., 20 Feb 2025). ModalChorus depends on a 2D projection and user-selected corrective actions, without guarantees that desirable projection-space movement corresponds to globally optimal high-dimensional alignment (Ye et al., 2024).

The literature also makes clear that dense correspondence, shared-space compatibility, and output-level consistency are distinct problems. Self-Supervised Spatial Correspondence Across Modalities provides dense affinity maps but not a final registration transform (Shrivastava et al., 3 Jun 2025). GPUA provides an orthogonal feature map but depends on approximate cross-space isomorphism and unlabeled fitting data (Yu et al., 3 Jun 2026). Aligned novel-view image and geometry synthesis achieves output-level alignment through cross-modal attention instillation, but only in the task-specific setting of joint image-and-pointmap diffusion (Kwak et al., 13 Jun 2025). This suggests that CGA should be treated as a layered design problem in which correspondence estimation, geometric prior selection, and downstream task structure are not interchangeable.

Several future directions are already explicit in the literature. AlignMamba identifies bidirectional or multi-anchor OT, soft or entropically regularized transport, jointly learned transport costs, and alignment under missing anchors as natural extensions (Li et al., 2024). GPUA notes class imbalance and proposes adaptive weighting or uncertainty-aware correspondence modeling as future work (Yu et al., 3 Jun 2026). GMapLatent explicitly suggests applications in large-scale domain adaptation, knowledge transfer, and multimodal information fusion, while also acknowledging that a deterministic design for guaranteed consistency in merging results remains future work (Zeng et al., 30 Mar 2025). Guided twin-autoencoder alignment states that extension to $P=[p_1,\dots,p_C]$3 domains is straightforward in principle but is not tested (Rhodes et al., 26 Sep 2025). Taken together, these works suggest that the next phase of CGA research will likely focus on relaxing anchor and topology assumptions, improving inductive generalization, and separating semantic preservation from target-distribution matching with greater precision.