Multi-Scale Distribution Alignment

Updated 9 July 2026

Multi-scale distribution alignment is a framework that exploits hierarchical structures to align distributions at various granularities such as spatial, semantic, and temporal scales.
It integrates explicit methods (e.g., Wasserstein, KL divergence, MMD, adversarial losses) with implicit strategies like contrastive and consistency losses to preserve both local and global features.
Empirical studies demonstrate that coupling multi-scale feature extraction with tailored alignment objectives improves downstream performance in multimodal, domain adaptation, and retrieval tasks.

Multi-Scale Distribution Alignment denotes a family of alignment strategies that preserve or exploit structure at more than one granularity while bringing distributions, representations, or conditional prediction relations into correspondence. In current work, “scale” is not a single fixed concept: it may refer to cluster-versus-sample hierarchy in optimal transport, sparse-support coarse-to-fine approximations, micro/meso/macro spatial neighborhoods, pixel/instance/category granularity, sentence-level versus aspect-level linguistic structure, local-versus-global multimodal semantics, or Hessian eigendirections in parameter space. Likewise, “alignment” ranges from explicit Wasserstein, KL, MMD, and adversarial objectives to scale-wise contrastive losses, consistency constraints on matching matrices, and inference-time reshaping of next-token distributions (Lee et al., 2019, Jeong et al., 19 Nov 2025, Zhou et al., 2022, Wu et al., 19 Feb 2025, Ballas et al., 8 May 2026).

1. Conceptual scope and taxonomic variants

Across the literature, multi-scale distribution alignment is best understood as an umbrella over methods that do not treat the data as a single flat distribution. Some methods introduce an explicit hierarchy, as in cluster-level and sample-level optimal transport; some build synchronized representations at several spatial or semantic scales; and some decompose the alignment problem into local and global components without a formal hierarchy. A common motivation is that flat alignment can collapse semantically distinct modes, overemphasize coarse structure, or ignore the fact that correspondences are easier to establish at one granularity than at another (Lee et al., 2019, Jeong et al., 19 Nov 2025).

Scale notion	Aligned object	Representative papers
Cluster/sample hierarchy	Cluster correspondences and within-cluster OT	(Lee et al., 2019, Mi et al., 2018, Zhou et al., 2021)
Spatial micro/meso/macro	Image, graph, or cross-modal embeddings at several spatial extents	(Jeong et al., 19 Nov 2025, Yang et al., 2024, Fan et al., 31 May 2025)
Semantic/structural granularity	Pixel/instance/category, sentence/aspect, within/between speaker	(Zhou et al., 2022, Wu et al., 19 Feb 2025, Zhou et al., 2023)
Parameter/output-space scale	Gradient geometry across eigendirections, next-token distributions	(Ballas et al., 8 May 2026, Xia et al., 20 Nov 2025)

A recurring distinction is between explicit and implicit distribution alignment. Explicit methods directly optimize divergences, transport costs, or adversarial domain losses. Implicit methods instead align via scale-wise contrastive objectives, instruction tuning, or consistency regularization, so that matched objects become close in the learned space without an explicit density-matching term. This distinction is especially important in multimodal work, where several papers are about “multi-scale alignment” in a representational sense, but not about direct marginal distribution matching (Wang et al., 2024, Fan et al., 31 May 2025).

Another recurring distinction is between scale-specific alignment and cross-scale coherence. Some systems independently align each scale and then add a second mechanism that forces relations learned at one scale to supervise or regularize other scales. Others rely on shared sample identity or a common optimizer to make the scales cohere implicitly. The latter is common in graph-image and image-text alignment systems, where the scale hierarchy is synchronized architecturally but not by an explicit cross-scale penalty (Jeong et al., 19 Nov 2025, Yang et al., 2024).

2. Mathematical objects and alignment objectives

The aligned object varies substantially across formulations. In regularized Wasserstein means, the object is a sparse discrete measure supported on centroids with weights, and the central regularized objective is

$\mathcal{L}(\pi,y)=\sum_j\|y_j-\tilde y_j\|_2^2+\lambda \mathcal L_{\text{reg}(y)},$

where $\tilde y_j$ is the transport-induced barycentric update. The important design choice is that regularization acts on the support update rather than on the transport coupling itself, which allows class, geometric, or topological structure to be imposed on the aligned support set (Mi et al., 2018).

In remote-sensing image-text retrieval, the aligned object is not a feature marginal but a batch-level matching matrix at each image scale. The method computes $\mathbf m^i\in\mathbb R^{b\times b}$ for each scale, applies a bidirectional contrastive loss on each matrix, and then uses KL divergence between row-wise softmax distributions so that the largest-scale matching matrix guides smaller scales. Here, “distribution alignment” is most accurately the alignment of similarity distributions or relational distributions rather than direct feature-distribution matching (Yang et al., 2024).

In multimodal recommendation, local alignment and global distribution alignment are separated. The DREAM module performs multi-scale local feature refinement through dilated convolutions plus channel and spatial attention, while the global alignment module applies MMD and InfoNCE to the latent visual and textual representations. The final objective combines BPR ranking loss with contrastive and MMD regularization, so alignment is jointly local-semantic and global-distributional (Ren et al., 11 Sep 2025).

In LLM inference-time alignment, the aligned object is the next-token probability distribution itself. SDA constructs a base distribution $\boldsymbol P_1(x_t\mid\mathcal Q)$ and an instruction-conditioned distribution $\boldsymbol P_2(x_t\mid\mathcal Q,\mathcal I)$ , forms a steering vector $\log\boldsymbol P_2-\log\boldsymbol P_1$ , and uses a JS-divergence-based dynamic temperature before sampling from the adjusted distribution. In this formulation, distribution alignment means instruction-conditioned redistribution of token probabilities rather than feature-space adaptation (Xia et al., 20 Nov 2025).

A more explicitly information-geometric formulation appears in multi-scale manifold alignment for LLM representations. There, the mapping error between adjacent scales is decomposed into a geometric term and a conditional KL term, e.g.

$\mathcal{E}^{\mathrm{info}}_{G\rightarrow I} = D_{\mathrm{KL}}\!\left( p(h_I\mid h_G) \;\middle\|\; p(f_{GI}(h_G)\mid h_G) \right),$

while a mutual-information objective preserves semantic content across scales. This formulation makes “distribution alignment” mean alignment of conditional latent distributions between semantic manifolds rather than only Euclidean proximity of embeddings (Zhang et al., 24 May 2025).

Taken together, these formulations show that the “distribution” in multi-scale distribution alignment may denote empirical measures over samples, sparse transport supports, batchwise retrieval relations, latent conditional distributions, or autoregressive token probabilities. The unifying theme is not a single divergence, but the insistence that alignment must respect structure that exists at more than one granularity.

3. Explicit hierarchical and transport-based formulations

The clearest explicit hierarchical formulation is Hierarchical Wasserstein Alignment. It introduces a two-level OT problem: a coarse-scale transport matrix $\mathbf P\in B_S$ over clusters, and fine-scale transport plans $\mathbf Q_{ij}\in U(n_{x,i},n_{y,j})$ within candidate cluster pairs. With a shared orthogonal transform $\mathbf R$ , the core objective is

$\tilde y_j$ 0

or, in the entropically regularized version,

$\tilde y_j$ 1

This formulation is explicitly multi-scale because cluster-level OT resolves mode correspondences and sample-level OT refines alignment inside matched modes. The hierarchy is fixed by a supplied or externally estimated clustering rather than learned end-to-end (Lee et al., 2019).

Regularized Wasserstein means is less explicit but still strongly related to coarse-to-fine alignment. It replaces a dense point-to-point mapping with a sparse discrete support transported into the target domain by a Monge-style map. A small number of support points gives a coarse approximation, and increasing the number of supports yields finer geometric fidelity. The method itself does not define a formal hierarchy, but the sparse-support viewpoint, barycentric centroid updates, and support-level regularizers make it naturally compatible with progressive refinement (Mi et al., 2018).

Iterative Alignment Flows also fit the topic through progressive residual correction rather than explicit scale decomposition. At each layer, the method finds projection directions that maximize the current multi-distribution discrepancy and then exactly aligns the projected one-dimensional marginals to a shared barycenter by closed-form OT maps. The full map is a composition of invertible updates,

$\tilde y_j$ 2

so earlier layers remove large discrepancy components and later layers refine residual mismatch. The paper is explicit that this is not a formal multi-scale architecture; the progressive character is implicit and greedy (Zhou et al., 2021).

A non-adversarial flow-based perspective is provided by cooperative alignment via a variational upper bound on generalized JSD. The central loss,

$\tilde y_j$ 3

aligns multiple distributions through domain-specific invertible maps and a shared latent density model $\tilde y_j$ 4. This framework is not explicitly multi-scale, and the paper states that it does not provide a hierarchical or multi-resolution objective. A plausible implication is that its shared latent-density view can serve as a foundation for future per-scale extensions, but that extension is not part of the paper itself (Cho et al., 2022).

Within transport-based work, then, multi-scale distribution alignment ranges from fully explicit nested OT to sparse-support approximations and iterative residual flows. The main design choice is whether the hierarchy is built into the objective, induced by the support representation, or realized only through repeated residual correction.

In histopathology–spatial-transcriptomics alignment, SIGMMA is a canonical micro/meso/macro design. On the image side, each H&E tile is partitioned into a $\tilde y_j$ 5 grid at micro scale, a $\tilde y_j$ 6 grid at meso scale, and a whole-tile macro view. On the transcriptomics side, each tile is represented as a cell graph whose neighborhoods are expanded hierarchically by stochastic edge addition under a neighbor-patch constraint. The model then applies a bidirectional InfoNCE-style alignment loss independently at the micro, meso, and macro scales, with the total objective

$\tilde y_j$ 7

The paper explicitly states that there is no extra cross-scale consistency term; cross-scale coherence is implicit, coming from shared sample identity and synchronized receptive fields across modalities (Jeong et al., 19 Nov 2025).

For remote-sensing image-text retrieval, MSA argues that multi-scale fused image features should not be aligned to text only after fusion. Instead, each image scale is aligned separately with localized text features and global textual context through the Multi-Scale Cross-Modal Alignment Transformer. The method then adds a cross-scale KL consistency loss so that the matching matrix from the largest image scale guides the smaller scales. This is one of the most direct examples of aligning both within-scale cross-modal relations and across-scale relational distributions in a single training objective (Yang et al., 2024).

In fine-grained multimodal grounding, TinyGroundingGPT uses “multi-scale alignment” in a different sense: object text, bounding-box coordinates, cropped object images, and whole-image context are aligned by staged supervised instruction tuning. The local/object level is aligned before detailed global knowledge alignment. The paper is explicit that it does not define contrastive, OT, MMD, or adversarial distribution-matching losses; the alignment is implemented through multimodal QA and conversation generation tasks. For the broader topic, it is therefore an example of multi-granularity multimodal alignment adjacent to, but not identical with, explicit distribution alignment (Wang et al., 2024).

In brain-assisted target-speaker extraction, M3ANet combines four temporal speech scales—implemented by Conv1D branches with filter lengths of 2.5 ms, 5 ms, 10 ms, and 20 ms—with an explicit contrastive alignment module between EEG and speech embeddings. The method addresses temporal inconsistency between modalities by aligning temporally structured latent features before fusion. The alignment loss is InfoNCE-based, but it operates on flattened segment embeddings rather than an explicit token-to-token alignment matrix, so the method is best viewed as segment-level contrastive alignment over temporally structured latent sequences (Fan et al., 31 May 2025).

Across these multimodal systems, a common pattern emerges. Alignment improves when the scales of one modality are made structurally compatible with those of the other, and when scale-specific relations are supervised directly rather than being expected to emerge from a single fused representation. Some methods make cross-scale coherence explicit through KL or teacher-style regularization; others leave coherence implicit in the architecture and paired-sample identity.

5. Structured local–global alignment beyond explicit hierarchies

A separate line of work replaces formal hierarchies with structured decompositions over semantic granularity. In unsupervised domain-adaptive object detection, MGADA jointly aligns source and target domains at the pixel, instance, and category levels, while an omni-scale gated fusion module improves the scale sensitivity of detection features before alignment. The total loss augments detector supervision with pixel-level, instance-level, and category-level adversarial losses, and the category-level discriminator explicitly separates class discrimination from source–target confusion within each class. This makes the method simultaneously scale-aware and semantically multi-granular (Zhou et al., 2022).

For cross-lingual aspect-based sentiment analysis, MSMO defines “multi-scale” as sentence-level and aspect-level alignment. Sentence-level alignment is adversarial and Wasserstein-style through a language discriminator trained on source, translated target, and code-switched variants. Aspect-level alignment is a symmetric KL consistency penalty on span-level predictive distributions of corresponding aspect terms across original, translated, and code-switched sequences. The framework is explicitly two-stage: coarse sentence-level alignment first, then fine aspect-level alignment (Wu et al., 19 Feb 2025).

In multimodal recommendation, MambaRec separates local semantic alignment from global distribution consistency. DREAM refines modality features with $\tilde y_j$ 8 and dilated $\tilde y_j$ 9 convolutions at dilation rates 6, 12, and 18, plus a global pooling branch, channel attention, and spatial attention. Global modality alignment is then imposed by MMD and InfoNCE. The method therefore exemplifies a local-global split in which multi-scale feature refinement and distribution-level regularization are complementary rather than interchangeable (Ren et al., 11 Sep 2025).

A cautionary result appears in multi-genre speaker recognition. There, several alignment strategies are compared across multiple genre-conditioned embedding distributions: DeepCORAL, MMD, CenterLoss, WDA, BDA, and WBDA. WBDA, which decomposes alignment into within-speaker and between-speaker covariance structure, performs relatively better, and WDA is stronger than BDA. However, no method consistently improves every cross-genre condition, and the paper concludes that aligning speaker-vector distributions alone does not fully solve multi-genre speaker recognition (Zhou et al., 2023).

A related but more interpretability-oriented decomposition is Multi-Scale Manifold Alignment for LLMs, which partitions latent space into global, intermediate, and local semantic manifolds and aligns adjacent levels through geometric losses, mutual-information preservation, and curvature regularization. Although this is not classical domain adaptation, it offers a formal cross-scale decomposition in which the alignment error is bounded by geometric and information terms measured partly through KL divergence (Zhang et al., 24 May 2025).

Finally, spectral-aware gradient alignment extends the topic into parameter space. The key excess-risk decomposition contains an alignment term controlled by

$\mathbf m^i\in\mathbb R^{b\times b}$ 0

and a curvature term controlled by

$\mathbf m^i\in\mathbb R^{b\times b}$ 1

The paper is explicit that it is not about multi-scale feature alignment; its relevance is that Hessian eigendirections define curvature scales, so disagreement across training distributions is weighted differently across spectral scales. This broadens the notion of multi-scale distribution alignment beyond spatial or representational scale (Ballas et al., 8 May 2026).

6. Empirical patterns, infrastructure, and recurring limitations

Empirically, the most consistent pattern is that multi-scale structure must usually be engineered on both the representation side and the alignment side. In SIGMMA, simply using a cell graph is not enough: the ablation on HEST1k-LUAD shows that adding the multi-scale loss and graph sparsification drives the gain, and the abstract reports average improvements of 9.78\% in gene-expression prediction and 26.93\% in cross-modal retrieval (Jeong et al., 19 Nov 2025). In remote-sensing retrieval, multi-scale fusion alone provides limited benefit, whereas separate scale-wise alignment plus cross-scale consistency produces the strongest results, and aligning all four image scales is best (Yang et al., 2024).

The same pattern appears in domain-adaptive detection. MGADA’s ablations on Cityscapes $\mathbf m^i\in\mathbb R^{b\times b}$ 2 FoggyCityscapes show that removing gated fusion or category-level alignment substantially reduces performance; with all components included, the FCOS VGG-16 model reaches 43.6 mAP, versus 39.3 without category-level discrimination and 41.3 without gated fusion (Zhou et al., 2022). This indicates that multi-scale feature extraction and multi-granularity alignment are not independent design choices; the strongest results come from coupling them.

Data infrastructure has also become part of the alignment story. SOMA-1M contributes a large-scale SAR–optical multi-resolution benchmark with 1,300,954 pixel-level aligned pairs of $\mathbf m^i\in\mathbb R^{b\times b}$ 3 images, spanning 0.5 m to 10 m and 12 land-cover categories. Its coarse-to-fine registration pipeline uses global matching on downsampled scenes, local refinement on $\mathbf m^i\in\mathbb R^{b\times b}$ 4 and $\mathbf m^i\in\mathbb R^{b\times b}$ 5 crops, and manual inspection of 100,000 randomly selected pairs, with a reported qualification rate exceeding 99.8\%. This does not itself solve statistical distribution alignment, but it provides precisely aligned multimodal data on which multi-scale alignment methods can be trained and evaluated (Wu et al., 5 Feb 2026).

A persistent limitation is that cross-scale coherence is often only partially formalized. SIGMMA explicitly states that it has no additional cross-scale consistency term beyond scale-wise InfoNCE and synchronized receptive fields (Jeong et al., 19 Nov 2025). M3ANet motivates temporal alignment at the level of time steps, but its actual InfoNCE is applied to flattened segment embeddings rather than a differentiable lag model (Fan et al., 31 May 2025). TinyGroundingGPT performs local and global multimodal alignment but without an explicit distribution-matching objective (Wang et al., 2024). These cases show that “multi-scale alignment” often refers to the construction of matched hierarchies more than to a single unified probabilistic objective.

Another limitation is dependence on structural priors or heuristics. Hierarchical OT requires known or externally estimated clusters and can fail under uninformative geometry such as equally spaced subspaces (Lee et al., 2019). MGADA depends on pseudo labels and hand-designed kernel choices (Zhou et al., 2022). MSMO depends on translation quality and span correspondence in code-switched data (Wu et al., 19 Feb 2025). MambaRec leaves several implementation details underspecified, including the exact point where MMD is applied and whether DREAM is applied separately to each modality (Ren et al., 11 Sep 2025). The speaker-recognition study makes the broader point that better alignment geometry does not guarantee universal downstream gains (Zhou et al., 2023).

The field therefore supports a broad but precise conclusion. Multi-Scale Distribution Alignment is not a single method class but a design principle: alignment should respect coarse and fine structure simultaneously, and the relevant notion of scale may be spatial, semantic, topological, temporal, or spectral. Current realizations include explicit hierarchical OT, sparse-support transport, scale-synchronous contrastive learning, local-global regularization, structured covariance matching, semantic manifold alignment, and output-distribution steering. What remains unresolved is how to combine explicit cross-scale coherence, strong geometric fidelity, and robust downstream generalization without relying excessively on handcrafted structure or task-specific heuristics.