Cross-Scale Visual Fusion Strategy

• Cross-scale visual fusion strategies are techniques that explicitly align, exchange, and aggregate multi-resolution features to exploit complementary fine and coarse details.
• They employ methods such as deformable alignment, dual cross-attention, and gated aggregation to mitigate spatial and semantic mismatches during fusion.
• Empirical results in MRI super-resolution, image classification, and dense mapping demonstrate improved metrics and stability by leveraging integrated multi-scale context.

Cross-scale visual fusion strategy denotes a class of methods that explicitly align, exchange, and aggregate information across different spatial resolutions, pyramid levels, or coarse-to-fine stages, rather than restricting fusion to features at the same scale. In contemporary formulations, the objective is to exploit complementary structure that becomes visible only when fine and coarse representations are jointly modeled, while suppressing the noise, semantic mismatch, and geometric inconsistency that arise when scales are fused naively. The strategy appears in multi-contrast MRI super-resolution, hierarchical Vision MLPs, multispectral–hyperspectral fusion, LiDAR–vision dense mapping, audio-visual learning, classical cross-modal image fusion, and multimodal detection, with recurring design themes that include cross-scale alignment, gated aggregation, dual-branch processing, residual refinement, and task-specific structural priors (Yang et al., 2024, Cui et al., 2023, Li, 12 Apr 2026).

1. Conceptual basis and problem formulation

At its core, cross-scale fusion assumes that informative correspondences exist not only within a single resolution level but also across scales. In multi-contrast MRI super-resolution, this is stated directly: textures exhibit self-similarity not only within the same scale but also across scales, and high-frequency structures in one scale often correspond to lower-frequency representations in another (Yang et al., 2024). In hierarchical Vision MLPs, the same idea is framed as a limitation of fixed-scale token mixing: equal-size spatial regions do not explicitly integrate features across scales, which weakens adaptability to objects of varying sizes (Cui et al., 2023). In multispectral and hyperspectral fusion, cross-scale interaction is described as the coupling of global semantics at coarse levels with local detail at fine levels through a pyramidal generator (Li, 12 Apr 2026).

The common failure mode is misalignment. In ECFNet, misalignment is both spatial and channel-semantic: receptive fields and sampling positions differ across scales, and channels at different resolutions need not carry comparable semantics (Yang et al., 2024). In LiDAR-VGGT, the analogous problem is geometric and metric rather than pixelwise: dense visual submaps are scale-ambiguous, LiDAR sessions are metric, and field-of-view inconsistency can distort similarity transforms during fine registration (Wang et al., 3 Nov 2025). In CS-Mixer, the limiting factor is computational rather than geometric: naïve full 3-axis spatial–channel mixing over $X \in \mathbb{R}^{H\times W\times C}$ requires $O((HWC)^2)$ parameters, so cross-scale interaction must be engineered through low-rank transformations and grouped aggregation (Cui et al., 2023).

A second conceptual distinction concerns the meaning of “scale.” In image restoration and recognition, scale usually refers to pyramid levels or token groupings. In LiDAR-VGGT, scale is literal metric scale recovered through coarse-to-fine Sim(3) optimization. In classical image fusion, scale can mean a low-frequency/high-frequency decomposition, as in the two-scale base/detail separation used for non-linear and selective fusion (Fang et al., 2019). This suggests that cross-scale visual fusion is better understood as a family of mechanisms for coordinating heterogeneous resolutions or abstraction levels, not as a single architectural primitive.

2. Canonical operators and recurring mechanisms

Recent work converges on a small set of reusable operators. One family addresses cross-scale alignment before aggregation. ECFNet upsamples a coarse feature $F_{k+1}$, predicts deformable offsets from $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$, and samples the upsampled feature by deformable convolution,

$$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$

thereby adapting the receptive field to the geometry of the higher-resolution feature (Yang et al., 2024). After spatial alignment, ECFNet applies a channel-alignment gate,

$$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$

to suppress redundant channels before attention (Yang et al., 2024).

A second family performs cross-scale mixing by attention or grouped token interaction. ECFNet uses dual cross-attention in spatial and channel dimensions, with the standard form

$$\mathrm{Attn}(Q,K,V)=\mathrm{softmax}(QK^\top/\sqrt{d})V,$$

where queries come from aligned target features and keys/values from reference features (Yang et al., 2024). CS-Mixer replaces full attention with local aggregation (LA) and global aggregation (GA), then performs spatial–channel mixing in a low-dimensional subspace. Its effective operator per head is described as

$$(I_L \otimes \tilde V_v^n)\cdot w^n \cdot (I_L \otimes W_v^n),$$

which approximates full 3-axis mixing while keeping $d \ll c$ (Cui et al., 2023). In CMQKA, the design objective shifts to linear complexity: a binary per-token mask derived from a Query stream gates the Key stream, avoiding an $N\times N$ attention matrix and enabling hierarchical fusion at high-resolution stages (Saleh et al., 31 Jan 2026).

A third family uses explicit decomposition or branch specialization. The 2019 non-linear image fusion framework decomposes each modality into base and detail layers, computes saliency and illuminance-based sigmoid fusion factors, and reconstructs the final image as $O((HWC)^2)$0 (Fang et al., 2019). CoFusion applies an analogous specialization at each pyramid level, but with spatial and spectral branches: SpaCAM captures multi-receptive-field spatial context with gated dilated convolutions, whereas SpeCAM decomposes spectral features by DWT and applies spectral coordinate mixing (Li, 12 Apr 2026).

A fourth family regularizes fine-scale fusion by anchoring it to a coarse-scale estimate. LiDAR-VGGT first estimates a coarse session-level Sim(3) alignment, then refines it with an ICP-like objective regularized toward the coarse scale,

$O((HWC)^2)$1

so that fine correspondence fitting does not destroy the coarse metric prior under weak overlap or field-of-view mismatch (Wang et al., 3 Nov 2025).

Paper	Domain	Cross-scale mechanism
ECFNet (Yang et al., 2024)	Multi-contrast MRI SR	Deformable alignment, channel gating, dual cross-attention
CS-Mixer (Cui et al., 2023)	Image classification	LA/GA grouping and low-rank 3-axis mixing
CoFusion (Li, 12 Apr 2026)	MHIF	Three-level pyramid with SpaCAM, SpeCAM, SSCFM
LiDAR-VGGT (Wang et al., 3 Nov 2025)	Dense mapping	Session-level coarse-to-fine Sim(3) fusion
CMQKA/SNNergy (Saleh et al., 31 Jan 2026)	Audio-visual learning	Linear-complexity binary masking in a hierarchy
Non-linear selective fusion (Fang et al., 2019)	Cross-modal image fusion	Two-scale LF/HF decomposition with illuminance gating

3. ECFNet as a detailed exemplar

The most explicit formulation of cross-scale visual fusion in the provided literature is ECFNet, an edge-guided and cross-scale feature fusion network for efficient multi-contrast MRI super-resolution (Yang et al., 2024). The input is a low-resolution target contrast $O((HWC)^2)$2 and a high-resolution reference contrast $O((HWC)^2)$3. Preprocessing interpolates $O((HWC)^2)$4 to the spatial size of $O((HWC)^2)$5 and computes an edge map with Sobel. A four-scale encoder produces target features $O((HWC)^2)$6 and reference features $O((HWC)^2)$7.

At each stage, the Cross-scale Feature Fusion Module first upsamples the next coarser target feature, aligns it to the current target scale by deformable convolution, and corrects channel mismatch by the CA gate. Only then does it perform dual cross-attention with the reference feature at the current scale, yielding a texture representation $O((HWC)^2)$8. This ordering is important: the paper attributes degraded same-scale or unaligned fusion to noisy aggregation caused by unresolved spatial and semantic mismatch. The resulting cross-scale fusion operator combines weighted channel gating, attention-based aggregation, depth-wise projection, and residual refinement (Yang et al., 2024).

The decoder introduces two further task-specific modules. The Texture Transfer Module remaps reference-derived texture into the target feature space by an AdaIN-like transformation:

$O((HWC)^2)$9

This is intended to preserve structural content while adapting the style distribution of the transferred texture. The Structure Information Collaboration Module uses Sobel-derived structure priors and asymmetric $F_{k+1}$0 and $F_{k+1}$1 convolutions to emphasize high-frequency detail and refine anatomically relevant boundaries. Its residual path is given as

$F_{k+1}$2

Training uses only reconstruction and edge losses,

$F_{k+1}$3

with total objective

$F_{k+1}$4

No explicit alignment loss or attention-consistency loss is introduced; alignment is learned end to end through reconstruction and edge supervision. The implementation details reported are four encoder/decoder scales, a CA gate with reduction ratio $F_{k+1}$5, Sobel-based structure extraction, and evaluation at $F_{k+1}$6 and $F_{k+1}$7 super-resolution on IXI and BraTS2020 (Yang et al., 2024).

The broader significance of ECFNet is that it makes the fusion strategy explicit at three coupled levels: cross-scale target alignment, cross-contrast texture transfer, and edge-guided structural refinement. The paper also states that this combination is distinctive within multi-contrast MRI super-resolution because it explicitly couples learnable geometric alignment across scales, attention-based cross-contrast texture transfer, and edge-guided structural refinement (Yang et al., 2024).

4. Realizations beyond MRI

Cross-scale visual fusion appears in substantially different architectural idioms across domains. CS-Mixer formulates the problem inside a hierarchical Vision MLP. Its initial cross-scale patch embedding uses four stride-4 convolutions with kernel sizes $F_{k+1}$8 and channel allocations $F_{k+1}$9, $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$0, $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$1, and $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$2, concatenated to produce $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$3 embeddings. Cross-scale patch merging between stages uses patch sizes $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$4 and stride 2. Inside each block, LA forms contiguous $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$5 windows, whereas GA interleaves tokens across the full image, so local detail and long-range context are mixed jointly through low-rank spatial–channel operators (Cui et al., 2023).

CoFusion applies the strategy to multispectral and hyperspectral image fusion. Its Multi-Scale Generator is a three-level, HRNet-inspired pyramid with parallel top-down and bottom-up flows. At each scale, SpaCAM uses gated dilated depth-wise convolutions with dilation rates $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$6 to recover spatial detail, while SpeCAM decomposes spectral features into low- and high-frequency components by DWT, pools saliency, performs Spectral Coordinate Mixing through Top-K traversal, and reconstructs a residual spectral representation. The Spatial-Spectral Cross-Fusion Module then fuses the branches with a spatial gate from large-kernel depth-wise convolutions and a spectral gate from squeeze-and-excitation, rather than explicit Q–K–V attention (Li, 12 Apr 2026).

LiDAR-VGGT transports the same logic to dense mapping. The “coarse” scale is session-level pose and scale estimation through Umeyama-based Sim(3) registration, linearity validation by PCA, and scale RANSAC across sessions. The “fine” scale is point-cloud alignment with alternating nearest-neighbor correspondence updates and a closed-form scale update regularized by source bounding-box size. Global pose graph optimization then enforces map-level consistency. This use of cross-scale fusion is notable because the fused entities are not feature tensors but session trajectories, local submaps, and global graph constraints (Wang et al., 3 Nov 2025).

CMQKA and SNNergy show that cross-scale fusion can be driven primarily by complexity constraints. SNNergy uses a three-stage hierarchy at $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$7, $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$8, and $\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})$9, with CMQKA in the first two stages and quadratic spiking self-attention only in the coarsest stage. The key idea is that high-resolution fusion should use linear-complexity binary masking, whereas global context can be deferred to low-resolution maps where $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$0 is small (Saleh et al., 31 Jan 2026).

Two further examples emphasize that cross-scale fusion is not restricted to attention-centric systems. The 2019 non-linear and selective fusion framework uses a two-scale decomposition, illuminance modeling in YCrCb, saliency normalization, and channel attention to fuse infrared–visible, medical, and multi-focus images (Fang et al., 2019). MSFNet-CPD constructs multi-scale token grids from both original and super-resolved pest images, concatenates them with BERT text tokens in a Transformer-based Image-Text Fusion module, and then reconstructs scale-specific spatial maps for a YOLOv4 neck. Here, cross-scale visual fusion is coupled to cross-modal semantic grounding rather than solely to spatial reconstruction (Zhang et al., 5 May 2025).

5. Empirical behavior, efficiency, and ablation evidence

The empirical literature supports two general claims: cross-scale fusion helps most when single-scale features are impoverished, and the benefit depends strongly on alignment or gating quality. ECFNet reports state-of-the-art PSNR/SSIM on IXI and BraTS2020 at both $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$1 and $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$2. On IXI it reaches 41.823 dB / 0.987 SSIM at $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$3 and 37.213 dB / 0.970 SSIM at $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$4; on BraTS2020 it reaches 41.218 dB / 0.995 SSIM at $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$5 and 34.985 dB / 0.972 SSIM at $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$6. Its ablation on IXI $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$7 is especially diagnostic: removing multi-scale feature alignment drops performance to 33.478 dB / 0.941 SSIM, removing TTM yields 35.437 dB / 0.968 SSIM, and removing the structure branch yields 35.312 dB / 0.965 SSIM. The authors explicitly attribute resilience at $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$8 to cross-scale fusion with alignment that borrows texture from other scales and the reference image (Yang et al., 2024).

In CS-Mixer, the evidence is primarily recognition accuracy under controlled compute. CS-Mixer-L reaches 83.2% top-1 on ImageNet-1k with 13.7 GFLOPs and 94.2M parameters, while the paper argues that explicit local and global cross-scale aggregation is central to its competitiveness relative to prior Vision MLPs that mix only one or two axes (Cui et al., 2023). CoFusion reports 38.3195 PSNR and 0.9824 SSIM on $$\Delta p = \mathrm{Conv}_{3\times3}(\mathrm{Concat}(F_k, F_{\mathrm{up},k+1})), \qquad y(p_0)=\sum_k w_k \cdot x(p_0+p_k+\Delta p_k),$$9 PaviaU, 53.0875 PSNR and 0.9981 SSIM on $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$0 Chikusei, and 48.9371 PSNR and 0.9963 SSIM on $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$1 Chikusei, alongside the lowest $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$2, $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$3, and highest QNR in unsupervised evaluation at $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$4. Its ablations show the largest drop when SpaCAM is removed, followed by SpeCAM and SSCFM, indicating that spatial detail recovery, spectral continuity, and cross-modal collaboration all contribute materially (Li, 12 Apr 2026).

Efficiency-oriented designs show a different trade-off. CMQKA is stated to have $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$5 complexity versus $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$6 for standard spiking self-attention. The stage-wise operation counts at $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$7 inputs show that the hybrid design—CMQKA in high- and mid-resolution stages, SSA in the coarsest stage—reduces total operations relative to all-SSA. SNNergy then reports 78.38 accuracy on CREMA-D, 72.14 on AVE, and 99.66 on UrbanSound8K-AV, with ablations showing that joint spatiotemporal fusion outperforms spatial-only or temporal-only variants (Saleh et al., 31 Jan 2026).

In LiDAR-VGGT, the benefits are geometric and photometric rather than recognition-based. Across major sequences it reports substantially lower Chamfer Distance, often higher than 50% ICP overlap, and best AWD/Fitness relative to VGGT-based alternatives and LIVO baselines; the registration ablation directly states that normal Sim(3) suffers “excessive scale distortion,” whereas the regularized variant reduces RMSE/AWD/CD and improves spatial consistency (Wang et al., 3 Nov 2025). In MSFNet-CPD, removing text reduces mAP from 46.06% to 35.64%, and removing the SR stream reduces it further to 34.82%, indicating that multiscale visual enhancement and cross-modal fusion are both integral to the final detector (Zhang et al., 5 May 2025).

A common pattern across these results is that coarse information is not merely contextual decoration. It often stabilizes fine-detail recovery, disambiguates local evidence, or prevents degenerate optimization. This suggests that the main empirical value of cross-scale fusion lies less in increasing representational capacity in the abstract than in constraining information flow across heterogeneous resolutions in a task-aligned manner.

6. Limitations, misconceptions, and open directions

A frequent misconception is that cross-scale fusion is equivalent to simple pyramid summation. The literature repeatedly rejects that interpretation. ECFNet explicitly contrasts its design with same-scale fusion and with purely hierarchical feature summation, arguing that without deformable alignment and channel gating, cross-scale aggregation becomes noisy (Yang et al., 2024). CS-Mixer likewise distinguishes explicit cross-scale aggregation from merely stacking fixed-window mixers (Cui et al., 2023). CoFusion shows that multiscale processing alone is insufficient unless spatial and spectral branches are coupled dynamically through SSCFM (Li, 12 Apr 2026).

A second misconception is that attention is always the correct or necessary fusion mechanism. Several counterexamples are explicit. CoFusion’s SSCFM is convolutional with spatial and spectral gates rather than Q–K–V attention (Li, 12 Apr 2026). The 2019 fusion framework is largely rule-based and uses attention only for channel selection after non-linear LF/HF fusion (Fang et al., 2019). LiDAR-VGGT relies on Sim(3), ICP-like alternation, and pose graph optimization rather than neural attention (Wang et al., 3 Nov 2025). The broader lesson is that cross-scale fusion is a systems-level design choice, not a synonym for transformer blocks.

The major practical limitations are also consistent across domains. Misregistration remains difficult: ECFNet notes that inter-contrast misalignment may not be fully compensated by deformable alignment, and LiDAR-VGGT notes that severe field-of-view mismatch or very small overlaps can still challenge correspondences (Yang et al., 2024, Wang et al., 3 Nov 2025). Generalization across unseen scales or modalities is uncertain: ECFNet is trained for specific contrasts and $$\phi = \mathrm{MLP}(\mathrm{GAP}(F_{\mathrm{concat},k}), \mathrm{GMP}(F_{\mathrm{concat},k})), \qquad F_{\mathrm{fused},k}=\phi \odot F_{\mathrm{concat},k}+F_{\mathrm{concat},k},$$8 settings, and CS-Mixer identifies missing ablations on group size and low-rank dimension as open issues (Yang et al., 2024, Cui et al., 2023). CoFusion explicitly notes reliance on simulated ground truth and the absence of measurement-consistency losses in the reported training (Li, 12 Apr 2026).

The proposed future directions are similarly convergent. ECFNet suggests multi-scale deformable attention, stronger positional or scale encoding, uncertainty-aware fusion, and iterative registration (Yang et al., 2024). CoFusion proposes transformer-based cross-scale fusion, adaptive scale selection, multi-resolution training with measurement consistency, and heteroscedastic uncertainty modeling (Li, 12 Apr 2026). LiDAR-VGGT points toward tighter end-to-end coupling between LiDAR cues and VGGT’s transformer rather than post hoc cross-modal registration (Wang et al., 3 Nov 2025). A plausible implication is that the next phase of cross-scale visual fusion research will be less about adding more scales and more about learning when, where, and how strongly scales should interact under uncertainty.

In that sense, cross-scale visual fusion strategy is best understood as a response to a specific representational asymmetry: fine scales contain sharp but incomplete evidence, coarse scales contain stable but underspecified structure, and effective systems must align the two without collapsing either one into the other. Across the surveyed work, successful designs do this by making scale interaction explicit, constrained, and task-dependent rather than incidental.