MSMM: Multi-Scale Mamba Module

Updated 15 January 2026

Multi-scale Mamba Module (MSMM) is a deep learning component that hierarchically aggregates features using selective state-space models for robust context modeling.
It employs parallel and cascaded fusion strategies to integrate multi-resolution representations, enhancing alignment and long-range dependency capture.
MSMMs improve performance across applications—vision, language, time-series, and spectral analysis—while maintaining linear computational complexity.

A Multi-scale Mamba Module (MSMM) is a structured deep learning component that enables hierarchical, multi-resolution aggregation of features using selective state-space models (SSMs) of the Mamba architecture. MSMMs have emerged as key primitives across diverse modalities—vision, language, multimodal learning, time-series, segmentation, recommendation, and spectral analysis—capitalizing on Mamba’s linear complexity and robust long-range dependency modeling. By fusing intermediate representations at multiple spatial, temporal, or perceptual scales, MSMMs consistently improve alignment, context modeling, and downstream accuracy, while remaining computationally tractable.

1. Core Architectural Principles

MSMMs universally instantiate multi-scale feature fusion by collecting latent representations from distinct points along a backbone (spatial depths for vision, time scales for sequence data, or spectral bands for remote sensing). Fusion typically occurs via either parallel or cascaded submodules:

Parallel multi-scale processing: Separate branches extract features at differing receptive fields (e.g., 3×3, 5×5, 7×7 in 2D/3D convolutions (Wang et al., 25 Mar 2025, Guan et al., 8 Jan 2026)), distinct temporal resolutions via sampling rates (Karadag et al., 10 Apr 2025), or decomposed frequency bands (Jeon, 7 Dec 2025).
Cascaded fusion blocks: Fused sequentially with cross-attention and residual SSM/Mamba layers (e.g., as in EMMA (Xing et al., 2024)).
Hierarchical alignment: Integration of coarse, intermediate, and fine-scale cues via joint fusion, skip connections, or adaptive windowed processing (Yang et al., 13 Jan 2025, Zheng et al., 17 Nov 2025, Zhang et al., 13 Jan 2026).

Fundamental to MSMMs is the embedding of Mamba SSMs—either as pure Mamba blocks or as hybrid submodules (e.g., interleaved with depthwise convolutions, deformable convolutions, cross-modal gates). These enforce long-range dependencies, global context propagation, and precise feature alignment across scales, all at linear computational cost.

2. Mathematical Formulation and Fusion Mechanics

The fusion process in MSMMs is formalized through blockwise compositions and aggregation operators. Let $\{\overline{X}_i\}$ denote feature sets from $K$ distinct scales:

Cascaded fusion (EMMA):

$\overline{X}_v = \mathcal{B}_2( \mathcal{B}_1(\overline{X}_i, \overline{X}_j),\; \overline{X}_k )$

Where each fusion block $\mathcal{B}(X,Y)$ combines cross-attention with a Mamba SSM and double residuals:

$\mathcal{B}(X,Y) = [X + \text{cross\_attn}(X, Y)] + \text{Mamba}(X + \text{cross\_attn}(X, Y))$

(Xing et al., 2024)

Parallel multi-scale convolution + SSM (Segmentation, Spectral Reconstruction):

For 3D input $X$ , compute depthwise convolutions $F_3, F_5, F_7$ (kernels $3,5,7$), concatenate, project, then SSM:

$F_\text{cat} = [F_3; F_5; F_7],\ U = W_\text{in} \cdot F_\text{cat},\ Y = \text{SSM}(U),\ \text{Out} = W_\text{out} \cdot Y + X$

(Wang et al., 25 Mar 2025)

Multi-rate temporal processing (ms-Mamba):

Parallel Mamba layers at $K$ sampling rates $\Delta_i$ :

$E^l_{m}(t,:) = \frac{1}{K} \sum_{i=1}^{K} \text{Mamba}(E^l; \Delta_i)(t,:)$

(Karadag et al., 10 Apr 2025)

Adaptive fusion (Spectral):

Fuse spatial, frequency, and spectral branches using learnable weights:

$F_\text{out} = \omega_a F_a + \omega_f F_f + \omega_e F_e + F_\text{in}$

(Zhang et al., 13 Jan 2026)

Fusion is frequently followed by pixel-wise or patch-wise supervision (e.g., a decoder for alignment loss), ensuring each scale’s contribution is propagated into model gradients. When applicable, skip connections concatenate encoder MSMM outputs to decoder inputs for multi-scale skip fusion.

3. Implementation Taxonomy and Representative Workflows

MSMMs span several canonical patterns depending on modality:

Vision (2D/3D Imaging):
- Multi-scale depthwise convolutional branches, followed by shared Mamba SSM or tri-scan directional SSMs. E.g., three branches with kernels {3,5,7}, SSM core, and pointwise projection + residual (Wang et al., 25 Mar 2025, Guan et al., 8 Jan 2026, Yang et al., 13 Jan 2025).
- Hierarchical scanning (full-res + downsampled, e.g., “MS2D”): concatenate outputs from different resolutions, optionally followed by ConvFFN for channel mixing (Shi et al., 2024).
- Fine-local to coarse-global fusion, e.g., pixel-level windowed Mamba (local) + patch-level pooled Mamba (global), then residual fusion (Yang et al., 13 Jan 2025).
Sequence Modeling / Time Series:
- Multiple parallel Mamba blocks with distinct sampling rates; module aggregates outputs via averaging or learned attention (Karadag et al., 10 Apr 2025, Jeon, 7 Dec 2025).
- FFT-based multi-scale enhancement: filter periodic components, run time-domain Mamba, fuse via adaptive gates (Zhang et al., 7 May 2025).
Multi-modal, Multi-view, Remote Sensing:
- Branches or module sequences for spatial, spectral, and cross-modal fusion (e.g., “MSpa-Mamba,” “Spe-Mamba,” “Fus-Mamba”) (Gao et al., 2024).
- Reference-centered dynamic scanning for multi-view stereo; cross-view concatenation, then independent Mamba sequence modeling for inter/intra-view context (Jiang et al., 3 Nov 2025).
- Cross-scale or cross-branch token swapping and gating for enhanced invariance and mutual supervision (Kuang et al., 1 Jun 2025).

Pseudocode for common fusion (EMMA style) (Xing et al., 2024):

def MultiScaleFusion([X_i, X_j, X_k]):
    H1 = X_i + CrossAttention(query=X_i, key=X_j, value=X_j)
    B1 = H1 + Mamba_SSM(H1)
    H2 = B1 + CrossAttention(query=B1, key=X_k, value=X_k)
    B2 = H2 + Mamba_SSM(H2)
    return B2

4. Functional Roles and Empirical Impact

MSMMs serve as adaptive, resolution-preserving bridges that rescue fine-detail features lost during deep stacking of SSM layers and enforce global structural alignment. Across applications:

Vision: MSMMs improve segmentation boundary sharpness, resolve fuzzy contours (deformable convolution), and model organ deformation (Guan et al., 8 Jan 2026, Yang et al., 13 Jan 2025, Wang et al., 25 Mar 2025). Ablation evidence shows Dice scores drop by 0.15–3.2% when MSMM fusion is removed (Yang et al., 13 Jan 2025, Guan et al., 8 Jan 2026).
Multi-modality: In EMMA, MFF (MSMM) was shown to lower hallucination rates and increase sensitivity to visual details, producing a multi-modal overall score increase by 5 points versus non-fused baselines (Xing et al., 2024).
Time Series: Parallel multi-scale Mamba blocks reduce errors by 2–4% over single-scale baselines and require orders-of-magnitude fewer parameters and MACs compared to Transformer approaches (Karadag et al., 10 Apr 2025, Jeon, 7 Dec 2025).
Spectral/Remote Sensing: Multi-perceptual fusion in M3SR improves spectral reconstruction, with ablation showing that removing any one branch (spatial, frequency, spectral) substantially increases RMSE (Zhang et al., 13 Jan 2026, Gao et al., 2024).
Efficiency: By leveraging multi-scale scanning, MSMMs cut sequence length tokens by up to 56% and lower FLOPs by ~17% without sacrificing accuracy (Shi et al., 2024, Gao et al., 2024).

Summary table of ablation results (selected modalities):

Domain	Removal Ablation (→ Score Drop)	Benchmark
Multi-modal VQA	–0.9 to –5.0 pts (EMMA MFF)	VQAv2, GQA
Segmentation	–0.15 to –3.2 Dice (MSMM off)	EchoNet, NIH
Spectral Recon	RMSE↑ by 0.003–0.005, PSNR↓	NTIRE2022
Time Series	+0.009–0.015 MSE	Solar-Energy

5. Hyperparameters, Complexity, and Training Protocols

Key hyperparameters for MSMMs include the number of scales/branches (2–4 typical), kernel sizes (3–7 for vision), sampling rates (e.g., [1,2,4,8] for time-series), and learnable fusion weights/scalars (e.g., $\alpha$ for residuals).

Parameter counts: MSMMs add 1.3–5× more parameters per block compared to vanilla Mamba, but still substantially less than Transformer self-attention or cross-attention (Wang et al., 25 Mar 2025, Zhang et al., 13 Jan 2026).
FLOPs: Linear scaling in token/pixel/voxel count via SSM; multi-scale design reduces token count per scan and overall activation memory (Shi et al., 2024).
Optimization: EMMA uses AdamW, cosine scheduler, full sharded data parallel (FSDP). Some modalities employ auxiliary supervision and PolyLoss for multi-scale alignment (Xing et al., 2024, Guan et al., 8 Jan 2026, Yang et al., 13 Jan 2025).
Initialization: Frozen vision encoder; random init for multi-scale fusion modules; pixel alignment removes the need for extra “visual scan” heads (Xing et al., 2024).

6. Adaptations and Modality-Specific Design

Design principles of MSMMs are tailored to the structural demands of each application:

Medical segmentation: Deformable convolutions in each branch adapt receptive fields to organ morphology; multi-layered decoders preserve fine-grained lesion details (Guan et al., 8 Jan 2026).
3D object detection: Window-shift and adaptive fusion strategies ensure cross-window continuity and semantic alignment at multiple spatial scales in voxel grids (Zheng et al., 17 Nov 2025).
Multi-view stereo: Dynamic scanning orders and inter-view concatenation maximize omnidirectional context propagation and feature matching (Jiang et al., 3 Nov 2025).
Remote sensing: Reduced scan redundancy via dual-resolution SSM preserves global spatial context at 37.5% lower compute cost (Gao et al., 2024).
Sequential recommendation: FFT-based multi-scale filtering enables periodic pattern modeling, adaptive gating fuses Mamba, frequency, and semantic signals for next-item accuracy (Zhang et al., 7 May 2025).

7. Empirical Validity, Limitations, and Perspectives

Comprehensive ablations and benchmarking across domains confirm MSMMs’ efficacy in multi-scale feature aggregation and context modeling, improving state-of-the-art performance while preserving linear runtime characteristics of Mamba SSMs.

However, MSMMs introduce additional parameters and modest constant-factor overhead compared to single-scale Mamba blocks. Complexity may become significant in extreme-scale models (large group counts, extensive branched fusion), necessitating careful balancing of accuracy versus computational cost (Zhang et al., 13 Jan 2026). In certain modalities, adding more scales results in diminishing returns or slight regressions on fine detail tasks (Shi et al., 2024).

In sum, MSMMs have become foundational for structurally aligned, contextually rich, and computationally efficient representation learning across modalities. Their continued evolution includes more adaptive gating, frequency-domain augmentation, and multi-modal generalization, positioning them as core modules in future unified neural architectures.