Hierarchical Gated Mamba Fusion
- Hierarchical Gated Mamba Fusion is a multimodal architecture that hierarchically aggregates multi-scale features using learned gating and Mamba state-space operators.
- It aligns visual and textual inputs into a common latent space, applying selective fusion to efficiently capture long-range dependencies.
- Empirical studies in systems like GraspMamba and GMF-Drive demonstrate enhanced performance in tasks such as grasp detection and autonomous driving.
Hierarchical Gated Mamba Fusion (GM-Fusion) denotes a family of multimodal fusion architectures that couple hierarchical feature processing with Mamba-style selective state-space models and learned gates over modalities, scales, branches, or experts. In the cited literature, the term appears explicitly in GMF-Drive and as a direct architectural extension of GraspMamba; related systems realize the same pattern through entropy-weighted branch fusion, reliability-aware Bird’s Eye View (BEV) gating, event-density-conditioned projection gates, expert routing, or cross-modal state-space scanning (Wang et al., 8 Aug 2025, Nguyen et al., 2024).
1. Definition and canonical formulation
A canonical GM-Fusion block takes multi-scale visual features and a conditioning representation , aligns them into a common latent space, computes an explicit gate, applies a Mamba-based fusion operator, and then aggregates the fused outputs hierarchically. In the generalized formulation derived from GraspMamba, the inputs are visual features , a text embedding , a Mamba-based fusion block , and a fusion gate . The per-scale computation is written as
After flattening spatial dimensions,
and the result is reshaped back to a feature map. Hierarchical aggregation is then defined as
where is bilinear upsampling plus a 0 convolution and 1 is a learned scalar gate per level (Nguyen et al., 2024).
This formulation separates three distinct design axes. First, it is hierarchical because it operates over a pyramid 2 rather than a single latent level. Second, it is gated because modality mixing and top-down aggregation are modulated by learned coefficients such as 3 and 4. Third, it is Mamba-based because the core token mixer is a selective state-space model rather than a quadratic self-attention layer. The same decomposition recurs across later systems, even when the conditioning modality is not text and the output head is not task-specific.
2. State-space foundations and Mamba operators
GM-Fusion inherits its computational core from structured state-space models. The standard continuous-time formulation used across the surveyed systems is
5
with the corresponding discretization
6
and recurrence
7
In Mamba, the effective state update becomes selective because 8, 9, and 0 are input-dependent; the practical result is linear-complexity sequence modeling with long-range context (Ye et al., 2024).
Vision-oriented GM-Fusion variants differ chiefly in how 2D or multi-scale structure is serialized before the selective scan. FusionMamba uses four-directional image flattening and reconstructs the output by summing directional branches, with an explicit multiplicative gate 1 in both the single-input Mamba block and the dual-input FusionMamba block (Peng et al., 2024). HiddenObject uses SS2D in Vision Mamba encoders and decoders, rearranging feature maps into sequences under four scan directions and then merging them back to 2D feature maps (Song et al., 28 Aug 2025). GMF-Drive introduces BEV-specific sequencing, including raster and zigzag scans, and combines them with direction-aware state transition matrices and distance decay in a BEV-SSM (Wang et al., 8 Aug 2025). MambaFusion serializes LiDAR voxels along a Hilbert space-filling curve and applies temporal Mamba over camera BEV embeddings 2 to aggregate spatiotemporal context in linear time (Narayanan et al., 8 Feb 2026). WaveMamba applies VMamba-style SS2D in the low-frequency branch after wavelet decomposition, while SMC-Mamba extends scanning to spatial-channel, cross-modal, and frequency-rotational domains (Zhu et al., 24 Jul 2025, Wang et al., 24 Dec 2025).
A central consequence is that “Mamba fusion” does not designate a single operator. In some systems Mamba is only the backbone, with fusion remaining convolutional; in others, the fusion path itself becomes state-space-driven. The distinction is architecturally important and recurs in the literature.
3. Gating mechanisms
The “gated” component of GM-Fusion is realized in multiple, non-equivalent ways. In the generalized GraspMamba extension, gating is explicit and scalar or spatial: 3 with modality blending
4
and hierarchical gating in the top-down path
5
The original GraspMamba does not include such explicit coefficients: its fusion is deterministic concatenation, 6 projection, 7 convolution, and recursive addition, and Mamba is used only in the visual backbone (Nguyen et al., 2024).
ME-Mamba uses several implicit gating mechanisms. Within each Attention-based Mamba Layer, every scan branch computes 8, 9, and then applies the Mamba-style gate 0. At cross-modal level, Optimal Transport matrices 1 and 2 act as token-level gates by selecting the most similar partner in the other modality, while the global MMD loss constrains the distributional space in which fusion is allowed (Zhang et al., 21 Sep 2025).
SlideMamba makes the gate explicit and confidence-driven. If 3 and 4 are branch probabilities, normalized entropy is
5
confidence is 6, and the fusion coefficient is
7
The fused representation is then
8
This is a scalar gate per block, derived from predictive entropy rather than from a learned cross-attention matrix (Khan et al., 25 Sep 2025).
MambaTrack couples temporal gating and cross-modal gating. Event density
9
is projected into
0
and the event-branch state transition becomes
1
At the fusion interface, the Gated Projection Fusion module computes
2
and fuses RGB into event space as
3
The same design is used symmetrically for event-to-RGB fusion (You et al., 15 Apr 2026).
MambaFusion adopts a spatially varying, reliability-aware gate. For each BEV cell 4, the descriptor
5
is passed to a small MLP with attended camera and LiDAR BEV tokens: 6 Per-cell uncertainties 7 and 8 then define inverse-variance fusion
9
This combines gating and uncertainty calibration in a single fusion rule (Narayanan et al., 8 Feb 2026).
SMC-Mamba generalizes gating to expert routing. After global average and max pooling over concatenated multimodal features, the expert weights are
0
and the fused output is
1
Inside each expert, cross-modal Mamba outputs are further gated by the opposite modality: 2 The result is a hierarchical gating stack: local gating, global gating, expert gating, and cross-modal gating (Wang et al., 24 Dec 2025).
4. Hierarchical structure across scales, branches, and domains
The hierarchical aspect of GM-Fusion is not restricted to a feature pyramid; it also appears as cross-scale recursion, wavelet sub-band decomposition, encoder-decoder skip fusion, multi-expert routing, and object-level refinement. GraspMamba is the most direct coarse-to-fine example. Its four-stage CNN–MambaVision backbone yields 3, and multimodal fusion is performed at every level. Local fusion 4 is combined with top-down context using
5
which is structurally analogous to an FPN, but with text-conditioned features at each level (Nguyen et al., 2024).
FusionMamba constructs an explicitly dual-branch hierarchy. A spatial U-Net extracts 6, a spectral U-Net extracts 7, and a dual-input FusionMamba block is inserted at each stage of the spectral path. Within each FusionMamba block, each modality conditions the other modality’s state-space parameters, and the two outputs are merged as
8
The ablation study attributes better performance to the double-U-Net hierarchy, four-directional scanning, and symmetric fusion rather than retaining only one branch output (Peng et al., 2024).
HiddenObject realizes a related hierarchy in a dual-stream encoder with four Vision Mamba blocks, an MMFF module at each stage, and a channel-wise Mamba decoder that fuses skip-connected multiscale features. MambaCAFU pushes the same idea further into a tri-branch encoder. Its main branch consists of four CoASMamba stages and one CoAMamba bottleneck, where co-attention gates combine transformer features, CNN features, and the running Mamba branch, and DoubleLCoA blocks apply gated multi-scale skip fusion in the decoder (Song et al., 28 Aug 2025, Bui et al., 4 Oct 2025).
WaveMamba introduces a different kind of hierarchy: frequency decomposition. At selected backbone stages 9, 0, and 1, DWT separates low-frequency 2 and high-frequency 3 components. The low-frequency path uses LMFB, which itself is hierarchical: Shallow Fusion Mamba first applies channel swapping and VSS-based refinement, then Deep Fusion Mamba performs deeper SS2D-based cross-modal fusion with gated attention. High-frequency components are fused by an absolute-maximum rule. The resulting architecture is hierarchical across both scale and frequency (Zhu et al., 24 Jul 2025).
GMF-Drive makes the hierarchy explicitly BEV-centric. It performs fusion at four backbone scales, and within each scale the fusion block proceeds from gated channel attention to BEV-SSM to Hierarchical Deformable Cross-Attention. The BEV-SSM itself contains multiple paths—direct, raster-scan, zigzag-scan, and multi-scale fusion—which are recombined adaptively. This is a hierarchical fusion pipeline inside a hierarchical backbone (Wang et al., 8 Aug 2025).
5. Representative instantiations and empirical evidence
The empirical literature spans robotics, pathology, remote sensing, tracking, hidden-object perception, multimodal fusion, and autonomous driving. The table lists representative instantiations in which hierarchical gating and Mamba-based fusion materially affect downstream performance.
| System | Domain | Representative evidence |
|---|---|---|
| GraspMamba (Nguyen et al., 2024) | Language-driven grasp detection | Hierarchical fusion raises 4 from 0.477 to 0.521 |
| SlideMamba (Khan et al., 25 Sep 2025) | Digital pathology | PRAUC 5 vs Mamba-only 6 |
| MambaTrack (You et al., 15 Apr 2026) | RGB-Event tracking | Full RGB+Event fusion: SR 7, PR 8; removing GPF reduces SR to 41.9 |
| GMF-Drive (Wang et al., 8 Aug 2025) | End-to-end autonomous driving | PDMS 88.85 vs baseline 88.10 |
| Fusion-Mamba (Dong et al., 2024) | RGB-IR object detection | Improves 9AP by 5.9% on 0 and 4.9% on FLIR-Aligned |
| MambaFusion (Narayanan et al., 8 Feb 2026) | Camera-LiDAR 3D detection | Full system reaches 77.9 NDS and 74.9 mAP |
In robotics grasp detection, hierarchical fusion in GraspMamba improves Seen success from 0.582 to 0.630, Unseen from 0.372 to 0.411, and 1 from 0.477 to 0.521; the same system reports real-robot gains from 0.43/0.43 to 0.54 in single-object settings and from 0.42/0.42 to 0.52 in cluttered scenes when compared with MaskGrasp and LLDG (Nguyen et al., 2024). In pathology, SlideMamba’s entropy-gated combination of a GNN branch and a Mamba branch achieves PRAUC 2, exceeding Mamba-only 3, GNN-only 4, and GAT-Mamba 5 (Khan et al., 25 Sep 2025). In RGB-Event tracking, the combination of event-adaptive DSSM and GPF yields SR 42.5 and PR 54.0; removing DSSM lowers SR to 42.1 and PR to 53.2, while removing GPF lowers SR to 41.9 and PR to 53.4 (You et al., 15 Apr 2026).
Autonomous-driving results point in the same direction. GMF-Drive improves PDMS from 88.10 for the DiffusionDrive baseline to 88.85 when 14D pillars and full GM-Fusion are used, and its ablations show that HCA alone, BEV-SSM alone, and channel attention alone are each weaker than the complete hierarchical gated design (Wang et al., 8 Aug 2025). MambaFusion reports a stepwise improvement from 71.4 NDS and 68.6 mAP for the baseline BEVTrans configuration to 77.9 NDS and 74.9 mAP for the full system with MTA, reliability-aware fusion, GCD, and temporal self-distillation (Narayanan et al., 8 Feb 2026). In RGB-IR detection, Fusion-Mamba attributes a 5.9% 6AP gain on 7 and a 4.9% gain on FLIR-Aligned to SSCS and DSSF, i.e., shallow channel swapping followed by deep dual state-space fusion in a hidden state space (Dong et al., 2024). These results do not establish a single dominant GM-Fusion template, but they do consistently associate multi-level gating plus state-space fusion with better robustness under clutter, misalignment, uncertainty, or degraded visibility.
6. Misconceptions, limitations, and active directions
A recurring misconception is that any multimodal architecture using Mamba is already GM-Fusion. The literature does not support that interpretation. GraspMamba, for example, uses Mamba only in the visual backbone; its original fusion path is CNN-based, and explicit gating appears only in the generalized extension (Nguyen et al., 2024). Conversely, FusionMamba, Fusion-Mamba, HiddenObject, and GMF-Drive place state-space operators directly in the fusion path, often with explicit multiplicative or routing gates (Peng et al., 2024, Dong et al., 2024, Song et al., 28 Aug 2025, Wang et al., 8 Aug 2025).
A second misconception is that replacing attention with a generic Mamba block is sufficient. GMF-Drive’s ablation shows that C-EffiMamba and CA+EffiMamba do not surpass the transformer baseline, whereas HCA+BEV-SSM does. The relevant distinction is not “Mamba versus transformer” in the abstract, but task-specific state-space design with directional priors, gating, and hierarchical integration (Wang et al., 8 Aug 2025). MambaCAFU makes a related point from segmentation: pure Mamba encoder-decoder designs are described as task-dependent and unstable, whereas Mamba is most effective when embedded inside a gated fusion pipeline involving CNN and transformer features (Bui et al., 4 Oct 2025).
The surveyed systems also expose concrete limitations. SlideMamba’s entropy gate is computed from branch outputs and applied uniformly within a block rather than spatially per patch, and the paper notes dependence on confidence calibration and the absence of explicit regularization for complementary branch behavior (Khan et al., 25 Sep 2025). WaveMamba uses a non-learned absolute-maximum strategy for high-frequency fusion, so the low-frequency branch is strongly optimized while the high-frequency branch remains comparatively simple (Zhu et al., 24 Jul 2025). HiddenObject is two-modality symmetric and modality-agnostic, but its published design still assumes paired streams rather than a fully general 8-modality routing mechanism (Song et al., 28 Aug 2025). MambaFusion addresses reliability and calibration explicitly, but its full detection pipeline includes token alignment, graph reasoning, and a structure-conditioned diffusion head, so the gains cannot be attributed to gating alone (Narayanan et al., 8 Feb 2026).
The most plausible synthesis is that GM-Fusion is best understood not as a single block but as an architectural principle: selective state-space modeling provides efficient long-range propagation, while gating determines which modality, scale, branch, or expert should dominate at a given point in the hierarchy. The surveyed papers suggest several directions for further refinement: soft OT rather than hard token assignment, explicit branch-level gates in multi-scan Mamba experts, spatially varying confidence gates, layer-specific transition modulation, and cross-scale consensus losses (Zhang et al., 21 Sep 2025, Nguyen et al., 2024).