Mamba-X: Hybrid SSM Architectures

• Mamba-X is a design space of extended/hybrid Mamba systems that combine input-adaptive selective SSMs with modules like CNNs, attention layers, and token learners.
• It enables linear-time sequence processing while flexibly integrating task-specific modules for applications ranging from multilingual ASR to image super-resolution.
• Mamba-X systems leverage dynamic interoperation, progressive sampling, and hardware co-design to optimize efficiency, performance, and scalability across diverse domains.

Mamba-X denotes an emerging family of extended or hybrid Mamba systems built around selective state-space models (SSMs), rather than a single standardized architecture. In current usage, the term covers task-specific designs in which Mamba is combined with CNN stems, attention layers, mixture-of-experts routing, token learners, curriculum schedules, parameter-efficient fine-tuning, dynamic early exits, or hardware co-design; it also names particular instantiated systems, including an end-to-end Vision Mamba accelerator for edge devices and a nonlinear state predictive control model for super-resolution (Yoon et al., 5 Aug 2025, Li et al., 22 Nov 2025). Across these usages, the common premise is that Mamba’s input-dependent SSMs can replace or complement attention while retaining linear-time sequence processing.

1. Scope and nomenclature

“Mamba-X” is used informally in multiple papers as shorthand for “extended Mamba” or “hybrid Mamba” designs. In multilingual ASR, MLMA is explicitly framed as “the kind of experiment you would run if you were exploring a broader ‘Mamba-X’ family,” namely whether a Mamba-style SSM can serve as the core sequence model of a practical multilingual speech recognizer (Ali et al., 21 Oct 2025). In industrial XCT inspection, NL-MambaXCT is described as a “typical Mamba-X style model” in the community sense of an extended Mamba backbone augmented with hybrid CNN stages, self-supervision, and nested optimization dynamics (Aldoboni et al., 25 May 2026). Retrieval, architecture adaptation, and survey papers use the term similarly: as a label for next-generation retrievers, Transformer-to-Mamba adaptation pipelines, or large-scale visual and multimodal Mamba systems (Zhang et al., 2024, Chen et al., 21 Feb 2025, Xu et al., 2024).

This usage implies that Mamba-X is best understood as a design space rather than a closed model lineage. Some works retain a standard Mamba kernel and move innovation to scanning, fusion, routing, or optimization; others alter the parameterization around the SSM itself. A notable example of the latter is MambaX for super-resolution, which replaces fixed or linearly generated control terms with nonlinear, dynamically learned discretization and output mappings (Li et al., 22 Nov 2025).

A separate, narrower usage is nominal. “Mamba-X” is also the title of a specific hardware paper that proposes an end-to-end Vision Mamba accelerator for edge computing devices, while “MambaX” names a super-resolution architecture based on nonlinear state predictive control (Yoon et al., 5 Aug 2025, Li et al., 22 Nov 2025). The shared label therefore refers both to a general architectural tendency and to individual named systems.

2. Core architectural principles

The common substrate is the selective SSM. In the discrete-time form used across several of these works, Mamba evolves a latent state as

$$h_t = \bar{A} h_{t-1} + \bar{B} x_t, \qquad y_t = C h_t,$$

with Zero-Order Hold discretization

$$\bar{A} = e^{\Delta A}, \qquad \bar{B} = (\Delta A)^{-1}\big(e^{\Delta A}-I\big)\,\Delta B.$$

Mamba departs from classical linear time-invariant SSMs by making $B$, $C$, and $\Delta$ input- and time-dependent, typically written as

$$B=f_B(x), \qquad C=f_C(x), \qquad \Delta=\mathrm{Broadcast}_D(f_\Delta(x)).$$

This selective mechanism gives linear-time sequence processing together with non-stationary, input-adaptive dynamics (Ali et al., 21 Oct 2025).

A first Mamba-X pattern is direct backbone substitution. MLMA uses a CNN front-end followed by an 18-layer ConMamba encoder and a CTC head for multilingual ASR; Mamba Retriever uses a Mamba bi-encoder with <EOS> pooling and cosine similarity for dense retrieval; LE-PDE++ replaces the latent evolution model of LE-PDE with a Mamba-based latent dynamics engine (Ali et al., 21 Oct 2025, Zhang et al., 2024, Liang et al., 2024). In all three cases, the primary claim is not that the SSM kernel is new, but that a Mamba backbone can replace attention or other latent dynamics modules in a full task pipeline.

A second pattern is hybridization with local modules. MambaNeXt-YOLO uses a MambaNeXt block that couples a ConvNeXt-style local branch, a Mamba global branch, and ResGate fusion; NL-MambaXCT uses RegNet blocks in early stages and Mamba mixers in deeper stages; MxT places Mamba beside a Spatial Reduced Self-Attention branch inside a hybrid inpainting module; Mamba Policy uses XMamba blocks with FiLM conditioning, a Mamba mixer, then an attention mixer (Lei et al., 4 Jun 2025, Aldoboni et al., 25 May 2026, Chen et al., 2024, Cao et al., 2024). The recurrent design principle is local–global factorization: convolution handles short-range texture or acoustic patterns, while Mamba handles long-range dependencies with linear sequence scaling.

A third pattern is scan engineering. The Visual Mamba survey identifies scan mode, scan axis, scan continuity, and scan sampling as central adaptation knobs for vision (Xu et al., 2024). Mamba-in-Mamba pushes this furthest for hyperspectral classification through centralized Mamba-Cross-Scan, Gaussian Decay Masking, Semantic Token Learner, and Semantic Token Fuser, so that every scan converges on the center pixel that defines the supervision target (Zhou et al., 2024). This suggests that, in visual Mamba systems, serialization is often as consequential as the SSM block itself.

A fourth pattern is dynamic interoperation with attention or sparse experts. TransMamba proposes shared parameter matrices between attention and SSM views, a Memory Converter for transferring attention prefixes into SSM-compatible states, and layer-specific TransPoint schedules for switching mechanisms within a layer (Li et al., 31 Mar 2025). BlackMamba instead combines Mamba with top-1 routed MoE blocks, using Sinkhorn routing for load balancing and the layer update

$$x_{l+1}=x_l+\mathrm{MoE}\big(\mathrm{LN}(x_l+\mathrm{mamba}(\mathrm{LN}(x_l)))\big),$$

which couples linear-time sequence mixing with sparse expert capacity (Anthony et al., 2024).

3. Adaptation, training, and dynamic computation

A large fraction of Mamba-X work focuses on how to adapt or fine-tune Mamba backbones rather than on altering the SSM equations. TransMamba is the clearest cross-architecture example: it treats a pretrained Transformer as teacher and a Mamba model as student, aligns intermediate features in a latent space, initializes the student through weight subcloning, and transfers knowledge with Weight Subcloning and Adaptive Bidirectional distillation. The reported result is stronger performance than training Mamba from scratch while using less than 75% of the training data typically required for scratch training (Chen et al., 21 Feb 2025).

Self-supervision and two-timescale optimization appear in domain-specialized Mamba-X systems. NL-MambaXCT pretrains on 19,961 unlabeled industrial XCT slices with masked image modeling, then fine-tunes on 2,000 labeled slices while adding Nested Learning at two levels: ContinuumLinear fast/slow projections inside Mamba-based stages and classifier head, and Deep Momentum SGD at the optimizer level. On the held-out production-order split, the MIM-pretrained NL-MambaXCT model reaches 96.91% accuracy and 96.8% macro F1, outperforming CNN, attention, and single-timescale Mamba baselines by 3.11–10.31 percentage points in accuracy (Aldoboni et al., 25 May 2026).

Curriculum-like scheduling is another recurring extension. LE-PDE++ couples its Mamba latent dynamics module with progressive sampling, exposing the model first to short temporal contexts and gradually extending the visible horizon. On Navier–Stokes, the best reported setting uses logarithmic growth with $\tau_0=0.3$ and achieves RMSE 0.21, improving over both the original LE-PDE and the no-progressive-sampling Mamba version (Liang et al., 2024). This is one of the clearest examples in which the benefit of Mamba is contingent on a task-specific training schedule.

Parameter-efficient tuning also appears in multimodal medical generation. EMRRG fine-tunes a pretrained X-ray Mamba network for radiology report generation and reports that Partial LoRA yields optimal performance for the SSM-based vision backbone, while the decoder is a hybrid LLM-based generator trained end-to-end (Zhang et al., 19 Oct 2025). The paper explicitly positions this against frameworks that either avoid vision fine-tuning or use simplistic LoRA without strengthening the cross-attention pathway.

Dynamic inference is addressed by DYNAMAX, which integrates early exits into decoder-only Transformers and Mamba models and repurposes a small Mamba block as an efficient early-exit classifier. On Mistral 7B and Codestral 7B Mamba, Mamba-based exit heads outperform FFN and CALM-style exit heads in the compute–performance trade-off, although overly aggressive thresholds can cause degenerate repetition (Nogales et al., 29 Apr 2025).

4. Representative instantiations across domains

The current Mamba-X landscape is highly application-specific.

System	Domain	Defining extension
MLMA (Ali et al., 21 Oct 2025)	Multilingual ASR	CNN front-end + 18-layer ConMamba encoder + CTC
Mamba Retriever (Zhang et al., 2024)	Dense retrieval	Mamba bi-encoder with <EOS> pooling and InfoNCE
LE-PDE++ (Liang et al., 2024)	PDE surrogates	Mamba latent dynamics + progressive sampling
BlackMamba (Anthony et al., 2024)	Language modeling	Mamba backbone + top-1 MoE with Sinkhorn routing
MxT (Chen et al., 2024)	Image inpainting	Hybrid Module with SRSA + Mamba + CBFN
Mamba Policy (Cao et al., 2024)	3D diffusion policy	XMamba block with FiLM, Mamba, attention, MLP
Mamba-in-Mamba (Zhou et al., 2024)	HSI classification	Centralized cross-scan + GDM + STL/STF
MambaNeXt-YOLO (Lei et al., 4 Jun 2025)	Real-time detection	CNN–Mamba hybrid block + MAFPN
NL-MambaXCT (Aldoboni et al., 25 May 2026)	Industrial XCT classification	RegNet+Mamba backbone + MIM + Nested Learning
MambaX (Li et al., 22 Nov 2025)	Super-resolution	Nonlinear state predictive control + cross-control fusion
TransMamba (Li et al., 31 Mar 2025)	Long-context LM	Shared attention/SSM weights + Memory Converter
EMRRG (Zhang et al., 19 Oct 2025)	X-ray report generation	Pretrained X-ray Mamba + Partial LoRA + hybrid decoder

Empirically, these systems cover a broad range of performance regimes. In ASR, ConMamba achieves 4.05 test-clean and 10.50 test-other WER on LibriSpeech, compared with 4.27 and 11.29 for a comparable Conformer, and the six-language MLMA model remains competitive across CommonVoice, VoxPopuli, MLS, and out-of-domain FLEURS (Ali et al., 21 Oct 2025). In retrieval, Mamba Retriever scales from 130M to 790M parameters with monotonic gains on MS MARCO and BEIR, reaches 90.7 average nDCG@10 on LoCoV0 at 8k tokens, and outperforms M2-BERT-8k and Jina-v2 in that setting (Zhang et al., 2024). In scientific computing, LE-PDE++ doubles inference speed relative to LE-PDE, and with progressive sampling obtains RMSE 0.21 on Navier–Stokes, 0.30 on Shallow Water Equations, and 0.42 on the Pollutant Transport Equation benchmark (Liang et al., 2024).

Vision and imaging instantiations are similarly heterogeneous. MambaNeXt-YOLO reports 66.6% mAP at 31.9 FPS on PASCAL VOC without pre-training and 27.8% AP on DOTA v1.5, while remaining deployable on Jetson Orin NX and Xavier NX (Lei et al., 4 Jun 2025). Mamba-in-Mamba reaches 92.08% OA on Indian Pines, 91.58% on Pavia University, and 92.89% on Houston 2013 under fixed disjoint splits, outperforming ViM and several HSI Transformers (Zhou et al., 2024). MambaX reports best or near-best PSNR, SSIM, SAM, ERGAS, and CC across CAVE, Chikusei, Pavia, WorldView-III, and Gaofen-2, with explicit gains over PanMamba and FusionMamba in multimodal fusion SR (Li et al., 22 Nov 2025). In robotics, Mamba Policy reduces parameter count by over 80% and FLOPs by 90.0% relative to the original DP3 policy network while improving average success rate from 68.9 to 70.1 across Adroit, MetaWorld, and DexArt (Cao et al., 2024).

5. Efficiency, deployment, and hardware co-design

Efficiency is the dominant argument for Mamba-X, but it manifests at several layers of the stack. Algorithmically, Mamba Retriever shows that passage encoding at 2k tokens is about 4× faster than standard Transformer-based models, and at 8k–32k tokens it remains faster than M2-BERT by factors of about 1.2× to 1.4× (Zhang et al., 2024). LE-PDE++ reports inference time of $6 \pm 0.6$ ms versus $12 \pm 2.5$ ms for LE-PDE on Navier–Stokes, with slightly fewer parameters (Liang et al., 2024). BlackMamba combines linear-complexity generation from SSMs with sparse MoE computation and reports competitive downstream performance against both Mamba and Transformer baselines while improving training and inference FLOPs (Anthony et al., 2024).

At the model-architecture level, Mamba Policy reports 47.9M parameters, 0.03G FLOPs, and 137.7 MB training-time GPU memory, versus 255.1M, 0.30G, and 996.1 MB for DP3, while improving several hard manipulation tasks (Cao et al., 2024). NL-MambaXCT retains only 9.03M parameters and 2.57 GFLOPs, yet runs at 11.01 FPS on CPU and exceeds ResNet-50, RegNetY, EfficientNet, ViT-B/16, Swin-T, and vanilla MambaXCT on the production-order test set (Aldoboni et al., 25 May 2026). MambaNeXt-YOLO is lighter than several YOLO baselines and remains real-time on embedded GPUs, although the paper notes that the fastest pure-CNN YOLO variants still achieve higher FPS (Lei et al., 4 Jun 2025).

One specific “Mamba-X” is hardware rather than algorithmic. The accelerator paper titled “Mamba-X” proposes an end-to-end Vision Mamba accelerator with a DMA unit, GEMM engine, VPU, SFU, Systolic Scan Array, and PPU with Long Input Support Unit. Its central hardware innovation is a systolic scan array for selective SSMs, coupled with a hybrid, hardware-friendly INT8 quantization scheme that uses tensor-level scales for weights, channel-wise scales for selective SSM activations, and power-of-two approximations for key scaling factors (Yoon et al., 5 Aug 2025). Relative to Jetson AGX Xavier, the reported gains are 11.6× speedup and about 11.5× energy-efficiency improvement on the selective SSM core, about 2.5× lower off-chip traffic in that block, and 2.3× end-to-end Vision Mamba speedup, with Top-1 degradation always below 1%-point on ImageNet-1K (Yoon et al., 5 Aug 2025). This is a distinct sense in which Mamba-X denotes not a model family but a hardware realization of Vision Mamba’s linear-time promise.

6. Limitations, misconceptions, and open directions

A common misconception is that Mamba-X denotes a single new backbone or a single new SSM kernel. The literature does not support that reading. Several representative systems explicitly retain the standard Mamba block and place novelty around it in scan strategy, hybrid fusion, self-supervision, routing, or optimization (Aldoboni et al., 25 May 2026, Lei et al., 4 Jun 2025, Chen et al., 2024). Even TransMamba’s main novelty lies in shared parameterization, Memory Converter design, and TransPoint scheduling rather than in redefining the selective scan itself (Li et al., 31 Mar 2025). MambaX for super-resolution is an exception in that it nonlinearizes the control parameterization through dynamic $$\bar{A} = e^{\Delta A}, \qquad \bar{B} = (\Delta A)^{-1}\big(e^{\Delta A}-I\big)\,\Delta B.$$0 and $$\bar{A} = e^{\Delta A}, \qquad \bar{B} = (\Delta A)^{-1}\big(e^{\Delta A}-I\big)\,\Delta B.$$1 generators, but it still frames itself as a composition of linear SSMs with nonlinear parameter generators (Li et al., 22 Nov 2025).

Another misconception is that replacing attention with Mamba is sufficient on its own. The strongest systems in these papers are usually hybrids. MambaNeXt-YOLO depends on a ConvNeXt local branch, MAFPN, and ResGate; Mamba Policy inserts self-attention after Mamba in XMamba blocks; MxT uses reduced self-attention beside Mamba; BlackMamba delegates much of capacity to sparse experts; and Mamba-in-Mamba derives much of its advantage from centralized cross-scan and semantic tokenization rather than from a raw SSM swap (Lei et al., 4 Jun 2025, Cao et al., 2024, Chen et al., 2024, Anthony et al., 2024, Zhou et al., 2024). This suggests that Mamba-X is often best interpreted as a structured division of labor between recurrence, locality, sparsity, and explicit interaction modules.

The limitations reported across domains are substantial. MLMA is restricted to six European languages, uses no explicit language-ID conditioning, and does not solve cross-language data imbalance (Ali et al., 21 Oct 2025). NL-MambaXCT is 2D slice-level, offline, and most robust when acquisition geometry and reconstruction protocol match training conditions (Aldoboni et al., 25 May 2026). LE-PDE++ relies on a latent linear time-invariant assumption and on carefully tuned progressive sampling; without that curriculum its Navier–Stokes RMSE degrades from 0.21 to about 0.31 (Liang et al., 2024). Mamba Retriever is still trained in a relatively simple dense-retrieval regime without the full repertoire of hard-negative mining or multi-stage supervision (Zhang et al., 2024). TransMamba depends on strong Transformer teachers and does not fully close the gap to those teachers on every task (Chen et al., 21 Feb 2025). DYNAMAX introduces pathological repetition at low exit thresholds (Nogales et al., 29 Apr 2025). The hardware Mamba-X accelerator is specialized to Vision Mamba inference and does not address training or arbitrary SSM variants (Yoon et al., 5 Aug 2025).

The survey perspective points to broader open problems: causality mismatch for non-causal visual data, scan redundancy, instability when scaling visual Mamba models, interpretability of hidden-state dynamics, and robustness under domain shift (Xu et al., 2024). A plausible implication is that future Mamba-X systems will continue to move in four directions already visible in this corpus: more adaptive scan schedules, tighter local–global hybrids, richer multimodal connectors, and stronger systems co-design spanning PEFT, dynamic inference, and accelerator support.