Visual Mamba: Efficient SSM Vision Models

• Visual Mamba is a family of vision architectures that adapts selective state-space modeling to images, retaining long-range contextual information with linear computational cost.
• The approach replaces standard convolutions and quadratic self-attention with input-dependent state-space scans to maintain native 2D structure while efficiently modeling global interactions.
• Survey studies classify Visual Mamba into pure backbones and hybrid variants, demonstrating its broad applicability across medical imaging, remote sensing, video processing, and more.

Visual Mamba denotes a family of vision architectures that adapts Mamba’s selective structured state-space modeling from 1D sequence processing to images and feature maps. In this family, convolutional kernels and quadratic self-attention are replaced, wholly or partially, by input-dependent state-space scans, with the central design objective being to retain long-range contextual modeling and linear spatial complexity while mitigating the loss of native 2D structure induced by serialization (Liu et al., 2024, Zhang et al., 2024). Survey papers treat Visual Mamba as an umbrella category covering pure state-space backbones such as VMamba and a broader set of hybrids that combine selective state-space models with convolution, recurrence, or attention across general vision, medical imaging, remote sensing, video, and multimodal tasks (Liu et al., 2024, Xu et al., 2024).

1. Historical emergence and conceptual scope

The immediate motivation for Visual Mamba is the tension between the local inductive bias of CNNs and the quadratic token-token interaction of Vision Transformers. Surveys of the area repeatedly frame Visual Mamba as an attempt to preserve the global receptive field and input-dependent weighting associated with transformer-style processing while avoiding the $O((HW)^2)$ cost that arises when an $H\times W$ image is treated as a fully connected token graph (Zhang et al., 2024, Rahman et al., 2024). In this sense, Visual Mamba is less a single architecture than a design program: images are either flattened into scanable sequences or processed by 2D state-space formulations, and the model state acts as a compressed carrier of long-range information.

The first canonical pure vision backbone in this line is VMamba, introduced as a hierarchical “Visual State-Space Model” built from Visual State-Space (VSS) blocks and a 2D Selective Scan (SS2D) module (Liu et al., 2024). Later work diversified the family in several directions. Some models retained sequential scanning but redesigned the scan order, locality, or hierarchy; others attempted to remove the 1D bottleneck by generalizing state-space dynamics directly to two spatial dimensions, as in V2M (Wang et al., 2024). A separate branch focused on engineering efficiency, quantization, and parameter-efficient adaptation, indicating that the field rapidly moved from proof-of-concept backbones to deployment-oriented system design (Cho et al., 2024, Deng et al., 12 Mar 2025).

A persistent theme in survey literature is that “Visual Mamba” includes both foundational backbones and enhanced variants. The surveys explicitly categorize models into foundational forms and hybrids augmented with convolution, recurrence, or attention, and they place applications across high/mid-level vision, low-level vision, medical visual tasks, and remote sensing (Zhang et al., 2024, Liu et al., 2024).

2. State-space foundations and selective dynamics

The mathematical core of Visual Mamba is the continuous-time state-space model

$h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$

or, in some formulations, $y(t)=C\,h(t)+D\,x(t)$ (Liu et al., 2024, Liu et al., 2024). After zero-order-hold discretization with step $\Delta$, one obtains

$$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$

and the discrete recurrence

$$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$

which admits both a recurrent interpretation and a convolutional interpretation through a long kernel induced by powers of $\overline A$ (Liu et al., 2024, Xu et al., 2024).

What differentiates Mamba from classical linear time-invariant SSMs is selectivity. Survey treatments describe the key innovation as making $B$, $C$, and $H\times W$0 input-dependent through lightweight selector networks, so that each token modulates the local state dynamics rather than passing through a fixed kernel (Zhang et al., 2024, Xu et al., 2024). This selective parameterization is the main reason Visual Mamba is often discussed as an alternative to attention rather than merely a recurrence: the recurrence is dynamic, content-conditioned, and implemented with hardware-aware scan kernels.

VMamba further recasts this 1D selective scan into a 2D setting. Its SS2D module unfolds a feature map into multiple directional sequences, scans each sequence with an S6-style selective SSM, and merges the outputs back into the original spatial layout (Liu et al., 2024). The VMamba paper also shows that the selective-SSM output can be written in a form resembling gated linear attention, but computed in $H\times W$1 time through associative scan rather than dense pairwise products (Liu et al., 2024). This attention-like interpretation has been important for later interpretability work on vision-based Mamba models (Wang et al., 28 Feb 2025).

3. Spatialization: scan order, 2D structure, and locality

The principal difficulty in adapting Mamba to vision is that images are not naturally causal 1D sequences. Survey work therefore treats scanning as the decisive design space of Visual Mamba and decomposes it into scan mode, scan axis, scan continuity, and scan sampling. The reported categories include single-direction versus bidirectional scans, horizontal/vertical/diagonal/depth axes, raster versus zigzag continuity, and global versus local versus atrous sampling (Xu et al., 2024). This taxonomy captures the fact that most Visual Mamba variants differ less in state-space algebra than in how they serialize or route 2D information.

VMamba addresses non-causality through cross-scan. Given a feature map, it generates four traversal orders—row-major, reverse row-major, column-major, and reverse column-major—applies an S6 scan to each route, and sums or averages the four outputs after restoring the original spatial arrangement (Liu et al., 2024). The VSS block combines this global mixing with a local branch, typically involving depth-wise convolution and gating, so that local texture and long-range context are modeled jointly (Liu et al., 2024).

Later models make this spatialization more explicit. V2M argues that flattening 2D images into 1D sequences necessarily loses 2D locality and proposes a 2D SSM derived from Roesser’s 2D state-space formulation, implemented as a row pass followed by a column pass and evaluated from four image corners in parallel (Wang et al., 2024). DefMamba makes the scan path itself content-aware by predicting point offsets and index offsets before scanning, so that sampling locations and token order can drift toward salient regions (Liu et al., 8 Apr 2025). This suggests a shift from fixed, hand-crafted scan topology toward learnable scan geometry.

Interpretability work confirms that scan order is not a minor implementation detail. A visual analytics study of vision-based Mamba models reports that different blocks within the same stage learn markedly different attention patterns, that early stages are strongly locality-driven while late stages become more content-sensitive, and that alternative orders such as diagonal, Morton-Z, and spiral achieve comparable accuracy at $H\times W$2 while inducing different patch-attention organization (Wang et al., 28 Feb 2025). The surveys likewise note that scan ordering injects a strong bias and remains a central open problem (Zhang et al., 2024).

4. Representative backbone families

Several backbone lines define the current Visual Mamba landscape. “Vision Mamba” (Vim), as summarized in the 2025 survey, follows the ViT patch-token paradigm but replaces transformer blocks with bidirectional SSM scans, typically organized in a four-stage hierarchy with patch embedding, patch merging, and task heads for classification, detection, or segmentation (Ibrahim et al., 11 Feb 2025). VMamba instead centers on VSS blocks and SS2D cross-scan as the first pure selective-SSM vision backbone (Liu et al., 2024). V2M generalizes the state-space update itself to 2D, while DefMamba and SF-Mamba alter the routing mechanism to improve structural fidelity or throughput (Wang et al., 2024, Liu et al., 8 Apr 2025, Yoshimura et al., 17 Mar 2026).

Backbone	Distinctive mechanism	Selected reported result
Vim	Bidirectional SSM scans over patch tokens in a hierarchical vision backbone	ViM-Tiny reaches AP box 42.3 on COCO and mIoU 44.7 on ADE20K in survey-reported benchmarks (Ibrahim et al., 11 Feb 2025)
VMamba	VSS blocks with SS2D and four-route cross-scan	VMamba-B achieves 83.9% Top-1 at 224×224 (Liu et al., 2024)
V2M	2D SSM via row-to-column passes from four corners	V2M-B achieves 83.8% Top-1 (Wang et al., 2024)
DefMamba	Deformable Scanning with learnable point and index offsets	DefMamba-B records 84.2% Top-1 (Liu et al., 8 Apr 2025)
SF-Mamba	Auxiliary patch swapping and batch folding with periodic state reset	SF-Mamba-Base reaches 84.4% Top-1 (Yoshimura et al., 17 Mar 2026)

Within VMamba itself, architectural and implementation refinements were essential. The original VMamba optimization sequence reports that a vanilla VMamba-Tiny (V0) reached only $H\times W$3 img/s at $H\times W$4 because of Python loops, whereas successive changes culminating in V9 reached $H\times W$5 img/s with unchanged accuracy of $H\times W$6 (Liu et al., 2024). This is a recurring pattern in the literature: algorithmic linearity does not automatically translate into efficient end-to-end kernels.

Efficiency-oriented variants diverge in how they attack this systems problem. SF-Mamba introduces auxiliary patch swapping to encode bidirectional information flow under a unidirectional scan and batch folding with periodic state reset to improve GPU parallelism for short token lengths (Yoshimura et al., 17 Mar 2026). MobileMamba takes a different route, using a three-stage design and a Multi-Receptive Field Feature Interaction module that combines Wavelet Transform-Enhanced Mamba, Efficient Multi-Kernel Depthwise Convolution, and identity pruning; it reports up to $H\times W$7 Top-1 and up to $H\times W$8 faster throughput than LocalVim on GPU (He et al., 2024).

5. Benchmark profile and application domains

On standard vision benchmarks, VMamba established that a pure SSM backbone could be competitive with contemporary CNN and transformer baselines. Trained on ImageNet-1K for 300 epochs, VMamba-T/S/B report Top-1 accuracy of $H\times W$9, $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$0, and $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$1 at $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$2; on downstream dense prediction, Mask R-CNN with VMamba-T/S/B yields box AP $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$3 and mask AP $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$4, while UperNet on ADE20K yields mIoU $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$5 (Liu et al., 2024). The same work emphasizes input scaling efficiency: when tested from $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$6 to $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$7 without fine-tuning, VMamba’s Top-1 degrades from $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$8 to $h'(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t),$9 and inference throughput at $y(t)=C\,h(t)+D\,x(t)$0 remains approximately $y(t)=C\,h(t)+D\,x(t)$1 img/s (Liu et al., 2024).

Alternative backbones show that performance gains are not confined to a single serialization strategy. V2M-B reaches $y(t)=C\,h(t)+D\,x(t)$2 Top-1 on ImageNet-1K and, in downstream settings, V2M-B* reaches box AP $y(t)=C\,h(t)+D\,x(t)$3 and mask AP $y(t)=C\,h(t)+D\,x(t)$4 on COCO and mIoU $y(t)=C\,h(t)+D\,x(t)$5 on ADE20K (Wang et al., 2024). DefMamba-S reports box AP $y(t)=C\,h(t)+D\,x(t)$6, mask AP $y(t)=C\,h(t)+D\,x(t)$7, and ADE20K mIoU $y(t)=C\,h(t)+D\,x(t)$8 single-scale and $y(t)=C\,h(t)+D\,x(t)$9 multi-scale (Liu et al., 8 Apr 2025). SF-Mamba-Tiny reaches $\Delta$0 mIoU on ADE20K and AP box $\Delta$1, AP mask $\Delta$2 with Cascade Mask R-CNN while also reporting higher throughput than MambaVision and Swin counterparts in the cited settings (Yoshimura et al., 17 Mar 2026).

Visual Mamba methods were quickly applied beyond generic recognition. In image quality assessment, QMamba is described as the first exploration of Vision Mamba for IQA and adapts a VMamba-like backbone with LocalMamba windowed scans for task-specific, universal, and transferable blind IQA (Guan et al., 2024). It reports task-specific average SRCC $\Delta$3 for QMamba-T/S and $\Delta$4–$\Delta$5 for LocalQMamba, universal SRCC $\Delta$6 for LocalQMamba-B, and transferable SRCC $\Delta$7 for StylePrompt with $\Delta$8 M extra parameters versus $\Delta$9 for full fine-tuning (Guan et al., 2024).

In crack segmentation, a VMamba-based encoder-decoder network reports $$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$0 M parameters versus $$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$1–$$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$2 M in comparison models, $$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$3 G FLOPs versus $$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$4–$$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$5 G FLOPs, inference time of $$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$6 ms/image versus $$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$7–$$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$8 ms, and up to $$\overline A=\exp(\Delta A),\qquad \overline B=(\Delta A)^{-1}\bigl(\exp(\Delta A)-I\bigr)\,\Delta B,$$9 fewer FLOPs at higher resolutions, while achieving mDS/mIoU values of $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$0 on Crack500, $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$1 on Ozgenel, and $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$2 on MC448 (Chen et al., 2024). In medical segmentation, Mamba-UNet reports Dice/IoU/HD95/ASD of $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$3 on ACDC and $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$4 on Synapse, outperforming several UNet-style baselines under the same hyper-parameter setting (Wang et al., 2024). Under scribble supervision, Weak-Mamba-UNet reports Dice $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$5, Accuracy $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$6, Precision $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$7, Sensitivity $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$8, HD $$h_t=\overline A\,h_{t-1}+\overline B\,x_t,\qquad y_t=C\,h_t,$$9, and ASD $\overline A$0 on ACDC (Wang et al., 2024).

The family has also shown utility in smaller or specialized datasets. On breast ultrasound classification, the best Mamba-based models on the B dataset show a $\overline A$1 average AUC and a $\overline A$2 average Accuracy improvement compared to the best non-Mamba-based model in that study (Nasiri-Sarvi et al., 2024). For 3D porous media permeability prediction, a Vision Mamba backbone with $\overline A$3 parameters reports $\overline A$4 and RMSE $\overline A$5 mD at patch size $\overline A$6, while the matched ViT has $\overline A$7 parameters and the CNN baseline has $\overline A$8 parameters (Kashefi et al., 16 Oct 2025). Survey papers place such task-specific results within a larger application map spanning restoration, generation, remote sensing, medical imaging, video, and multimodal learning (Zhang et al., 2024, Xu et al., 2024).

6. Adaptation, prompting, and quantization

As pre-trained Visual Mamba backbones proliferated, efficient adaptation became a separate research thread. In IQA, QMamba introduces StylePrompt, motivated by the claim that synthetic, authentic, and AIGC IQA domains differ mainly in feature statistics such as mean and variance (Guan et al., 2024). StylePrompt generates lightweight prompt vectors, maps them to affine channel parameters $\overline A$9, and modulates feature maps by

$B$0

Its reported parameter cost is only $B$1 M extra parameters, approximately $B$2 of Q-Mamba-Base’s $B$3 M, compared with $B$4–$B$5 M for cross-attention or convolutional prompt schemes (Guan et al., 2024).

Selective Visual Prompting extends efficient tuning to pre-trained Vim models. It argues that prepended prompt tokens are poorly matched to sequential token-wise state propagation, because they influence mainly early positions and do not adequately activate update and forget gates across the full sequence (Yao et al., 2024). SVP therefore generates token-wise prompts through a dual-path design of Cross-Prompting and Inner-Prompting, keeps the pre-trained Vim weights frozen, and reports an average accuracy of $B$6 on the HTA benchmark with only $B$7 M tunable parameters, compared with $B$8 for full fine-tuning; on VTAB-1k it reports an overall average of $B$9 (Yao et al., 2024).

Quantization turned out to be unusually delicate for Visual Mamba. PTQ4VM identifies three model-specific difficulties arising from fixed-order sequential token processing: token-wise variance, channel-wise outliers, and a long tail of activations (Cho et al., 2024). It introduces Per-Token Static quantization and Joint Learning of Smoothing Scale and Step Size, converts pre-trained backbones to quantized form in under 15 minutes, and reports up to $C$0 GPU speedup with negligible accuracy loss compared to FP16 in the cited settings (Cho et al., 2024). ViM-VQ reaches lower-bit compression through codebook-based vector quantization tailored to the weight distributions of Visual Mamba blocks; for ImageNet-1K classification it reports, for example, Vim-Tiny Top-1 of $C$1 at 3-bit, $C$2 at 2-bit, and $C$3 at 1-bit (Deng et al., 12 Mar 2025).

A plausible implication is that adaptation and deployment are not ancillary concerns for Visual Mamba but part of its architectural identity. The literature repeatedly couples state-space modeling with hardware-aware kernels, frozen-backbone prompt tuning, and post-training compression, indicating that efficiency claims are evaluated at the combined algorithm–system level rather than at asymptotic complexity alone (Cho et al., 2024, He et al., 2024).

7. Limitations, misconceptions, and open problems

A common misconception is that linear complexity automatically guarantees superior throughput. Multiple papers qualify this. Surveys note that Mamba does not always outperform Transformers in raw efficiency on modern hardware (Xu et al., 2024). MobileMamba states that current lightweight Mamba-based models exhibit suboptimal throughput despite low FLOPs (He et al., 2024). SF-Mamba identifies short-token underutilization of GPU threads as a concrete bottleneck for visual workloads and proposes batch folding specifically to address it (Yoshimura et al., 17 Mar 2026). The technical record therefore distinguishes asymptotic linearity from realized latency.

A second misconception is that scan order is a superficial choice. The surveys and interpretability studies indicate the opposite: scan ordering injects a strong inductive bias, affects spatial continuity, and changes learned attention organization even when accuracy remains similar across permutations (Zhang et al., 2024, Wang et al., 28 Feb 2025). This is one reason later work moved toward local windows, multi-route scans, 2D SSMs, or deformable scans instead of relying on a single fixed rasterization (Wang et al., 2024, Liu et al., 8 Apr 2025).

A third limitation concerns spatial structure. Flattening a 2D map into 1D may separate neighboring pixels, weaken local coherence, and require auxiliary convolutional branches to restore local bias (Wang et al., 2024, Xu et al., 2024). V2M’s direct 2D formulation and DefMamba’s deformable scanning can be read as explicit attempts to remove this bottleneck. This suggests that future progress may depend less on enlarging state dimension than on better spatial operators.

Survey papers converge on several open directions: learnable or data-driven scan orders, hybrid SSM–convolution or SSM–attention blocks, large-scale pre-training, quantization and sparsity for deployment, extension to high-resolution imagery and video, and a deeper theoretical account of input-dependent SSM expressivity relative to self-attention (Zhang et al., 2024, Liu et al., 2024). Application papers add domain-specific variants of the same agenda. QMamba points to domain-specific pre-training, quantization or pruning, and extension to video and medical imaging as next steps (Guan et al., 2024). Taken together, these works portray Visual Mamba not as a settled replacement for CNNs or transformers, but as an active research area centered on how selective state-space dynamics should be spatialized, optimized, adapted, and interpreted in vision.