---
title: 'VCMamba: Hybrid CNN-SSM Vision Backbone'
url: https://www.emergentmind.com/topics/vcmamba
type: topic
---

# VCMamba: Hybrid CNN-SSM Vision Backbone

VCMamba is a hierarchical hybrid vision backbone that integrates convolutional feature extraction with multi-directional Mamba state-space modeling for efficient visual representation. It was introduced to bridge a recurrent trade-off in computer vision backbone design: CNNs retain strong local inductive bias but have limited long-range reasoning, whereas Vision Transformers and vision SSMs can model global context more effectively but may underutilize fine-grained local structure, especially in early processing. VCMamba addresses this by using a convolutional stem and convolutional blocks in its early stages, then introducing multi-directional Mamba blocks in the deepest stage to model long-range dependencies while maintaining linear complexity with respect to image resolution [2509.04669].

## 1. Conceptual positioning within visual Mamba research

VCMamba belongs to the broader family of visual Mamba backbones that adapt selective state-space models to 2D vision. In that design space, the central difficulty is that Mamba is fundamentally a 1D sequence model, whereas images are 2D and non-causal. Broader surveys on visual Mamba do not explicitly discuss VCMamba, but they identify scanning strategy, bidirectionality, and hierarchical design as the key axes along which visual Mamba methods differ [2404.18861]. A later survey on Mamba architecture for vision applications likewise does not define VCMamba directly, but emphasizes selective state-space modeling, bidirectional scanning, hierarchical organization, and local-global feature extraction as recurring design themes in vision-oriented Mamba systems [2502.07161].

The immediate motivation of VCMamba is framed against three families of backbones. CNNs possess strong inductive biases for local features but have fixed receptive fields. Vision Transformers capture global interactions well but incur quadratic complexity in the number of patches. Existing Mamba-based vision models, including VMamba, Vim, and PlainMamba, are presented as attractive because Mamba offers linear-time sequence modeling, but the paper argues that many such models still rely heavily on patch embeddings or 1D sequence views of images and therefore may not fully exploit convolutional local feature extraction [2509.04669].

This places VCMamba in a distinctive subline of vision Mamba research. VMamba adapts Mamba to images through a hierarchical stack of VSS blocks with SS2D and four scanning routes across 2D feature maps [2401.10166]. By contrast, VCMamba reserves Mamba-based global modeling for the final stage of a CNN-like pyramid and lets convolutional stages dominate early high-resolution processing [2509.04669]. A plausible implication is that VCMamba treats Mamba less as a universal replacement for all visual computation and more as a late-stage global-context module.

## 2. Backbone organization and model variants

The architecture begins with a convolutional stem composed of two sequential \(3 \times 3\) convolutions, each with stride 2 and each followed by BatchNorm and ReLU. This reduces the input resolution to \(H/4 \times W/4\). After the stem, the network has four hierarchical stages. Between stages, downsampling is performed by a strided \(3 \times 3\) convolution followed by BatchNorm, producing the standard pyramid resolutions \(H/4 \times W/4\), \(H/8 \times W/8\), \(H/16 \times W/16\), and \(H/32 \times W/32\) [2509.04669].

The first three stages are entirely built from convolutional FFN blocks. The early part of stage 4 also remains convolutional before transitioning into multi-directional Mamba blocks. The model therefore uses convolution where high-resolution local structure is most important and defers sequence-style global modeling to the lowest-resolution stage, where it is computationally cheaper. Classification is performed by global average pooling followed by a linear classifier [2509.04669].

| Variant | Stage layout | Params / GMACs / Top-1 |
|---|---|---|
| VCMamba-S | FFN\(\times 4\) / FFN\(\times 4\) / FFN\(\times 12\) / FFN\(\times 4\) + MDM\(\times 4\) | 10.5M / 1.1 / 78.7% |
| VCMamba-M | FFN\(\times 4\) / FFN\(\times 4\) / FFN\(\times 12\) / FFN\(\times 2\) + MDM\(\times 4\) | 21.0M / 2.3 / 81.5% |
| VCMamba-B | FFN\(\times 4\) / FFN\(\times 4\) / FFN\(\times 12\) / FFN\(\times 2\) + MDM\(\times 4\) | 31.5M / 4.0 / 82.6% |

The channel schedules are also scale-dependent. VCMamba-S uses stage channels \(32 \rightarrow 64 \rightarrow 144 \rightarrow 288\), VCMamba-M uses \(48 \rightarrow 96 \rightarrow 224 \rightarrow 448\), and VCMamba-B uses \(64 \rightarrow 128 \rightarrow 320 \rightarrow 512\). The scaling strategy therefore increases width across all stages while slightly altering the convolutional/Mamba balance in stage 4 [2509.04669].

## 3. Convolutional blocks, multi-directional Mamba, and state-space formulation

The convolutional FFN block follows an inverted-residual-like structure. It uses a \(1 \times 1\) convolution to expand channels, a \(3 \times 3\) depthwise convolution for spatial mixing, and a \(1 \times 1\) convolution to project back. BatchNorm is used, and GeLU is the activation [2509.04669].

The Multi-Directional Mamba block is the distinctive global-context component. For an input feature map \(X \in \mathbb{R}^{B \times C \times H \times W}\), the block first applies BatchNorm, then a 2D-adapted Mamba module, then another BatchNorm and a convolutional MLP/FFN in residual-style composition. Before the selective scan, the feature map is projected with a \(1 \times 1\) convolution to an inner dimension, combined with positional embeddings, normalized, flattened into a sequence, and passed through a \(3 \times 3\) depthwise convolution followed by SiLU to inject local spatial context [2509.04669].

The core scan stage uses four spatially continuous traversal paths, described conceptually as row-wise and column-wise snake patterns. Each scan direction has learnable directional parameters \(\Theta_k\), so the state update is direction-aware rather than shared across all routes. The paper gives the standard continuous-time SSM:
\[
h'(t) = \mathbf{A}h(t) + \mathbf{B}x(t), \qquad y(t) = \mathbf{C}h(t) + \mathbf{D}x(t),
\]
and the discretized selective recurrence:
\[
h_k = \overline{\mathbf{A}}_k h_{k-1} + \overline{\mathbf{B}}_k x_k, \qquad y_k = \mathbf{C}_k h_k + \mathbf{D}_k x_k.
\]
For the direction-aware update used in VCMamba, the paper writes
\[
h_{k,i} = \overline{\mathbf{A}}_{k,i} h_{k,i-1} + \left(\overline{\mathbf{B}}_{k,i} + \overline{\boldsymbol{\Theta}}_{k,i}\right)x_i,
\]
where \(k\) indexes one of the four scan directions and \(i\) indexes sequence position along that direction [2509.04669].

After the four directional scans, their outputs are summed:
\[
Y = \sum_{k=1}^{4} Y^{(k)}.
\]
This fused result is then passed through LayerNorm, a \(1 \times 1\) convolution, and BatchNorm. The design intent is clear: multi-directional scanning reduces directional bias, exposes tokens to complementary traversal orders, and makes a 1D selective scan more compatible with 2D visual geometry [2509.04669].

## 4. Training protocol and benchmark results

On ImageNet-1K, VCMamba is trained from scratch for 300 epochs at \(224 \times 224\) resolution using PyTorch and timm, with AdamW, learning rate \(2 \times 10^{-3}\), cosine annealing, RandAugment, Mixup, CutMix, and random erasing. On ADE20K, ImageNet-pretrained VCMamba backbones are fine-tuned for 40K iterations with Semantic FPN, AdamW, initial learning rate \(2 \times 10^{-4}\), polynomial decay with power 0.9, and \(512 \times 512\) input resolution [2509.04669].

The principal ImageNet-1K results are 78.7% top-1 for VCMamba-S, 81.5% for VCMamba-M, and 82.6% for VCMamba-B. The paper emphasizes that VCMamba-B reaches 82.6% top-1, surpassing PlainMamba-L3 by 0.3% while using 37% fewer parameters, and outperforming Vision GNN-B by 0.3% with 64% fewer parameters [2509.04669].

The ADE20K results reported in the paper are 42.0 mIoU for VCMamba-S and 47.1 mIoU for VCMamba-B. The main segmentation claim is that VCMamba-B exceeds EfficientFormer-L7 by 2.0 mIoU while using 62% fewer parameters [2509.04669].

These numbers position VCMamba as a competitive CNN-SSM hybrid rather than as a maximal-scale pure Mamba model. Relative to VMamba, which applies VSS blocks with SS2D across the hierarchy and reports 82.5%, 83.6%, and 83.9% top-1 for T/S/B variants on ImageNet-1K, VCMamba trades some of that fully Mamba-style hierarchy for stronger convolutional inductive bias in the early stages [2401.10166]. This suggests that VCMamba targets a different balance point in the design space rather than a simple replacement for hierarchical visual SSM backbones.

## 5. Ablation evidence and internal design logic

The ablation study begins from a baseline that uses PlainMamba layers in the final stage and progressively refines the block design. The starting baseline has 33.0M parameters and 80.2% top-1 on ImageNet-1K. Replacing a multiplicative branch with a skip connection yields 31.0M parameters and 80.7%. Interleaving Mamba and FFN layers raises accuracy to 81.5%. Adding LayerNorm inside the Mamba layer increases performance to 82.2%. Replacing linear projection heads with convolutions gives 82.5%. Adding stage-wise BatchNorm produces the final 82.6% of VCMamba-B [2509.04669].

These ablations support several architectural interpretations. First, the skip connection is not merely simpler; it is both more accurate and more parameter-efficient than the initial multiplicative design. Second, the interleaving of Mamba and FFN blocks gives the largest single jump after the skip-connection change, which directly supports the paper’s central thesis that local convolutional refinement and global sequence modeling should alternate rather than be segregated too coarsely. Third, normalization is unusually important: LayerNorm inside the Mamba branch and stage-wise BatchNorm both improve results. Fourth, replacing linear projections with convolutions indicates that 2D-aware projection remains beneficial even inside a state-space stage [2509.04669].

The ablation coverage is, however, selective. The paper does not provide dedicated experiments on alternative scan patterns, different directional aggregation strategies, or moving Mamba to earlier stages. It also does not study the number of Mamba stages beyond the predefined S/M/B configurations. A plausible implication is that the published evidence is strongest for block design and normalization choices, and weaker for broader architectural alternatives.

## 6. Relation to adjacent methods and limitations

VCMamba is best understood as a CNN-SSM hybrid backbone rather than a new foundational scan algorithm. The four-way multi-directional scan is explicitly said to leverage the mechanism in PlainMamba, so the main novelty lies in how that scan is embedded inside a hierarchical convolutional pyramid [2509.04669]. This distinguishes VCMamba from approaches such as GlobalMamba, whose principal contribution is a frequency-based image serialization strategy placed in front of otherwise standard vision Mamba blocks rather than a reallocation of convolution and Mamba across stages [2410.10316].

It also differs from VMamba in emphasis. VMamba adapts Mamba to images through SS2D and VSS blocks across a hierarchical backbone, giving each spatial location access to context from four scan routes throughout the network [2401.10166]. VCMamba instead treats convolution as the preferred operator in early high-resolution stages and confines Mamba-based global modeling to the deepest stage [2509.04669]. This suggests a different answer to the central question identified in visual Mamba surveys: not only how to scan visual data, but where in the hierarchy Mamba should be applied at all [2404.18861].

Several limitations are explicit or implied in the paper. The directional scan mechanism is inherited rather than newly formalized. Practical efficiency claims rely mainly on parameter counts, GMACs, and the linear-complexity argument of Mamba; the paper does not report throughput, latency, or memory usage [2509.04669]. The experimental scope is limited to ImageNet-1K and ADE20K, without COCO detection, robustness, or runtime benchmarking. In addition, several implementation details remain underspecified, including exact sequence-order indexing and some low-level hyperparameters. These omissions do not alter the paper’s main contribution, but they constrain how fully its efficiency and generality can be assessed.

In the broader Vision Mamba landscape, VCMamba therefore represents a specific architectural synthesis: convolutional hierarchical local modeling in the high-resolution stages, followed by multi-directional selective state-space modeling in the deepest stage. Its significance lies less in redefining Mamba itself than in showing that a selective late-stage use of Mamba can yield a strong local-global trade-off for image classification and semantic segmentation [2509.04669].

Source: https://www.emergentmind.com/topics/vcmamba