Parallel Vision Mamba (PVM) Layer

• The PVM layer is an architectural unit that uses hardware-aware, bidirectional state-space modeling to efficiently process visual tokens.
• It employs simultaneous spatial and directional parallelism with selective scanning to achieve linear or near-linear computational complexity.
• Empirical results demonstrate that integrating the PVM layer reduces memory and FLOP requirements while preserving competitive accuracy on vision tasks.

The Parallel Vision Mamba (PVM) layer is an architectural unit within the Vision Mamba (ViM) family that implements hardware-aware, parallelized state space modeling for vision tasks. By structuring bidirectional and parallel recurrences as efficient, batched state-space scans, the PVM layer provides scalable, high-throughput alternatives to quadratic-cost attention, with provable linear or near-linear complexity and demonstrated empirical competitiveness on dense prediction and global classification tasks (Ibrahim et al., 11 Feb 2025). PVM variants underpin a broad class of recent vision backbones, integrating bidirectional selective scanning, data-dependent adaptation, and spatial parallelism.

1. Architectural Overview

The PVM layer forms the computational core of a ViM block, ingesting an input sequence of visual tokens and outputting a sequence of equal dimensionality. At each block, the input tensor $T_{l-1}\in\mathbb{R}^{B\times M\times D}$ is layer-normalized and projected into two feature maps: one for state updates ($x\in\mathbb R^{B\times M\times E}$), and one for gating ($z\in\mathbb R^{B\times M\times E}$). Two parallel state space model (SSM) "scanners" run in opposite directions—forward and backward—each constructing direction-aware state-space matrices $\bar A_o,\ \bar B_o$ and outputting a sequence $y_o$. The directional outputs are then gated and fused before projection back to the original embedding dimension, with residual addition of the layer input. This block is inserted within stacks interleaved with patch embedding, positional encoding, and task-specific heads (Ibrahim et al., 11 Feb 2025).

The architecture implements spatial and directional parallelism, processing all spatial (or temporal) positions simultaneously, and fusing bidirectional state propagation in a tightly batched GPU kernel (Yoon et al., 5 Aug 2025). Selective scanning—data-driven adaptation of integration step size—enables dynamic focus on high-frequency feature regions.

2. Mathematical Formulation

Each SSM scanner at position $m$ and direction $o\in\{\text{fwd},\text{bwd}\}$ realizes the update

$$x_{t+1}^o = A_o\,x_t^o + B_o\,u_t^o\,,\qquad y_t^o = C_o\,x_t^o$$

where

$$A_o = \operatorname{diag}(\Delta_o)\,\text{Parameter}^A_o\,,\quad B_o = \Delta_o\otimes B_o^{\text{lin}}\,,\quad C_o = C_o^{\text{lin}}$$

The step size

$$\Delta_o = \log(1 + e^{W_o^\Delta x_o' + b_o^\Delta})$$

affords data dependence, and the SSM kernel propagation can be computed via convolutional (FFT-based) techniques. Both forward and backward scans are realized concurrently:

$x\in\mathbb R^{B\times M\times E}$0

Gated fusion uses $x\in\mathbb R^{B\times M\times E}$1 as elementwise mask followed by summation, projection, and skip connection.

3. Parallelization Strategy

Key to PVM is the simultaneous execution of SSM updates over all spatial positions and in both directions, implemented as a fused batched kernel (Yoon et al., 5 Aug 2025). Two principal forms of parallelism are exploited:

• Spatial parallelism: the full $x\in\mathbb R^{B\times M\times E}$2-length sequence is processed in batch within the SSM; FFT-style convolution and chunk-wise scan methods reduce wall-clock runtime.
• Directional parallelism: both forward and backward recurrences are executed within a single kernel call, doubling throughput.
• Selective scanning: dynamic adjustment of $x\in\mathbb R^{B\times M\times E}$3 supports adaptive, per-location integration speed.

Optimized hardware variants further map these recurrences to systolic scan arrays, processing in $x\in\mathbb R^{B\times M\times E}$4 time and memory with only local inter-PE communication, yielding order-of-magnitude acceleration on edge devices (Yoon et al., 5 Aug 2025).

4. Complexity Analysis and Empirical Results

Theoretical complexity per PVM layer is as follows:

Operation	Complexity
2 × Conv1D + SiLU	$x\in\mathbb R^{B\times M\times E}$5
Parallel SSM rollout	$x\in\mathbb R^{B\times M\times E}$6 or $x\in\mathbb R^{B\times M\times E}$7
Hidden state storage	$x\in\mathbb R^{B\times M\times E}$8

Empirically, ViM (VMamba-Local, VMamba-Plain) reports up to 2× speedup over comparable Transformer blocks, with <1% ImageNet top-1 accuracy loss and ~30% FLOP reduction (COCO object detection) (Ibrahim et al., 11 Feb 2025). Hardware acceleration (Mamba-X) realizes 11.6× SSM throughput and 2.3× end-to-end latency reduction versus GPU baselines, with energy per SSM reduced by 11.5× (Yoon et al., 5 Aug 2025). Hybrid INT8 quantization with power-of-two scale approximation preserves model accuracy within 1% of FP16.

5. Block-Level Architecture

A typical ViM block containing a PVM layer proceeds as:

$z\in\mathbb R^{B\times M\times E}$1 (Ibrahim et al., 11 Feb 2025)

PVM blocks are alternated with patch-merging (in encoders) or upsampling (in decoders). In segmentation pipelines, PVMs are positioned between patch embedding and decoder heads.

6. Ablation Studies and Design Trade-Offs

Ablation studies highlight the importance of bidirectional recurrences: removing the backward scan leads to a 0.8% ImageNet drop. Fixing $x\in\mathbb R^{B\times M\times E}$9 (removing data dependence) impairs both accuracy and stability. Replacing PVM with standard $z\in\mathbb R^{B\times M\times E}$0 attention increases memory up to 3× and results in a 1.2-point AP drop (COCO) (Ibrahim et al., 11 Feb 2025).

Parameter reduction via parallelization—splitting channels into multiple narrow SSM streams and recombining—enables UltraLight and ASP-VMUNet backbones to achieve 60–75% lower parameter count and >50% GFLOP reduction versus conventional Mamba/Vision Transformer blocks, often with matching or improved performance (Wu et al., 2024, Bao et al., 25 Mar 2025).

7. Practical Impact and Applications

PVM enables efficient scaling of state-space vision models to higher spatial resolution and larger token counts. It underpins ViM backbones for image classification (ImageNet), dense prediction (COCO, ADE20K), and medical image segmentation (ISIC, PH2), and has been adapted for robust mask-aware inference and partial-input tasks (Mas et al., 4 Mar 2026). Variants such as multi-head scan, partial SSM, and locally-bidirectional recurrences further generalize the PVM paradigm to structured, multimodal, and irregular data domains.

Integrations in persistent visual memory pathways within LVLMs (PVM as “looking path”) provide length-agnostic visual perception with minimal parameter overhead and demonstrable accuracy improvements, resisting signal dilution in long autoregressive decoding (Huang et al., 1 May 2026).

Taken together, the PVM layer represents the high-throughput, hardware-conscious, and mathematically principled evolution of state-space sequence modeling for contemporary vision architectures (Ibrahim et al., 11 Feb 2025).