---
title: Bidirectional Mamba State-Space Modeling
url: https://www.emergentmind.com/topics/bidirectional-mamba-state-space-modeling
type: topic
---

# Bidirectional Mamba State-Space Modeling

Bidirectional Mamba State-Space Modeling

Bidirectional Mamba State-Space Modeling refers to a class of deep sequence models that extend the “Mamba” family of selective state-space models (SSMs) with parallel or alternating forward and backward state-propagation mechanisms. This bidirectional processing architecture allows hidden state updates at each sequence position to incorporate both past and future context, thereby overcoming the limitations of causality inherent in traditional unidirectional SSMs. Bidirectional Mamba architectures are instantiated across diverse modalities, including speech, vision, graph, time series, and multimodal molecular modeling, and are characterized by their hardware efficiency—scaling in compute and memory linearly with sequence length—while rivaling or surpassing Transformer performance on key tasks.

## 1. Core Mathematical Principles

The foundational principle of Bidirectional Mamba State-Space Modeling is the discrete-time SSM, which for an input sequence $\{x_t\}$ produces a hidden state $h_t$ and output $y_t$ by
\[
h_t = \overline{A}_t h_{t-1} + \overline{B}_t x_t , \quad y_t = C_t h_t
\]
with matrices (possibly vectors or even scalars per channel in compact implementations) $\overline{A}_t, \overline{B}_t, C_t$ that may depend on the input at time $t$. Bidirectional extensions introduce a simultaneous backward scan:
\[
\overleftarrow{h}_t = \overline{A}_t \overleftarrow{h}_{t+1} + \overline{B}_t x_t
\]
so that at each position both $h_t$ (forward) and $\overleftarrow{h}_t$ (reverse) are available. These are fused—typically by summing, concatenation and projection, or data-dependent gating—before further layer processing or downstream prediction [2411.06968, 2401.09417, 2412.07299].

Mamba’s “selective” property leverages input-dependent modulations through small feed-forward networks that produce SSM parameters conditioned on $x_t$, enabling flexible, context-sensitive dynamic modeling at each step [2411.06968].

## 2. Architectural Variants and Fusion Schemes

Bidirectional Mamba can be implemented using several fusion paradigms:

- **Serial and Parallel Blocks**: In speech modeling (e.g., MADEON) both “serial” (forward SSM, then speech-token reversal, then backward SSM) and “parallel” (forward and backward SSMs applied to the same (normalized, projected) input, then outputs fused) designs are used [2411.06968].
- **Concatenation and Gating**: Vision Mamba and Graph Mamba fuse forward and backward outputs by concatenation followed by linear or gated fusion. For example, in Vision Mamba, the fused hidden state is computed as $h_k = W[h^{\to}_k ; h^{\gets}_k] + b$, or, alternatively, using a data-dependent gate $g_k = \sigma(W_g [h^{\to}_k ; h^{\gets}_k] + b_g)$, $h_k = g_k \odot h^{\to}_k + (1-g_k) \odot h^{\gets}_k$ [2401.09417].
- **Local and Global Bidirectionality**: LBMamba (Locally Bi-directional Mamba) adds a backward recurrence only within local windows in each CUDA thread, combining the result with the forward hidden state, thus achieving local bidirectional context at minimal extra computational cost. Alternating scan direction every two layers recovers global receptive field while avoiding a full global backward pass [2506.15976].
- **Self-fusion with Residuals**: Audio Mamba fuses forward and backward hidden states by summation $z_i = h^{\text{fwd}}_i + h^{\text{bwd}}_i$, followed by residual, normalization, and feed-forward mixing [2406.03344]; similar patterns appear in graph and time series domains.

## 3. Applications Across Modalities

Bidirectional Mamba state-space modeling has demonstrated efficacy in a wide range of domains:

- **Speech Recognition and Separation**: MADEON deploys serial and parallel bidirectional speech-prefixing over tokenized inputs, significantly reducing word error rates and outperforming non-selective or unidirectional SSMs, while scaling sub-quadratically. Dual-path and SepMamba variants leverage short-/long-term and U-Net architectures, maintaining linear complexity with competitive SI-SNRi and SDRi on WSJ0-2mix and LibriSpeech benchmarks [2411.06968, 2403.18257, 2410.20997].
- **Vision and Medical Imaging**: Vision Mamba (ViM), Surface Vision Mamba, and ABS-Mamba exploit spatial patch ordering and bidirectional SSM fusions for context aggregation, achieving superior accuracy, throughput, and memory efficiency compared to Transformer backbones in tasks spanning ImageNet, COCO, ADE20K, and spherical/cortical manifold segmentation [2401.09417, 2501.14679, 2505.07687].
- **Time Series Forecasting**: Bi-Mamba+ incorporates forward and reversed SSM passes and combines them with gating and adaptive tokenization strategies, yielding state-of-the-art performance over 8 real-world multivariate datasets [2404.15772].
- **Graph Learning**: Graph Mamba Networks (GMNs) employ bidirectional SSM “token” sequences constructed from local subgraph and random-walk encodings to systematically propagate context along both graph-theoretic “directions”, empirically mitigating the over-squashing phenomenon [2402.08678].
- **Sequential Recommendation and Multimodal Fusion**: EchoMamba4Rec applies spectral filtering and bidirectional Mamba for sequential recommendation, showing systematic gains in HR@10 and NDCG@10. CrossLLM-Mamba fuses embeddings from biological LLMs for RNA interaction prediction via bidirectional SSM “alignment”, outperforming static fusion schemes and setting new benchmarks on RPI and binding affinity [2406.02638, 2602.22236].
- **Biomedical Signals and Video**: SR-Mamba and UltraLBM-UNet deploy bidirectional selective SSMs within surgical phase recognition and lightweight U-Net architectures respectively for robust context modeling and efficiency [2407.08333, 2512.21584].

## 4. Computational Complexity and Efficiency

A defining advantage of bidirectional Mamba models is their preservation of the base SSM’s $O(L\,N)$ compute and $O(L\,N)$ memory scaling, where $L$ is sequence length and $N$ is SSM state size, even under bidirectional extensions. Whereas bidirectional Transformers double both compute and memory due to quadratic attention, and naively running a global backward SSM sweep in Mamba would incur the same, Mamba’s selective SSMs use either in-thread local backward passes (LBMamba) or parallel forward/backward scans that do not significantly expand parameter count or runtime [2401.09417, 2506.15976, 2410.20997].

Benchmark results confirm these claims: MADEON-2SP achieves training speed (6 h) and memory (20 GB) nearly half that of Transformer baselines (8 h, 40 GB), with comparable parameter counts and test-accuracy on LibriSpeech and GigaSpeech [2411.06968]. Vision Mamba achieves $2.8\times$ higher throughput than DeiT while saving 86.8% GPU memory at high resolutions [2401.09417]. UltraLBM-UNet preserves zero parameter overhead for bidirectionality by weight sharing between directions [2512.21584].

## 5. Empirical Performance and Ablation

Bidirectional Mamba models consistently deliver superior or comparable accuracy to Transformers and nonlinear attention-free baselines:

| Architecture                 | Main Task                       | Gain from Bidirectionality          | Source           |
|------------------------------|----------------------------------|-------------------------------------|------------------|
| MADEON-2SP                   | ASR (LibriSpeech)                | –0.5% WER dev-clean vs uniSSMs      | [2411.06968]     |
| Vision Mamba (ViM)           | ImageNet, COCO, ADE20k           | $+0.4$–$+6.1$ pts vs DeiT/backbones | [2401.09417]     |
| Dual-path Mamba              | Speech Separation (WSJ0-2mix)    | $>3$ dB SI-SNRi over forward-only   | [2403.18257]     |
| Graph Mamba Networks (GMN)   | Node Classification (Long-Range) | $>4$ pts accuracy on heterophilic   | [2402.08678]     |
| Bi-Mamba+                    | Time-Series Forecasting          | –3.25% MSE vs forward-only          | [2404.15772]     |
| EchoMamba4Rec                | Sequential Recommendation        | $+4.5$% HR@10 vs uniMamba           | [2406.02638]     |
| CrossLLM-Mamba               | RNA–protein interaction          | +4.4 pts MCC vs static fusion       | [2602.22236]     |
| SR-Mamba                     | Surgical Phase Recognition       | +4.4 pts accuracy vs uniMamba       | [2407.08333]     |
| UltraLBM-UNet                | Skin Lesion Segmentation         | +1.17 pt IoU, no extra params       | [2512.21584]     |
| LBMamba/LBVim                | ImageNet, Pathology MIL          | +0.8–1.6% accuracy under constraint | [2506.15976]     |

Ablation studies consistently demonstrate large degradations (up to 4.4 pp accuracy, 3.25% MSE, or multiple dB SI-SNRi) when replacing bidirectional blocks with single-directional SSMs [2411.06968, 2404.15772, 2403.18257, 2407.08333].

## 6. Algorithmic and Implementation Details

Bidirectional Mamba implementations share several algorithmic and engineering optimizations:

- **Hardware-aware parallel scan**: Both forward and backward recurrences are executed in a parallel “scan” pattern, exploiting GPU-friendly segmented prefix algorithms.
- **Parameter sharing**: Some implementations (e.g., UltraLBM-UNet) share weights between forward and backward passes, minimizing parameter count while maximizing performance [2512.21584].
- **Context fusion**: Fusion may be performed via summation, concatenation with projection, or data-adaptive gates (e.g., SiLU or sigmoid), with variations affecting empirical performance and stability.
- **Efficient memory and compute**: Localized bidirectionality (LBMamba) leverages in-register recurring backward sweeps within each thread to avoid double off-chip memory traffic [2506.15976].
- **Spectral and gating augmentations**: EchoMamba4Rec augments bidirectional SSMs with frequency-domain (FFT) learned filters and GLU units, further enhancing signal quality and convergence [2406.02638].
- **Domain-specific tokenization**: MADEON applies STR (speech-token reversal) to the prefix, while Graph Mamba Networks construct ordered subgraph “token” sequences for bidirectional SSM passes [2411.06968, 2402.08678].

## 7. Limitations, Extensions, and Outlook

While Bidirectional Mamba architectures are empirically robust and efficient, some limitations and open directions remain:

- **Double-loop inefficiency**: Naive global forward–backward sweeps can negate linearity; local or alternating schemes correct this, but may introduce partial context gaps unless carefully alternated [2506.15976].
- **Information flow topology**: Graph-structured domains may lack a canonical sequence, making token ordering and permutation effects in bidirectional SSMs nontrivial [2402.08678].
- **Task-specific fusion**: Optimal fusion method (sum, concat, gate) may be domain or task-dependent and interacts with normalization, positional encoding, and depth.
- **Uncertainty quantification**: As current bidirectional Mamba models are discriminative and often determinate, additional machinery is needed for calibrated Bayesian or ensemble uncertainty estimates [2412.07299].

The bidirectional selective SSM paradigm is rapidly being adopted across domains requiring non-causal, context-rich long-sequence modeling, frequently at sub-quadratic complexity and without the architectural rigidity or inefficiency of self-attention layers. Ongoing developments focus on further optimizing bidirectional propagation at the hardware level, augmenting with spectral and graph-based regularization, and extending applications to emerging scientific and clinical data types.

Source: https://www.emergentmind.com/topics/bidirectional-mamba-state-space-modeling