---
title: Bidirectional Mamba Blocks
url: https://www.emergentmind.com/topics/bidirectional-mamba-blocks
type: topic
---

# Bidirectional Mamba Blocks

A Bidirectional Mamba Block is a neural architectural primitive that extends the hardware-optimized, selective state-space modeling mechanisms of the Mamba model to process sequential or tokenized data in both forward (causal) and backward (anti-causal) directions. By running two independent state-space model (SSM) recurrences—one left-to-right and one right-to-left—and then fusing their outputs, Bidirectional Mamba enables efficient global context integration analogous to bidirectional LSTMs or bidirectional Transformers, but with strict linear complexity in sequence length. This design improves long-range dependency modeling, context-aware inference, and computational efficiency across domains including speech, vision, time series, multimodal, and biomedical signal processing.

## 1. Core Principles and Mathematical Foundations

A Bidirectional Mamba Block consists of two parallel, parameter-tied or independent selective SSM streams:
- **Forward SSM:** At time $t$, updates hidden state $h_t^\rightarrow = A_t^\rightarrow h_{t-1}^\rightarrow + B_t^\rightarrow u_t$, outputting $y_t^\rightarrow = C_t^\rightarrow h_t^\rightarrow$.
- **Backward SSM:** Processes the time-reversed sequence, $h_t^\leftarrow = A_t^\leftarrow h_{t+1}^\leftarrow + B_t^\leftarrow u_t$,  $y_t^\leftarrow = C_t^\leftarrow h_t^\leftarrow$.
- Input-dependent gating is typically realized by a small neural network: $u_t = \sigma(W_g x_t + b_g) \odot (W_u x_t + b_u)$; matrix parameters $A_t$, $B_t$, $C_t$ are usually light-weight MLP-parameterized or diagonal.

Fusion is performed either by concatenation and linear projection, simple summation, or learned gating, yielding a unified representation:
$$
z_t = W_f \begin{bmatrix} y_t^\rightarrow \\ y_t^\leftarrow \end{bmatrix} + b_f, \quad \text{or} \quad z_t = \alpha_t \odot y_t^\rightarrow + (1-\alpha_t) \odot y_t^\leftarrow
$$
Here, $W_f$ is a learnable $D\times 2D$ matrix, and $\alpha_t$ may be a dynamic gate.

A residual connection and normalization (e.g., LayerNorm or RMSNorm) are always applied:
$$
\mathrm{out}_t = \mathrm{LayerNorm}(x_t + z_t)
$$
Each Bidirectional Mamba Block is commonly stacked $L$ times to form a deep model.

## 2. Algorithmic Variants and Architectural Design

Multiple Bidirectional Mamba block instantiations have been reported:
- **Dual-Column (DuaBiMamba):** Contains two independent Mamba columns (forward/backward), fuses by concatenation $\rightarrow$ fully connected layer $\rightarrow$ residual $+$ layer norm, deployed atop pre-trained acoustic models for spoof detection [2411.10027].
- **Parallel Scan (BiMamba):** Runs two SSMs on original and reversed input in parallel, concatenates or sums features elementwise, typical in time series, EEG, and diffusion models [2412.07299, 2404.15772, 2410.13338].
- **Task-Axis, Cross-Feature, and Spiral Scans:** Adaptive scan orderings in vision/multitask/2D biomedical applications, e.g., spiral scan in medical imaging or cross-task scan in dense prediction for improved spatial/contextual coverage [2508.20376, 2505.07687].
- **Locally Bi-directional (LBMamba):** Fuses a forward global scan with lightweight in-register local backward scans within each GPU thread, followed by alternating global direction reversal across blocks for full receptive field with minimal memory overhead [2506.15976].

Bidirectional Mamba blocks often integrate additional innovations:
- **Attention and gating mechanisms:** Temporal attention weighting, cross-channel/channel-mixing, and forget gates augment the basic bidirectional state-space construction [2410.13338, 2404.15772].
- **Integration in composite modules:** Used with front-end CNNs, adaptive feature recalibration, spectral-temporal attention, and channel-wise transformations for high-dimensional or multivariate data [2411.01589, 2505.13930].

## 3. Computational Complexity and Efficiency

Bidirectional Mamba blocks maintain strict linear computational and activation memory complexity in sequence length $T$ (for time-indexed models) or $N$ (for spatially flattened images or patches), given fixed state or channel size $D$:
- **Forward/Backward scan cost:** $O(2\,T\,(D^2 + D))$ per block.
- **Fusion/Projection cost:** $O(T\,2D \times D)$ for fully connected merge.
- **Residual and normalization add negligible overhead.**
- The parallel scan implementation yields $O(TD + D^2 \log T)$ on modern hardware; dual-column/bidirectional designs double compute cost relative to single-path, but remain $\ll O(T^2)$ of self-attention [2411.10027, 2401.09417, 2506.15976].

Empirical benchmarking demonstrates substantial improvement in throughput, memory footprint, and scalability compared to self-attention backbones, especially for long or high-resolution sequences [2401.09417, 2506.15976, 2405.15881].

## 4. Applications Across Modalities

Bidirectional Mamba blocks have been integrated and validated in a range of domains:
- **Speech and audio:** Spoofing attack detection, speech enhancement, ASR, deepfake detection—DuaBiMamba, BiCrossMamba-ST, and generic BiMamba blocks enable context-rich classification and artifact localization with linear-time kernels [2411.10027, 2411.01589, 2505.13930, 2405.12609].
- **Vision:** Vision Mamba adopts bidirectional state-space processing as the backbone for classification, detection, segmentation with global receptive field efficiency [2401.09417]. Spiral/2D-bidirectional scans are key in medical image translation (ABS-Mamba) [2505.07687]. LBMamba/LBVim introduces hardware-efficient local bidirectional context for ultra-high-res imagery [2506.15976].
- **Time Series and Forecasting:** Efficient modeling of long-range dependencies, bidirectional context, and multi-channel interactions for forecasting, imputation, and anomalous diffusion inference [2412.07299, 2410.13338, 2404.15772, 2504.02013, 2408.11451].
- **Multitask and Multimodal:** BIM employs task/position bidirectional scans for dense prediction; bidirectional Mamba enables robust cross-task interaction with linear scaling in number of tasks [2508.20376].
- **Point Cloud and 3D Geometric Data:** Hybrid Transformer + BiMamba models enhance global feature extraction under tight compute/memory constraints [2406.06069].

## 5. Empirical Evidence and Design Trade-offs

Ablation and benchmarking data demonstrate:
- **Predictive Advantage:** Bidirectionality consistently improves accuracy, as measured by mIoU, FID, minDCF, error rates, etc., across speech, vision, diffusion, and time series tasks [2411.10027, 2401.09417, 2505.07687, 2404.15772, 2505.13930]. For example, bidirectional configuration in Vision Mamba yields a $+0.7\%$ top-1 ImageNet gain over unidirectional, and in XLSR-Mamba, dual-column bidirectional design dominates challenging “in-the-wild” benchmarks [2411.10027, 2401.09417].
- **Trade-offs:** The computational cost is roughly $2\times$ relative to a single unidirectional scan, but remains $O(N)$ or $O(T)$ and is significantly lower than Transformer-style $O(N^2)$. Local bidirectional variants (LBMamba) recover most benefits with only 2–3% throughput penalty and near-zero increase in off-chip memory traffic [2506.15976].
- **Parameterization Choices:** Gate sharing (InnBiMamba) is more efficient but less flexible than path-separated (ExtBiMamba); gating and fusion types modulate expressivity for various end tasks [2405.12609].
- **Fusion Methods:** Concatenation plus linear, summation, and learned dynamic gates all appear viable, with detailed design tuned empirically [2412.07299, 2401.09417, 2505.13930].

## 6. Architectural Patterns and Theoretical Implications

Bidirectional Mamba blocks inherit the advantages of SSM kernels for long-range dependency, but supplement RNN-like history with explicit anti-causal modeling:
- **Global context without quadratic attention:** Each token or spatial location “sees” both past and future, enabling non-causal inference essential for generative modeling, detection, and segmentation [2401.09417, 2405.15881].
- **Selective gating:** Small MLPs choose which parts of the sequence or spatial structure to integrate at each step, enabling noise suppression, context mixing, and adaptation to heterogeneous data [2412.07299, 2505.13930].
- **Fusion with locality and multi-scale context:** Designs such as spiral scan and position/task-wise fusions allow hierarchical and global context mixing at any desired spatial or temporal granularity [2508.20376, 2505.07687].

## 7. Limitations, Open Problems, and Prospective Directions

Despite strong empirical evidence for performance and efficiency, several design constraints persist:
- **Double compute cost:** Full bidirectional blocks incur $2\times$ SSM compute per layer; local bidirectional variants (e.g., LBMamba) offer mitigation but at possible cost to global receptive field unless augmented with alternating data order or hybrid architectures [2506.15976].
- **Expressiveness relative to full attention:** Despite superior speed and memory, bidirectional SSMs lack pairwise content-based selection inherent to self-attention, which may still be desirable for certain modeling tasks [2401.09417, 2405.12609].
- **Parameter sharing and data efficiency:** When applied to low-data or high-variance regimes (e.g., speech spoofing), careful integration of pre-training, regularization, and gate-sharing is required [2411.10027].
- **Application to complex or non-sequential modalities:** Extending bidirectional Mamba to multi-modal, graph-structured, or hierarchical data remains an ongoing area of research.

Bidirectional Mamba Blocks represent a significant advancement in scalable, context-rich sequence and tensor modeling, combining the operational simplicity and linear scaling of SSMs with the full-context capabilities of bidirectional inference, yielding state-of-the-art results and enabling new application domains with practical hardware-awareness and algorithmic flexibility [2411.10027, 2412.07299, 2401.09417, 2506.15976, 2508.20376, 2405.15881, 2411.01589].

Source: https://www.emergentmind.com/topics/bidirectional-mamba-blocks