---
title: Bidirectional Mamba Block
url: https://www.emergentmind.com/topics/bidirectional-mamba-block
type: topic
---

# Bidirectional Mamba Block

A Bidirectional Mamba Block is a neural architectural construct that generalizes the selective State Space Model (SSM) known as Mamba by running parallel recurrence scans in both chronological (forward) and anti-chronological (backward) directions, then fusing their outputs to create richer, globally contextualized representations. This design preserves the linear time and memory complexity intrinsic to SSMs, while more fully leveraging bidirectional dependencies—thus closing the gap with quadratic-attention mechanisms but at a lower computational cost. It has been instantiated in various forms across speech, vision, time series, point cloud, and biomedical domains, with multiple empirical and implementational variants.

## 1. Architectural Foundations and Variants

### Dual-Column and Parallel Designs
The canonical Bidirectional Mamba (e.g., DuaBiMamba [2411.10027]) comprises two independent Mamba pipelines:
- **Forward column**: Processes sequence $X = [x_1, ..., x_T]$ in temporal order using a selective, input-driven SSM recurrence.
- **Backward column**: Processes the same sequence in reverse ($t=T\rightarrow1$).
Each column consists of input projection, selective SSM, local Conv1D/gated mixing, and output projection to a shared embedding dimension. The outputs $Y^f \in \mathbb{R}^{T \times D}$ and $Y^b \in \mathbb{R}^{T \times D}$ are then merged—commonly by concatenation along channels, with a linear projection to return to dimension $D$.

Bidirectionality generalizes to local-window (LBMamba [2506.15976]) and cross-task (BIM [2508.20376]) settings:
- **LBMamba** embeds small-windowed local backward scans within the main forward scan, eliminating the need for a full global reverse sweep and reducing overhead to ~2% kernel time.
- **Bidirectional Interaction Mamba (BIM)** uses parallel forward/backward cross-task Mamba scans (BI-Scan) to couple task-specific features for multi-task dense prediction, integrating both task-first and position-first serialization patterns.

## 2. Mathematical Formulation

At each position $t$:
- **Forward SSM**: $h_t^f = A_t^f h_{t-1}^f + B_t^f x_t$, $y_t^f = C_t^f h_t^f$
- **Backward SSM**: $h_t^b = A_t^b h_{t+1}^b + B_t^b x_t$, $y_t^b = C_t^b h_t^b$
where $A_t^\ast, B_t^\ast, C_t^\ast$ are parameterized via gating networks, potentially selective per input, and recurrence steps may be pointwise (1D convolution) or channelwise (spatial, temporal, or task axes).

Outputs are merged:
- **Concatenation**: $[y_t^f; y_t^b]$ (DuaBiMamba, VisionMamba, UltraLBM-UNet [2512.21584])
- **Summation**: $y_t^f + y_t^b$ (PointABM [2406.06069], SAMBA [2410.03707])
- **Gated fusion**: $g_t \odot y_t^f + (1-g_t) \odot y_t^b$ (MotionMamba BSM [2403.07487], BiT-MamSleep [2411.01589])

In bidirectional interaction or multi-task designs, scan axes and merge strategies generalize to both feature and task dimensions.

## 3. Computational Complexity and Scalability

Bidirectional Mamba blocks double the forward pass cost relative to a unidirectional Mamba block but remain:
- **Time/space complexity**: $O(T d)$, where $T$ is sequence length, $d$ embedding dimension, versus $O(T^2 d)$ for Transformer attention.
- **Local windowed variants**: LBMamba fuses a local backward scan in per-thread registers, incurring only ~27% more FLOPs but no additional inter-thread or HBM traffic—substantially more efficient than full global backward passes [2506.15976].
- **BIM/BI-Scan**: Cross-task bidirectional scan maintains linear cost $O(T \cdot H \cdot W \cdot C)$, compared to $O(T^2)$ in naïve full task-pair interaction [2508.20376].

Bidirectional SSMs naturally parallelize in-place and are hardware friendly, which underpins their success in long-sequence and high-resolution vision, speech, and bio applications.

## 4. Empirical Performance and Comparative Benchmarks

Empirical studies uniformly demonstrate gains over both vanilla (unidirectional) Mamba and Transformer baselines:
- **Spoofing attack detection (XLSR-Mamba)**: DuaBiMamba achieves the lowest EER (0.93% LA, 1.88% DF), exceeding XLSR+Conformer and other Mamba variants, with faster inference (lower real-time factor) [2411.10027].
- **Computer vision (LBVim with LBMamba)**: At constant throughput, gains of 0.8-1.6% in ImageNet-1K Top-1, ~2.7% ADE20K mIoU, and 0.9-1.1% in COCO detection APb/APm are observed over Vim-Ti and Vim-S while maintaining high efficiency [2506.15976].
- **Multi-task dense prediction (BIM)**: Gains +1.29 mIoU (NYUD-v2) over baseline Mamba, total +1.58 mIoU vs. MTMamba, with only a 26% parameter and 1% FLOP overhead [2508.20376].
- **Time-series forecasting and financial data (Bi-Mamba+, SAMBA)**: Outperforms state-of-the-art Transformers on real-world long-term forecasting and stock price prediction tasks, with near-linear scaling and improved accuracy [2404.15772], [2410.03707].
- **Biomedical and motion domains**: UltraLBM-UNet yields state-of-the-art segmentation at ultra-low resource budget (<0.034M params, <0.06 GFLOPs) [2512.21584]; Motion Mamba BSM block yields a 64% FID improvement and higher R-Precision for long-horizon motion modeling [2403.07487].

Ablations confirm that bidirectionality, even when locally windowed, accounts for the majority of these gains; fusion method and gate design account for additional, but smaller, improvements.

## 5. Implementation Patterns and Pseudocode

A generic forward pass for a Dual-Column Bidirectional Mamba block adheres to the following pattern:

```python
def DuaBiMamba(X):
    # Forward direction
    h_f = zeros(state_dim)
    Y_f = []
    for t in range(T):
        A_f, B_f, C_f = GatingNetworks_forward(X[t])
        h_f = A_f @ h_f + B_f @ X[t]
        y_f = C_f @ h_f
        Y_f.append(y_f)
    # Backward direction
    h_b = zeros(state_dim)
    Y_b = []
    for t in reversed(range(T)):
        A_b, B_b, C_b = GatingNetworks_backward(X[t])
        h_b = A_b @ h_b + B_b @ X[t]
        y_b = C_b @ h_b
        Y_b.insert(0, y_b)  # reverse order
    # Merge
    Z = concat_along_channel(Y_f, Y_b)  # shape: T x (2d)
    output = Linear_out(Z)              # shape: T x d
    return output
```
[2411.10027]

Windowed/local and cross-task variants (e.g., LBMamba, BIM) traverse either in per-thread or per-task windows and reverse sequence between blocks or task axes as context demands.

## 6. Domain-specific Extensions and Fusion Strategies

Bidirectional Mamba blocks serve as architectural primitives across modalities:
- **Speech and audio**: Serve as drop-in replacements for multi-head self-attention, delivering improved word/estimation error rates, speech intelligibility, and robustness [2405.12609].
- **EEG and biomedical**: Core bidirectional temporal filter, yielding robust sleep stage or lesion classification even under sequence imbalance and hardware constraint [2411.01589], [2512.21584].
- **Multi-task and multimodal contexts**: BI-Scan extends bidirectional recurrence to both spatial and task axes, allowing simultaneous cross-task and within-task attention at linear complexity [2508.20376].
- **Vision/image generation**: Bidirectional context across both token and channel axes underpins advances in segmentation (UltraLBM-UNet), masked image generation (MaskMamba), and efficient gigapixel whole-slide processing [2409.19937], [2506.15976], [2512.21584].
- **Partial bidirectionality**: Partially Flipped (PF-) Mamba in SIGMA selectively flips sequence prefixes to adapt bidirectionality flexibly, with gating for per-branch weighting [2408.11451].

Fusion strategies vary: channelwise concatenation (with linear projection), summation, or learned gate-based interpolation.

## 7. Limitations, Trade-offs, and Research Directions

While doubling the recurrence cost relative to unidirectional Mamba, Bidirectional Mamba blocks remain vastly more efficient than quadratic attention, with no principal loss in global receptive field—given sufficient layer depth, sequence reversal, or windowing schedule (LBVim strategy [2506.15976]). Empirical studies consistently report that alternating scan direction (full sequence reversals) is essential to fully restore global context.

A plausible implication is that trade-offs between window size, block count, and throughput can be tuned to suit hardware or memory constraints, especially in massive-scale vision or long-range bio applications. Continual improvements in local windowing, fusion gate design, and task-aware scan axes may further optimize expressive power for cross-modal dense prediction and domain-specific high-resolution modeling.

---

**References**:  
[2411.10027], [2506.15976], [2508.20376], [2409.19937], [2502.07161], [2412.07299], [2404.15772], [2410.03707], [2405.12609], [2411.01589], [2512.21584], [2403.07487], [2408.11451].

Source: https://www.emergentmind.com/topics/bidirectional-mamba-block