---
title: 'Bidirectional Mamba: Dual SSM Architectures'
url: https://www.emergentmind.com/topics/bidirectional-mamba
type: topic
---

# Bidirectional Mamba: Dual SSM Architectures

Bidirectional Mamba denotes a family of Mamba-based selective state-space architectures that augment the standard unidirectional scan with reverse-order processing, reverse-order approximations, or other mechanisms that let representations depend on both preceding and succeeding context. Across the recent literature, the term does not refer to a single canonical block. Instead, it covers several related design patterns: paired forward/backward Mamba branches, dual-column directional stacks, local backward scans embedded inside a forward scan, partial sequence reversal, prefix-only bidirectional conditioning, and task-order bidirectional interaction. These variants have been used in anomalous diffusion inference, EEG sleep staging, spoofing detection, masked image generation, forecasting, imputation, sequential recommendation, speech recognition, medical image translation, and dense prediction [2412.07299][2411.01589][2411.10027][2404.15772].

## 1. Conceptual basis

Standard Mamba is consistently framed in these works as a selective state-space model with input-dependent selection and hardware-aware sequence processing, but also as inherently directional: at a given position, the state summarizes tokens that have already been scanned and is blind to later tokens [2412.07299][2506.15976]. This directional asymmetry is natural in causal language modeling, but it is repeatedly treated as a limitation in non-causal settings such as utterance-level speech classification, sleep staging, image token reconstruction, and retrospective biosignal analysis, where the full sequence is available and future context is informative [2411.10027][2411.01589][2502.06438].

The basic motivation is therefore stable across domains. A backward scan over a time-reversed or otherwise reordered sequence lets a representation incorporate information that a forward-only scan cannot access. In short, bidirectionality is introduced to reduce scan-order bias, improve whole-sequence contextualization, and make Mamba competitive in tasks where the target decision depends on evidence distributed across both sides of a position [2409.19937][2405.20142].

At the same time, the literature does not present Bidirectional Mamba as a fundamentally new state-space theory. Several papers explicitly characterize their contribution as an architectural bidirectionalization of standard Mamba rather than a new recurrence law or a new selective SSM formulation [2411.10027][2412.07299][2502.06438].

## 2. Architectural families

The main architectural variants differ in how the reverse context is constructed, how directional branches share parameters, and how outputs are fused.

| System | Bidirectional mechanism | Fusion |
|---|---|---|
| Bi-Mamba for anomalous diffusion | Original-order and time-reversed trajectories through two separate Mamba blocks | Concatenation, then feedforward [2412.07299] |
| BiT-MamSleep | Forward and backward Conv1D+SSM branches after TRCNN and AFR | Gated sum, linear projection, residual [2411.01589] |
| XLSR-Mamba / DuaBiMamba | Dual columns for forward and backward speech features | Concatenation [2411.10027] |
| MaskMamba Bi-Mamba-V2 | Forward SSM plus flipped backward SSM after standard convolution | Concatenation, then linear projection [2409.19937] |
| LBMamba | Global forward scan plus local backward scan inside the forward selective scan | Fused hidden state inside one block [2506.15976] |
| SIGMA PF-Mamba | Original sequence plus partially flipped sequence with preserved suffix | Input-sensitive gated sum [2408.11451] |
| MADEON speech prefixing | Bidirectional processing only on speech-prefix tokens via speech token reversal | Mamba-SP block fusion inside decoder-only ASR [2411.06968] |
| BIM for dense prediction | Forward and reversed task-order scanning plus task-first/position-first serialization | Concatenation of forward/backward task-interaction outputs [2508.20376] |

A recurrent pattern is the straightforward two-pass design: process the original sequence, process a reversed sequence, realign the backward output if needed, and combine the two streams. This appears in anomalous diffusion inference, speech spoofing detection, spoken term detection, image generation, and several speech and medical-imaging systems [2412.07299][2411.10027][2411.14100][2505.13930][2505.07687].

Other works modify this template to match domain constraints. LBMamba eliminates the second global sweep by embedding a local backward scan inside the forward selective scan and then alternating scan direction across layers to recover a global receptive field over depth [2506.15976]. SIGMA does not reverse the entire interaction sequence; it reverses only the prefix and preserves the last \(r\) items, arguing that full reversal weakens short-term preference signals in sequential recommendation [2408.11451]. MADEON makes only the speech prefix bidirectional while keeping text generation causal, thereby implementing a prefix-bidirectional, continuation-causal decoder-only ASR model [2411.06968]. BIM for multi-task dense prediction treats “bidirectionality” as forward and reversed task-order scanning over task-specific representations, rather than ordinary left-to-right and right-to-left scanning over one token stream [2508.20376].

## 3. Formal mechanisms and fusion rules

Several papers restate the state-space background in the standard form
\[
h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),
\]
or its discrete counterpart after zero-order hold discretization, before specifying how bidirectional branches are constructed [2411.01589][2502.06438][2411.14100]. In this sense, Bidirectional Mamba typically inherits the standard Mamba selective SSM internals and changes the block-level composition rather than the underlying state-space equations.

The most explicit block-level formulation appears in "MaskMamba: A Hybrid Mamba-Transformer Model for Masked Image Generation" [2409.19937]. Its redesigned Bi-Mamba-V2 first applies a standard convolution, then runs a forward SSM and a backward SSM on a flipped sequence, adds the two directional SSM outputs, and finally concatenates that bidirectional representation with a parallel convolved branch before a linear projection. This formulation ties bidirectionality directly to non-autoregressive masked prediction, where global context from both sides is required.

Fusion mechanisms vary markedly. Summation is used in S-Mamba for inter-variate correlation encoding, where forward and backward Mamba outputs are simply added before a residual connection [2403.11144]. DiffImp’s Bidirectional Attention Mamba also uses additive fusion after weighting the forward and backward streams with a learned temporal gate [2410.13338]. In contrast, anomalous diffusion Bi-Mamba, DuaBiMamba, MaskMamba Bi-Mamba-V2, and BiCrossMamba-ST use concatenation-based fusion at some stage, typically followed by a learned projection [2412.07299][2411.10027][2409.19937][2505.13930]. MSSC-BiMamba states that the two scans are averaged [2405.20142].

One paper exposes a methodological inconsistency. BiT-MamSleep first describes bidirectional fusion as
\[
y_{\text{bi}}=\frac{1}{2}(y_{\text{fwd}}+y_{\text{bwd}}),
\]
but its block-level algorithm specifies a richer implementation with per-direction Conv1D+SSM processing, multiplicative gating by \(\mathrm{SiLU}(z)\), summation, a linear output layer, and a residual connection [2411.01589]. This discrepancy matters because it shows that “Bidirectional Mamba” may denote either a conceptual bidirectional averaging rule or a more elaborate gated residual block.

LBMamba is the clearest efficiency-driven deviation from full two-pass bidirectionality. It defines a local backward recurrence inside a chunk of length \(M\),
\[
h_{t}^b =
\begin{cases}
B^f x_t & \text{if } t \% M = 0 \\
\bar{A}^f h_{t+1}^b + \bar{B}^f x_t & \text{otherwise},
\end{cases}
\]
and fuses it with the global forward state as
\[
h_t = h_t^f + (h_t^b - B^f x_t), \qquad y_t = C^f h_t + D^f x_t.
\]
This yields full past context and only local future context within one layer, with global bidirectional effect emerging across two layers through sequence reversal between encoders [2506.15976].

## 4. Domain-specific realizations

In time-series science and forecasting, Bidirectional Mamba is used to improve inference from short or multivariate histories. In anomalous diffusion characterization, Bi-Mamba processes short, noisy two-dimensional trajectories in both time directions to infer the effective diffusion coefficient \(K\) and anomalous exponent \(\alpha\), with a multitask architecture for segmentation, \(K\) regression, and \(\alpha\) regression [2412.07299]. In long-term forecasting, Bi-Mamba+ combines a forward and backward Mamba+ encoder over patch tokens and adds a forget gate that mixes state-space features with local convolutional features [2404.15772]. In S-Mamba, the bidirectional scan is not over time but over the variate dimension, because channels are treated as tokens and the goal is inter-variate correlation encoding rather than temporal causality [2403.11144]. In probabilistic imputation, DiffImp uses a Bidirectional Attention Mamba block for temporal context and a separate Channel Mamba Block for inter-variable dependence, explicitly targeting bidirectional temporal dependency and cross-channel structure inside a diffusion denoiser [2410.13338].

In EEG and sleep analysis, the pattern is consistently non-causal and encoder-centric. BiT-MamSleep places BiMamba after triple-resolution CNN feature extraction and adaptive feature recalibration, using bidirectional Conv1D+SSM branches to capture both short- and long-term temporal dependencies in sleep staging [2411.01589]. FEMBA uses stacked Bidirectional Mamba blocks as the encoder of a self-supervised EEG foundation model trained on over 21,000 hours of unlabeled EEG, then reuses that Bi-Mamba encoder for downstream abnormality, artifact, and slowing-event tasks [2502.06438]. MSSC-BiMamba combines Efficient Channel Attention with a Bidirectional State Space Model for multimodal PSG-based sleep stage classification and sleep health prediction, using forward and backward scans whose outputs are averaged [2405.20142].

Speech applications reveal several distinct interpretations. XLSR-Mamba places a dual-column bidirectional Mamba on top of XLS-R or wav2vec 2.0 features for spoofing detection, with one full Mamba column for forward features and one for backward features [2411.10027]. BEST-STD uses two structurally identical Mamba blocks to produce contextual frame-level speech embeddings for vector quantization and spoken term detection [2411.14100]. MADEON makes only the speech prefix bidirectional via speech token reversal while keeping the text suffix autoregressive, thereby adapting bidirectional Mamba to decoder-only ASR [2411.06968]. "Mamba in Speech: Towards an Alternative to Self-Attention" argues that BiMamba consistently outperforms vanilla Mamba in speech enhancement and ASR encoder settings, especially when it replaces MHSA inside Conformer rather than acting as a standalone stack [2405.12609]. Fake-Mamba, BiCrossMamba-ST, and related anti-spoofing systems use bidirectional Mamba as a self-attention alternative for synthetic speech detection, but differ in whether the sequence is a single utterance-level stream, a spectro-temporal dual branch, or a branch-conditioned fusion design [2508.09294][2505.13930].

Vision and medical imaging introduce yet more specialized variants. MaskMamba redesigns bidirectional Mamba for masked image modeling by replacing causal convolution with standard convolution and replacing multiplicative branch fusion with concatenation [2409.19937]. UltraLBM-UNet uses a shared-weight bidirectional Mamba on flattened 2D feature sequences inside a global-local multi-branch segmentation block [2512.21584]. ABS-Mamba serializes 2D medical features with a spiral scan and processes them through two independent Mamba modules in forward and reverse sequence order inside a Bidirectional Mamba Residual Network [2505.07687]. BiSegMamba constructs three orthogonal serialized views of a 3D volume, creates forward and reversed sequences for each, processes all six in one shared Mamba call, then learns forward/backward and orientation-specific fusion weights [2605.30972]. BIM for multi-task dense prediction departs furthest from the canonical notion by scanning task-specific features in forward and reversed task order and combining task-first and position-first traversals inside a linear-complexity interaction decoder [2508.20376].

Sequential recommendation offers a notably task-specific adaptation. SIGMA’s PF-Mamba runs one Mamba block on the original sequence and another on a partially flipped sequence that preserves the recent suffix, then fuses the two directional outputs with an input-sensitive Dense Selective Gate [2408.11451]. This suggests that “bidirectional” can be tuned not only by adding a reverse branch but also by choosing which subsequence should remain unreversed because it carries privileged short-term signal.

## 5. Empirical behavior and efficiency

Across the surveyed papers, bidirectionality is usually justified empirically rather than theoretically. In anomalous diffusion, Bi-Mamba is reported to perform better overall and in each considered AnDi-2 category than a bidirectional RNN baseline, with lower losses and more stable training dynamics. The reported challenge results for the latest model are 7th in \(\alpha\) inference with MAE \(=0.27\), 9th in \(K\) inference with MSLE \(=0.05\), 3rd in diffusion-type measurement with “F1 error” \(=0.91\), and 10th in change-point detection with RMSE \(=2.7\) [2412.07299].

In sleep staging, BiT-MamSleep reports qualitative ablation evidence that full BiMamba outperforms no Mamba, forward-only Mamba, and backward-only Mamba, with the gains described as especially important for imbalanced classes and subtle sleep-stage transitions. The full system reports ACC 85.18, MF1 78.51, and \(\kappa\) 79.59 on Sleep-EDF-20, and similarly competitive results on Sleep-EDF-78, SHHS, and ISRUC-S3 [2411.01589]. MSSC-BiMamba reports its best ISRUC-S3 result with ACC 0.852, F1 0.824, and Kappa 0.803 using CNN + ECA + 1 BiMamba, and shows that deeper BiMamba stacks do not monotonically improve performance [2405.20142].

Speech spoofing results show that the exact bidirectional form matters. XLSR+DuaBiMamba achieves LA EER 0.93, LA min t-DCF 0.208, DF EER 1.88, and average EER 1.41, outperforming the unidirectional XLSR+Mamba baseline on average and surpassing strong XLSR+Conformer baselines on ASVspoof 2021 and In-the-Wild [2411.10027]. Fake-Mamba reports EER 0.97% on ASVspoof 21 LA, 1.74% on ASVspoof 21 DF, and 5.85% on In-The-Wild, describing these as relative gains over XLSR-Conformer and XLSR-Mamba while maintaining real-time inference across utterance lengths [2508.09294]. BiCrossMamba-ST reports that replacing unidirectional CrossMamba-ST with bidirectional BiCrossMamba-ST improves ASVspoof LA19 from minDCF 0.0341 / EER 1.40 to 0.0281 / 1.08 and DF21 from 0.4409 / 16.07 to 0.3942 / 14.77 [2505.13930].

Forecasting and imputation studies also isolate the effect of bidirectionality. Bi-Mamba+ reports that removing the backward direction worsens average performance by 3.25% in MSE and 2.44% in MAE [2404.15772]. DiffImp shows on MuJoCo with 90% missing ratio that bidirectional temporal modeling plus temporal attention plus inter-channel dependency achieves \(5.46\times 10^{-4}\) MSE, outperforming forward-only and backward-only variants [2410.13338]. SIGMA’s ablation on Beauty shows that removing partial flipping reduces HR@10 from 0.0986 to 0.0953, while removing the Dense Selective Gate reduces it to 0.0954 [2408.11451].

Efficiency results reveal why multiple papers continue to pursue bidirectionality despite its extra scans. LBMamba quantifies that its register-resident local backward scan increases FLOPs by about 27% at the kernel level but only raises running time by about 1.9%–2.3%, while memory is unchanged; at the model level, this enables better throughput–accuracy trade-offs than full global bidirectional scanning [2506.15976]. FEMBA reports that its Huge model uses 58.74B FLOPs and 1886.17 MB, versus 202.17B FLOPs and 2758.42 MB for LaBraM-Huge, while a 7.8M-parameter Tiny model remains viable for resource-constrained EEG analysis [2502.06438]. MaskMamba attributes part of its high-resolution speedup to the bidirectional redesign itself, reporting a 17.77% inference speed improvement over prior Bi-Mamba and a 54.44% inference speed improvement over Transformer at \(2048\times 2048\) resolution [2409.19937].

## 6. Limits, ambiguities, and open directions

A first limitation is conceptual non-uniformity. “Bidirectional Mamba” can mean two full directional passes over one sequence, two full columns over SSL speech features, a local backward recurrence inside a forward scan, a partially reversed recommendation history, a prefix-only bidirectional speech conditioner, or a task-order interaction mechanism in multi-task dense prediction [2411.10027][2506.15976][2408.11451][2411.06968][2508.20376]. This suggests that the term functions more as an architectural family name than as a single well-defined operator.

A second limitation is incomplete theoretical specification. Several papers explicitly do not derive new bidirectional SSM mathematics and instead leave the standard Mamba internals implicit, contributing chiefly at the block-composition level [2411.10027][2502.06438][2505.07687]. This is often sufficient for empirical studies, but it means that “Bidirectional Mamba” is usually defined by engineering diagrams, branch structure, and fusion policy rather than by a common set of equations.

Reproducibility is also uneven. The anomalous diffusion Bi-Mamba paper does not provide hidden sizes, parameter counts, dropout rates, learning-rate tables, or a clean ablation against unidirectional Mamba [2412.07299]. BiT-MamSleep specifies functional ingredients but omits the exact number of BiMamba blocks, hidden dimension, state dimension, sequence length used for temporal context, and kernel sizes inside BiMamba [2411.01589]. FEMBA clearly states that stacked Bidirectional Mamba blocks are the encoder core, but does not publish explicit block-level bidirectional fusion equations [2502.06438].

Some papers expose internal ambiguities. BiT-MamSleep’s average-fusion equation and algorithmic gated-sum block are not perfectly consistent [2411.01589]. The anomalous diffusion paper reports an “F1 error of 0.91,” even though F1 is usually a score rather than an error [2412.07299]. Such issues do not invalidate the broader direction, but they show that the terminology and reporting conventions of Bidirectional Mamba remain unsettled.

Finally, many bidirectional variants are explicitly non-causal and therefore better suited to offline or fixed-window settings than to streaming. BiT-MamSleep’s design assumes access to future EEG context [2411.01589]. MADEON’s speech prefixing makes the conditioning prefix bidirectional but leaves generation causal, thereby preserving autoregressive decoding only after the full speech prefix has been observed [2411.06968]. This suggests a broader trade-off: bidirectionality improves full-context modeling, but it can conflict with low-latency or strictly causal deployment unless it is confined to prefixes, local windows, or alternating-layer approximations.

Taken together, the literature presents Bidirectional Mamba as a practical response to the mismatch between Mamba’s causal scan and non-causal sequence understanding tasks. The dominant research pattern is architectural rather than theoretical: preserve Mamba’s linear-time selective scan where possible, then recover reverse context through paired scans, restricted scans, or structured interaction schemes chosen for the data modality and inference regime.

Source: https://www.emergentmind.com/topics/bidirectional-mamba