Bidirectional Mamba: Dual SSM Architectures

• Bidirectional Mamba is a family of selective state-space models that merge forward and reverse scans to reduce scan-order bias.
• It employs multiple architectural designs such as dual-column branches, local backward scans, and partial sequence reversal with varied fusion mechanisms.
• The approach has been successfully applied in domains like anomalous diffusion, EEG sleep staging, and speech recognition to improve contextual inference.

Bidirectional Mamba denotes a family of Mamba-based selective state-space architectures that augment the standard unidirectional scan with reverse-order processing, reverse-order approximations, or other mechanisms that let representations depend on both preceding and succeeding context. Across the recent literature, the term does not refer to a single canonical block. Instead, it covers several related design patterns: paired forward/backward Mamba branches, dual-column directional stacks, local backward scans embedded inside a forward scan, partial sequence reversal, prefix-only bidirectional conditioning, and task-order bidirectional interaction. These variants have been used in anomalous diffusion inference, EEG sleep staging, spoofing detection, masked image generation, forecasting, imputation, sequential recommendation, speech recognition, medical image translation, and dense prediction (Lavaud et al., 2024, Zhou et al., 2024, Xiao et al., 2024, Liang et al., 2024).

1. Conceptual basis

Standard Mamba is consistently framed in these works as a selective state-space model with input-dependent selection and hardware-aware sequence processing, but also as inherently directional: at a given position, the state summarizes tokens that have already been scanned and is blind to later tokens (Lavaud et al., 2024, Zhang et al., 19 Jun 2025). This directional asymmetry is natural in causal language modeling, but it is repeatedly treated as a limitation in non-causal settings such as utterance-level speech classification, sleep staging, image token reconstruction, and retrospective biosignal analysis, where the full sequence is available and future context is informative (Xiao et al., 2024, Zhou et al., 2024, Tegon et al., 10 Feb 2025).

The basic motivation is therefore stable across domains. A backward scan over a time-reversed or otherwise reordered sequence lets a representation incorporate information that a forward-only scan cannot access. In short, bidirectionality is introduced to reduce scan-order bias, improve whole-sequence contextualization, and make Mamba competitive in tasks where the target decision depends on evidence distributed across both sides of a position (Chen et al., 2024, Zhang et al., 2024).

At the same time, the literature does not present Bidirectional Mamba as a fundamentally new state-space theory. Several papers explicitly characterize their contribution as an architectural bidirectionalization of standard Mamba rather than a new recurrence law or a new selective SSM formulation (Xiao et al., 2024, Lavaud et al., 2024, Tegon et al., 10 Feb 2025).

2. Architectural families

The main architectural variants differ in how the reverse context is constructed, how directional branches share parameters, and how outputs are fused.

System	Bidirectional mechanism	Fusion
Bi-Mamba for anomalous diffusion	Original-order and time-reversed trajectories through two separate Mamba blocks	Concatenation, then feedforward (Lavaud et al., 2024)
BiT-MamSleep	Forward and backward Conv1D+SSM branches after TRCNN and AFR	Gated sum, linear projection, residual (Zhou et al., 2024)
XLSR-Mamba / DuaBiMamba	Dual columns for forward and backward speech features	Concatenation (Xiao et al., 2024)
MaskMamba Bi-Mamba-V2	Forward SSM plus flipped backward SSM after standard convolution	Concatenation, then linear projection (Chen et al., 2024)
LBMamba	Global forward scan plus local backward scan inside the forward selective scan	Fused hidden state inside one block (Zhang et al., 19 Jun 2025)
SIGMA PF-Mamba	Original sequence plus partially flipped sequence with preserved suffix	Input-sensitive gated sum (Liu et al., 2024)
MADEON speech prefixing	Bidirectional processing only on speech-prefix tokens via speech token reversal	Mamba-SP block fusion inside decoder-only ASR (Masuyama et al., 2024)
BIM for dense prediction	Forward and reversed task-order scanning plus task-first/position-first serialization	Concatenation of forward/backward task-interaction outputs (Cao et al., 28 Aug 2025)

A recurrent pattern is the straightforward two-pass design: process the original sequence, process a reversed sequence, realign the backward output if needed, and combine the two streams. This appears in anomalous diffusion inference, speech spoofing detection, spoken term detection, image generation, and several speech and medical-imaging systems (Lavaud et al., 2024, Xiao et al., 2024, Singh et al., 2024, Kheir et al., 20 May 2025, Yuan et al., 12 May 2025).

Other works modify this template to match domain constraints. LBMamba eliminates the second global sweep by embedding a local backward scan inside the forward selective scan and then alternating scan direction across layers to recover a global receptive field over depth (Zhang et al., 19 Jun 2025). SIGMA does not reverse the entire interaction sequence; it reverses only the prefix and preserves the last $r$ items, arguing that full reversal weakens short-term preference signals in sequential recommendation (Liu et al., 2024). MADEON makes only the speech prefix bidirectional while keeping text generation causal, thereby implementing a prefix-bidirectional, continuation-causal decoder-only ASR model (Masuyama et al., 2024). BIM for multi-task dense prediction treats “bidirectionality” as forward and reversed task-order scanning over task-specific representations, rather than ordinary left-to-right and right-to-left scanning over one token stream (Cao et al., 28 Aug 2025).

3. Formal mechanisms and fusion rules

Several papers restate the state-space background in the standard form

$h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$

or its discrete counterpart after zero-order hold discretization, before specifying how bidirectional branches are constructed (Zhou et al., 2024, Tegon et al., 10 Feb 2025, Singh et al., 2024). In this sense, Bidirectional Mamba typically inherits the standard Mamba selective SSM internals and changes the block-level composition rather than the underlying state-space equations.

The most explicit block-level formulation appears in "MaskMamba: A Hybrid Mamba-Transformer Model for Masked Image Generation" (Chen et al., 2024). Its redesigned Bi-Mamba-V2 first applies a standard convolution, then runs a forward SSM and a backward SSM on a flipped sequence, adds the two directional SSM outputs, and finally concatenates that bidirectional representation with a parallel convolved branch before a linear projection. This formulation ties bidirectionality directly to non-autoregressive masked prediction, where global context from both sides is required.

Fusion mechanisms vary markedly. Summation is used in S-Mamba for inter-variate correlation encoding, where forward and backward Mamba outputs are simply added before a residual connection (Wang et al., 2024). DiffImp’s Bidirectional Attention Mamba also uses additive fusion after weighting the forward and backward streams with a learned temporal gate (Gao et al., 2024). In contrast, anomalous diffusion Bi-Mamba, DuaBiMamba, MaskMamba Bi-Mamba-V2, and BiCrossMamba-ST use concatenation-based fusion at some stage, typically followed by a learned projection (Lavaud et al., 2024, Xiao et al., 2024, Chen et al., 2024, Kheir et al., 20 May 2025). MSSC-BiMamba states that the two scans are averaged (Zhang et al., 2024).

One paper exposes a methodological inconsistency. BiT-MamSleep first describes bidirectional fusion as

$$y_{\text{bi}}=\frac{1}{2}(y_{\text{fwd}}+y_{\text{bwd}}),$$

but its block-level algorithm specifies a richer implementation with per-direction Conv1D+SSM processing, multiplicative gating by $\mathrm{SiLU}(z)$, summation, a linear output layer, and a residual connection (Zhou et al., 2024). This discrepancy matters because it shows that “Bidirectional Mamba” may denote either a conceptual bidirectional averaging rule or a more elaborate gated residual block.

LBMamba is the clearest efficiency-driven deviation from full two-pass bidirectionality. It defines a local backward recurrence inside a chunk of length $M$,

$$h_{t}^b = \begin{cases} B^f x_t & \text{if } t \% M = 0 \ \bar{A}^f h_{t+1}^b + \bar{B}^f x_t & \text{otherwise}, \end{cases}$$

and fuses it with the global forward state as

$$h_t = h_t^f + (h_t^b - B^f x_t), \qquad y_t = C^f h_t + D^f x_t.$$

This yields full past context and only local future context within one layer, with global bidirectional effect emerging across two layers through sequence reversal between encoders (Zhang et al., 19 Jun 2025).

4. Domain-specific realizations

In time-series science and forecasting, Bidirectional Mamba is used to improve inference from short or multivariate histories. In anomalous diffusion characterization, Bi-Mamba processes short, noisy two-dimensional trajectories in both time directions to infer the effective diffusion coefficient $K$ and anomalous exponent $\alpha$, with a multitask architecture for segmentation, $K$ regression, and $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$0 regression (Lavaud et al., 2024). In long-term forecasting, Bi-Mamba+ combines a forward and backward Mamba+ encoder over patch tokens and adds a forget gate that mixes state-space features with local convolutional features (Liang et al., 2024). In S-Mamba, the bidirectional scan is not over time but over the variate dimension, because channels are treated as tokens and the goal is inter-variate correlation encoding rather than temporal causality (Wang et al., 2024). In probabilistic imputation, DiffImp uses a Bidirectional Attention Mamba block for temporal context and a separate Channel Mamba Block for inter-variable dependence, explicitly targeting bidirectional temporal dependency and cross-channel structure inside a diffusion denoiser (Gao et al., 2024).

In EEG and sleep analysis, the pattern is consistently non-causal and encoder-centric. BiT-MamSleep places BiMamba after triple-resolution CNN feature extraction and adaptive feature recalibration, using bidirectional Conv1D+SSM branches to capture both short- and long-term temporal dependencies in sleep staging (Zhou et al., 2024). FEMBA uses stacked Bidirectional Mamba blocks as the encoder of a self-supervised EEG foundation model trained on over 21,000 hours of unlabeled EEG, then reuses that Bi-Mamba encoder for downstream abnormality, artifact, and slowing-event tasks (Tegon et al., 10 Feb 2025). MSSC-BiMamba combines Efficient Channel Attention with a Bidirectional State Space Model for multimodal PSG-based sleep stage classification and sleep health prediction, using forward and backward scans whose outputs are averaged (Zhang et al., 2024).

Speech applications reveal several distinct interpretations. XLSR-Mamba places a dual-column bidirectional Mamba on top of XLS-R or wav2vec 2.0 features for spoofing detection, with one full Mamba column for forward features and one for backward features (Xiao et al., 2024). BEST-STD uses two structurally identical Mamba blocks to produce contextual frame-level speech embeddings for vector quantization and spoken term detection (Singh et al., 2024). MADEON makes only the speech prefix bidirectional via speech token reversal while keeping the text suffix autoregressive, thereby adapting bidirectional Mamba to decoder-only ASR (Masuyama et al., 2024). "Mamba in Speech: Towards an Alternative to Self-Attention" argues that BiMamba consistently outperforms vanilla Mamba in speech enhancement and ASR encoder settings, especially when it replaces MHSA inside Conformer rather than acting as a standalone stack (Zhang et al., 2024). Fake-Mamba, BiCrossMamba-ST, and related anti-spoofing systems use bidirectional Mamba as a self-attention alternative for synthetic speech detection, but differ in whether the sequence is a single utterance-level stream, a spectro-temporal dual branch, or a branch-conditioned fusion design (Xuan et al., 12 Aug 2025, Kheir et al., 20 May 2025).

Vision and medical imaging introduce yet more specialized variants. MaskMamba redesigns bidirectional Mamba for masked image modeling by replacing causal convolution with standard convolution and replacing multiplicative branch fusion with concatenation (Chen et al., 2024). UltraLBM-UNet uses a shared-weight bidirectional Mamba on flattened 2D feature sequences inside a global-local multi-branch segmentation block (Fan et al., 25 Dec 2025). ABS-Mamba serializes 2D medical features with a spiral scan and processes them through two independent Mamba modules in forward and reverse sequence order inside a Bidirectional Mamba Residual Network (Yuan et al., 12 May 2025). BiSegMamba constructs three orthogonal serialized views of a 3D volume, creates forward and reversed sequences for each, processes all six in one shared Mamba call, then learns forward/backward and orientation-specific fusion weights (Zada et al., 29 May 2026). BIM for multi-task dense prediction departs furthest from the canonical notion by scanning task-specific features in forward and reversed task order and combining task-first and position-first traversals inside a linear-complexity interaction decoder (Cao et al., 28 Aug 2025).

Sequential recommendation offers a notably task-specific adaptation. SIGMA’s PF-Mamba runs one Mamba block on the original sequence and another on a partially flipped sequence that preserves the recent suffix, then fuses the two directional outputs with an input-sensitive Dense Selective Gate (Liu et al., 2024). This suggests that “bidirectional” can be tuned not only by adding a reverse branch but also by choosing which subsequence should remain unreversed because it carries privileged short-term signal.

5. Empirical behavior and efficiency

Across the surveyed papers, bidirectionality is usually justified empirically rather than theoretically. In anomalous diffusion, Bi-Mamba is reported to perform better overall and in each considered AnDi-2 category than a bidirectional RNN baseline, with lower losses and more stable training dynamics. The reported challenge results for the latest model are 7th in $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$1 inference with MAE $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$2, 9th in $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$3 inference with MSLE $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$4, 3rd in diffusion-type measurement with “F1 error” $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$5, and 10th in change-point detection with RMSE $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$6 (Lavaud et al., 2024).

In sleep staging, BiT-MamSleep reports qualitative ablation evidence that full BiMamba outperforms no Mamba, forward-only Mamba, and backward-only Mamba, with the gains described as especially important for imbalanced classes and subtle sleep-stage transitions. The full system reports ACC 85.18, MF1 78.51, and $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$7 79.59 on Sleep-EDF-20, and similarly competitive results on Sleep-EDF-78, SHHS, and ISRUC-S3 (Zhou et al., 2024). MSSC-BiMamba reports its best ISRUC-S3 result with ACC 0.852, F1 0.824, and Kappa 0.803 using CNN + ECA + 1 BiMamba, and shows that deeper BiMamba stacks do not monotonically improve performance (Zhang et al., 2024).

Speech spoofing results show that the exact bidirectional form matters. XLSR+DuaBiMamba achieves LA EER 0.93, LA min t-DCF 0.208, DF EER 1.88, and average EER 1.41, outperforming the unidirectional XLSR+Mamba baseline on average and surpassing strong XLSR+Conformer baselines on ASVspoof 2021 and In-the-Wild (Xiao et al., 2024). Fake-Mamba reports EER 0.97% on ASVspoof 21 LA, 1.74% on ASVspoof 21 DF, and 5.85% on In-The-Wild, describing these as relative gains over XLSR-Conformer and XLSR-Mamba while maintaining real-time inference across utterance lengths (Xuan et al., 12 Aug 2025). BiCrossMamba-ST reports that replacing unidirectional CrossMamba-ST with bidirectional BiCrossMamba-ST improves ASVspoof LA19 from minDCF 0.0341 / EER 1.40 to 0.0281 / 1.08 and DF21 from 0.4409 / 16.07 to 0.3942 / 14.77 (Kheir et al., 20 May 2025).

Forecasting and imputation studies also isolate the effect of bidirectionality. Bi-Mamba+ reports that removing the backward direction worsens average performance by 3.25% in MSE and 2.44% in MAE (Liang et al., 2024). DiffImp shows on MuJoCo with 90% missing ratio that bidirectional temporal modeling plus temporal attention plus inter-channel dependency achieves $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$8 MSE, outperforming forward-only and backward-only variants (Gao et al., 2024). SIGMA’s ablation on Beauty shows that removing partial flipping reduces HR@10 from 0.0986 to 0.0953, while removing the Dense Selective Gate reduces it to 0.0954 (Liu et al., 2024).

Efficiency results reveal why multiple papers continue to pursue bidirectionality despite its extra scans. LBMamba quantifies that its register-resident local backward scan increases FLOPs by about 27% at the kernel level but only raises running time by about 1.9%–2.3%, while memory is unchanged; at the model level, this enables better throughput–accuracy trade-offs than full global bidirectional scanning (Zhang et al., 19 Jun 2025). FEMBA reports that its Huge model uses 58.74B FLOPs and 1886.17 MB, versus 202.17B FLOPs and 2758.42 MB for LaBraM-Huge, while a 7.8M-parameter Tiny model remains viable for resource-constrained EEG analysis (Tegon et al., 10 Feb 2025). MaskMamba attributes part of its high-resolution speedup to the bidirectional redesign itself, reporting a 17.77% inference speed improvement over prior Bi-Mamba and a 54.44% inference speed improvement over Transformer at $h'(t)=Ah(t)+Bx(t), \qquad y(t)=Ch(t),$9 resolution (Chen et al., 2024).

6. Limits, ambiguities, and open directions

A first limitation is conceptual non-uniformity. “Bidirectional Mamba” can mean two full directional passes over one sequence, two full columns over SSL speech features, a local backward recurrence inside a forward scan, a partially reversed recommendation history, a prefix-only bidirectional speech conditioner, or a task-order interaction mechanism in multi-task dense prediction (Xiao et al., 2024, Zhang et al., 19 Jun 2025, Liu et al., 2024, Masuyama et al., 2024, Cao et al., 28 Aug 2025). This suggests that the term functions more as an architectural family name than as a single well-defined operator.

A second limitation is incomplete theoretical specification. Several papers explicitly do not derive new bidirectional SSM mathematics and instead leave the standard Mamba internals implicit, contributing chiefly at the block-composition level (Xiao et al., 2024, Tegon et al., 10 Feb 2025, Yuan et al., 12 May 2025). This is often sufficient for empirical studies, but it means that “Bidirectional Mamba” is usually defined by engineering diagrams, branch structure, and fusion policy rather than by a common set of equations.

Reproducibility is also uneven. The anomalous diffusion Bi-Mamba paper does not provide hidden sizes, parameter counts, dropout rates, learning-rate tables, or a clean ablation against unidirectional Mamba (Lavaud et al., 2024). BiT-MamSleep specifies functional ingredients but omits the exact number of BiMamba blocks, hidden dimension, state dimension, sequence length used for temporal context, and kernel sizes inside BiMamba (Zhou et al., 2024). FEMBA clearly states that stacked Bidirectional Mamba blocks are the encoder core, but does not publish explicit block-level bidirectional fusion equations (Tegon et al., 10 Feb 2025).

Some papers expose internal ambiguities. BiT-MamSleep’s average-fusion equation and algorithmic gated-sum block are not perfectly consistent (Zhou et al., 2024). The anomalous diffusion paper reports an “F1 error of 0.91,” even though F1 is usually a score rather than an error (Lavaud et al., 2024). Such issues do not invalidate the broader direction, but they show that the terminology and reporting conventions of Bidirectional Mamba remain unsettled.

Finally, many bidirectional variants are explicitly non-causal and therefore better suited to offline or fixed-window settings than to streaming. BiT-MamSleep’s design assumes access to future EEG context (Zhou et al., 2024). MADEON’s speech prefixing makes the conditioning prefix bidirectional but leaves generation causal, thereby preserving autoregressive decoding only after the full speech prefix has been observed (Masuyama et al., 2024). This suggests a broader trade-off: bidirectionality improves full-context modeling, but it can conflict with low-latency or strictly causal deployment unless it is confined to prefixes, local windows, or alternating-layer approximations.

Taken together, the literature presents Bidirectional Mamba as a practical response to the mismatch between Mamba’s causal scan and non-causal sequence understanding tasks. The dominant research pattern is architectural rather than theoretical: preserve Mamba’s linear-time selective scan where possible, then recover reverse context through paired scans, restricted scans, or structured interaction schemes chosen for the data modality and inference regime.