BiMamba: Bidirectional State-Space Architecture
- The paper introduces BiMamba as a bidirectional state-space architecture that uses forward and backward scans to integrate past and future context while preserving linear sequence scaling.
- BiMamba is applied across various domains such as ASR, vision, and time-series forecasting, employing different fusion operators and scan topologies to optimize performance under deployment constraints.
- Research shows that BiMamba achieves competitive accuracy and efficiency compared to attention-based models, with measurable gains in metrics like WER, Dice, and F1 scores.
Searching arXiv for recent BiMamba papers to ground the article. Bidirectional Mamba (BiMamba) denotes a family of bidirectional selective state-space architectures derived from Mamba, in which a forward scan and a backward scan are combined so that each position can incorporate both past and future context while preserving linear sequence scaling up to a constant-factor overhead. Across recent work, BiMamba is not a single fixed block but a recurrent design pattern instantiated with different fusion operators, scan orders, and deployment constraints in automatic speech recognition, time-series forecasting, vision, medical imaging, biosignal analysis, audio event localization, spoofing detection, motion generation, neural decoding, and multi-task dense prediction (She et al., 12 Feb 2026, Zhang et al., 2024, Liang et al., 2024).
1. State-space foundation and bidirectional extension
BiMamba inherits its mathematical core from continuous-time state-space models. In the standard formulation used across the literature, the latent state satisfies
with zero-order-hold discretization yielding
or, in selective Mamba form,
where
In TC-BiMamba, are content-dependent gating parameters computed by a small feed-forward network , and Mamba’s complexity is explicitly characterized as in sequence length (She et al., 12 Feb 2026).
The bidirectional extension duplicates this scan in opposite temporal directions. A forward branch aggregates , while a backward branch aggregates . The simplest formulations average the two directional outputs,
as in MSSC-BiMamba and BiT-MamSleep, whereas other variants use summation, concatenation followed by projection, or learned channel-wise weighting (Zhang et al., 2024, Zhou et al., 2024). TC-BiMamba, for example, defines
0
with 1 a learned scalar vector (She et al., 12 Feb 2026).
The central motivation is consistent across domains: unidirectional Mamba is inherently causal and therefore blind to “future” tokens, frames, or spatial positions. BiMamba restores symmetric context without reverting to quadratic self-attention. This makes it especially attractive in tasks that benefit from full-context inference but remain sensitive to long sequence length, such as full-utterance ASR, long-horizon forecasting, volumetric segmentation, and long-form audio analysis (Zhang et al., 2024, Jia et al., 6 Apr 2026).
2. Architectural patterns and fusion regimes
Although implementations differ, most BiMamba blocks follow a recognizable pattern: normalization and projection, a local mixing stage such as convolution or depthwise convolution, two directional state-space scans, directional fusion, and a residual refinement stage. In speech, BiMamba is often used as a plug-in replacement for the multi-head self-attention block in Transformer or Conformer encoders, with the feed-forward network and residual structure retained unchanged (Zhang et al., 2024).
The speech literature makes the dependence on surrounding architecture explicit. “Mamba in Speech” evaluates both Inner BiMamba and External BiMamba and reports that independent BiMamba layers without appropriate nonlinearity and residual structure fail on ASR, with ablations yielding WER 2; the paper therefore recommends replacing MHSA with ExtBiMamba inside Transformer or Conformer layers rather than deploying isolated BiMamba blocks (Zhang et al., 2024). BioMamba uses two parallel Mamba scans, fuses them by addition plus a residual skip,
3
and then applies a Sparse Feed-Forward layer and LayerNorm (Qian et al., 14 Mar 2025). BiT-MamSleep inserts Conv1D, SiLU gating, directional SSM parameter generation, and a final linear fusion: 4 (Zhou et al., 2024).
Other domains emphasize different fusion regimes. XLSR-Mamba uses a dual-column architecture with independent forward and backward Mamba stacks and time-wise concatenation
5
followed by a small projection before the classification head (Xiao et al., 2024). Stereo SELD systems likewise concatenate forward and backward hidden sequences and project back to the original embedding width, whereas TC-BiMamba uses dimension-wise addition with a learned vector 6 (Gao et al., 16 Jun 2025, She et al., 12 Feb 2026).
A plausible implication is that “BiMamba” is best understood as an architectural family rather than a canonical operator. The invariant element is the paired forward/backward state-space scan; the variable elements are weight sharing, scan scheduling, and fusion algebra.
3. Complexity, memory behavior, and efficiency engineering
A recurring claim in the literature is that BiMamba preserves the linear scaling of state-space models while avoiding the quadratic memory and compute of attention. BiTDiff states this comparison directly: a BiMamba layer has time complexity 7 and memory complexity 8, whereas a full Transformer layer has time complexity 9 and memory complexity 0 (Jia et al., 6 Apr 2026).
However, the practical cost of bidirectionality depends strongly on scan organization. Standard global bidirectional scan doubles the computational load, and LBMamba was proposed precisely because a global backward sweep erodes much of Mamba’s efficiency advantage in vision. LBMamba embeds a lightweight locally backward scan inside the forward selective scan and executes it entirely in per-thread registers; the reported overhead is approximately 1 more FLOPs inside the scan kernel but only a 2 runtime penalty, with zero additional HBM traffic or inter-thread communication (Zhang et al., 19 Jun 2025).
TC-BiMamba addresses an analogous issue in unified streaming and non-streaming ASR. Without Trans-Chunk, chunking the backward pass as in LC-BiMamba requires physically reshaping 3, leading to a 4 slowdown relative to offline training and 5 memory. With Trans-Chunk, TC-BiMamba simultaneously achieves 1.3 times training speedup, reduces training memory by 50%, and improves model performance since it can capture bidirectional context (She et al., 12 Feb 2026).
BiMamba’s efficiency arguments also extend outside sequence modeling in the narrow sense. In the Mamba message-passing decoder for long LDPC codes, local Tanner-graph aggregation is combined with bidirectional Mamba blocks, and on the 6 LDPC code MMPD achieves a 7 dB gain over CrossMPT at a target BER of 8 while reducing memory consumption by a factor of 9; the reported training memory is 0 GB for CrossMPT versus 1 GB for MMPD (Gusev et al., 11 May 2026). At the opposite extreme of model scale, UltraLBM-UNet uses shared weights for its forward and backward Mamba branches, so the bidirectional extension incurs no extra SSM weights; the full model has only 2M parameters and 3 GFLOPs, while the distilled UltraLBM-UNet-T has 4M parameters and 5 GFLOPs (Fan et al., 25 Dec 2025).
4. Scan topology, locality, and deployment-specific variants
A major line of BiMamba research concerns the geometry of the scan itself. In ASR, TC-BiMamba treats bidirectionality under dynamic chunk-size training, where each batch samples a chunk size 6 with 7 and 8, enabling a single model to decode offline or at multiple streaming latencies; on LibriSpeech, chunk 9 corresponds to 0 ms 1 ms look-ahead (She et al., 12 Feb 2026). This directly counters the assumption that bidirectionality necessarily implies fixed-latency, full-context inference.
In computer vision, LBMamba localizes bidirectionality to short windows. The sequence is broken into contiguous tiles of size 2, the local backward scan is executed entirely in registers, and global receptive field is recovered by alternating the global scan direction every two layers in LBVim (Zhang et al., 19 Jun 2025). ABS-Mamba takes a different route for 2D medical image translation: it converts feature maps into 1D sequences via a spiral scan, processes the spiral sequence and its reverse with two independent Mamba modules, then concatenates the results with the original features before reprojection (Yuan et al., 12 May 2025).
Three-dimensional segmentation introduces another family of scan topologies. BiSegMamba defines three orthogonal re-orientations 3, forms both forward and reversed sequences for each orientation, concatenates all six sequences along the batch dimension into one large Mamba call, and then applies Adaptive Directional Fusion to re-weight the three orientation streams channel-wise (Zada et al., 29 May 2026). In multi-task dense prediction, Bidirectional Interaction Mamba uses BI-Scan and MS-Scan: BI-Scan combines task-first and position-first sequence modeling, and MS-Scan introduces multiple window scales while retaining 4 time and memory (Cao et al., 28 Aug 2025).
This diversity shows that bidirectionality is not limited to temporal reversal. It can be imposed over chunks, tiles, spirals, orthogonal volumetric orientations, task axes, and position-first/task-first serializations, depending on the structure of the underlying data.
5. Empirical record across domains
In speech recognition, TC-BiMamba provides a representative benchmark for unified offline and streaming ASR. On LibriSpeech clean/other test sets, TC-BiMamba(S) with 5M parameters achieves offline WER 6 and chunk16 WER 7, compared with U2++ at 8 offline and 9 for chunk16. TC-BiMamba(L), with 0M parameters, further reduces offline WER to 1 and chunk16 WER to 2 (She et al., 12 Feb 2026). More broadly, “Mamba in Speech” reports that ConExtBiMamba reaches 3 dev-other and 4 test-other WER on LibriSpeech-960, slightly improving over Conformer at 5 and 6, while training approximately 7–8 faster per epoch and decoding at comparable or lower real-time factor (Zhang et al., 2024).
In vision, LBMamba demonstrates that local bidirectionality can preserve throughput. On ImageNet-1K, LBVim-Ti runs at 9 img/s with Top-1 0, LBVim-300 at 1 img/s with Top-1 2, and LBVim-S at 3 img/s with Top-1 4; on ADE20K, LBVim-528 reaches 5 mIoU at 6 fps, and on COCO, LBVim-300 obtains 7 AP8 and 9 AP0 at 1 fps (Zhang et al., 19 Jun 2025). In multi-task dense prediction, BiMamba improves over MTMamba on NYUD-V2 from 2 to 3 Semseg mIoU and from 4 to 5 Depth RMSE, and on PASCAL-Context from 6 to 7 Parsing mIoU while increasing Boundary odsF from 8 to 9 (Cao et al., 28 Aug 2025).
In audio and speech security, the replacement of Conformer or MHSA by BiMamba has produced consistent gains. On DCASE2025 Task 3, the pretrained stereo BiMambaAC decoder reaches 0 1, 2 DOAE, and 3 RDE with 4M parameters, compared with the pretrained Conformer decoder at 5, 6, and 7 RDE with 8M parameters (Gao et al., 13 Jul 2025). In speech deepfake detection, Fake-Mamba reports EERs of 9 on ASVspoof 21 LA, 0 on 21 DF, and 1 on In-The-Wild, while maintaining real-time inference across utterance lengths (Xuan et al., 12 Aug 2025).
Time-series and biomedical applications show the same pattern. Bi-Mamba+ achieves the lowest average MSE and MAE on eight real-world multivariate forecasting datasets, improving over S-Mamba by 2 MSE and 3 MAE and over iTransformer by 4 MSE and 5 MAE (Liang et al., 2024). MSSC-BiMamba reaches ACC 6, F1 7, and Kappa 8 on ISRUC-S3 after adding ECA, and ACC 9, F1 00, and Kappa 01 on ISRUC-S1 (Zhang et al., 2024). BiT-MamSleep reports ACC 02, MF1 03, and 04 on Sleep-EDF-20 for the full bidirectional model (Zhou et al., 2024). BioMamba sets new SOTA on 05 biosignal datasets, with average Accuracy 06 versus 07 and average F1 08 versus 09, while using approximately 10–11M parameters versus Medformer at approximately 12–13M (Qian et al., 14 Mar 2025).
Medical and scientific modeling further extend the empirical range. ABS-Mamba ablations on BraTS2019 show SSIM 14 with 15 BRMN, 16 with 17 BRMN, and 18 with 19 BRMN, indicating that two bidirectional spiral-Mamba blocks were optimal in that setting (Yuan et al., 12 May 2025). UltraLBM-UNet reports IoU 20 and DSC 21 for its preferred LMBP + 22 GLMBP configuration, and its ablation attributes the best result to the branch structure “1 DwConv + 2 BiMamba + Identity” (Fan et al., 25 Dec 2025). BiSegMamba reaches Vessel Dice 23 versus SegMamba-V2’s 24, Plaque Dice 25 versus 26, and Foreground Dice 27 versus 28 on the carotid CTA dataset, while on BraTS2023 it reduces FLOPs from 29 G to 30 G (Zada et al., 29 May 2026). In anomalous diffusion, Bi-Mamba achieves MAE 31 for 32 inference, MSLE 33 for 34 inference, F1 35 for diffusion-type classification, and RMSE 36 frames for change-point detection on AnDi-2 test datasets (Lavaud et al., 2024).
6. Misconceptions, limitations, and research directions
A common misconception is that BiMamba is synonymous with a full reverse pass over the entire sequence. The literature contradicts this: dynamic chunked backward context in TC-BiMamba, thread-local backward scans in LBMamba, spiral serialization in ABS-Mamba, tri-oriented volumetric scans in BiSegMamba, and task-first/position-first BI-Scan in dense prediction all implement bidirectionality under different structural constraints (She et al., 12 Feb 2026, Zhang et al., 19 Jun 2025, Yuan et al., 12 May 2025, Zada et al., 29 May 2026, Cao et al., 28 Aug 2025).
Another misconception is that bidirectionality necessarily dominates unidirectional or attention-based alternatives. Several papers report more nuanced trade-offs. UltraLBM-UNet finds that “Bi-Mamba but unshared weights” yields IoU 37, worse than “Single-direction Mamba” at 38 and worse than the shared-weight design at 39 (Fan et al., 25 Dec 2025). ABS-Mamba shows that increasing BMRN depth from 40 to 41 reduces SSIM from 42 to 43 (Yuan et al., 12 May 2025). LBMamba notes that class-token variants underperform simple global-average pooling because local backward windows dilute the class token’s global summary role (Zhang et al., 19 Jun 2025).
Task-dependent limitations also remain explicit. TC-BiMamba was motivated partly because existing BiMamba-based streaming methods were limited to fixed chunk size decoding (She et al., 12 Feb 2026). The anomalous-diffusion Bi-Mamba is architected for up to two change-points, and out-of-distribution biological processes may require fine-tuning or domain adaptation (Lavaud et al., 2024). BioMamba identifies dataset-specific frequency resolution tuning as a burden and points to low-precision optimization and multimodal biosignal extensions as future directions (Qian et al., 14 Mar 2025). UltraLBM-UNet likewise identifies further low-precision optimization as necessary for real-time on-device deployment (Fan et al., 25 Dec 2025).
A plausible synthesis is that the main research frontier has shifted from the basic question of whether bidirectionality helps to the more technical question of how bidirectionality should be scheduled, localized, fused, and regularized for a given deployment regime. The recent literature suggests that BiMamba’s enduring value lies less in a single canonical block than in its ability to express full-context state-space reasoning under strict latency, memory, and domain-structure constraints.