---
title: 'BiMamba: Bidirectional State-Space Architecture'
url: https://www.emergentmind.com/topics/bidirectional-mamba-bimamba
type: topic
---

# BiMamba: Bidirectional State-Space Architecture

Searching arXiv for recent BiMamba papers to ground the article.
Bidirectional Mamba (BiMamba) denotes a family of bidirectional selective state-space architectures derived from Mamba, in which a forward scan and a backward scan are combined so that each position can incorporate both past and future context while preserving linear sequence scaling up to a constant-factor overhead. Across recent work, BiMamba is not a single fixed block but a recurrent design pattern instantiated with different fusion operators, scan orders, and deployment constraints in automatic speech recognition, time-series forecasting, vision, medical imaging, biosignal analysis, audio event localization, spoofing detection, motion generation, neural decoding, and multi-task dense prediction [2602.11546] [2405.12609] [2404.15772].

## 1. State-space foundation and bidirectional extension

BiMamba inherits its mathematical core from continuous-time state-space models. In the standard formulation used across the literature, the latent state satisfies
\[
\dot h(t)=A\,h(t)+B\,x(t),\qquad y(t)=C\,h(t)+D\,x(t),
\]
with zero-order-hold discretization yielding
\[
h_t=\bar A\,h_{t-1}+\bar B\,x_t,\qquad y_t=C\,h_t,
\]
or, in selective Mamba form,
\[
h_t=\bar A_t\,h_{t-1}+\bar B_t\,x_t,\qquad y_t=C\,h_t+D\,x_t,
\]
where
\[
\bar A_t=\exp(\Delta_tA),\qquad \bar B_t=(\Delta_tA)^{-1}(\exp(\Delta_tA)-I)\,\Delta_tB.
\]
In TC-BiMamba, \(\Delta_t, B_t, C_t\) are content-dependent gating parameters computed by a small feed-forward network \(f(x_t)\), and Mamba’s complexity is explicitly characterized as \(O(T)\) in sequence length [2602.11546].

The bidirectional extension duplicates this scan in opposite temporal directions. A forward branch aggregates \(x_{1:t}\), while a backward branch aggregates \(x_{t:T}\). The simplest formulations average the two directional outputs,
\[
y_t=\tfrac12\bigl(y_t^f+y_t^b\bigr),
\]
as in MSSC-BiMamba and BiT-MamSleep, whereas other variants use summation, concatenation followed by projection, or learned channel-wise weighting [2405.20142] [2411.01589]. TC-BiMamba, for example, defines
\[
\mathrm{Fused}(x_t)=\beta\odot F_t+(1-\beta)\odot B_t,
\]
with \(\beta\in\mathbb R^{1\times d}\) a learned scalar vector [2602.11546].

The central motivation is consistent across domains: unidirectional Mamba is inherently causal and therefore blind to “future” tokens, frames, or spatial positions. BiMamba restores symmetric context without reverting to quadratic self-attention. This makes it especially attractive in tasks that benefit from full-context inference but remain sensitive to long sequence length, such as full-utterance ASR, long-horizon forecasting, volumetric segmentation, and long-form audio analysis [2405.12609] [2604.04395].

## 2. Architectural patterns and fusion regimes

Although implementations differ, most BiMamba blocks follow a recognizable pattern: normalization and projection, a local mixing stage such as convolution or depthwise convolution, two directional state-space scans, directional fusion, and a residual refinement stage. In speech, BiMamba is often used as a plug-in replacement for the multi-head self-attention block in Transformer or Conformer encoders, with the feed-forward network and residual structure retained unchanged [2405.12609].

The speech literature makes the dependence on surrounding architecture explicit. “Mamba in Speech” evaluates both Inner BiMamba and External BiMamba and reports that independent BiMamba layers without appropriate nonlinearity and residual structure fail on ASR, with ablations yielding WER \(>35\%\); the paper therefore recommends replacing MHSA with ExtBiMamba inside Transformer or Conformer layers rather than deploying isolated BiMamba blocks [2405.12609]. BioMamba uses two parallel Mamba scans, fuses them by addition plus a residual skip,
\[
\mathbf Z'=\mathbf Z_1+\mathbf Z_2+\mathbf Z,
\]
and then applies a Sparse Feed-Forward layer and LayerNorm [2503.11741]. BiT-MamSleep inserts Conv1D, SiLU gating, directional SSM parameter generation, and a final linear fusion:
\[
Y_{\mathrm{bi}}=\mathrm{Linear\_combine}(y_{\mathrm{fwd}}+y_{\mathrm{bwd}})+X
\]
[2411.01589].

Other domains emphasize different fusion regimes. XLSR-Mamba uses a dual-column architecture with independent forward and backward Mamba stacks and time-wise concatenation
\[
h_t^{\mathrm{bidi}}=\mathrm{Concat}(h_t^{\rightarrow},h_t^{\leftarrow}),
\]
followed by a small projection before the classification head [2411.10027]. Stereo SELD systems likewise concatenate forward and backward hidden sequences and project back to the original embedding width, whereas TC-BiMamba uses dimension-wise addition with a learned vector \(\beta\) [2506.13455] [2602.11546].

A plausible implication is that “BiMamba” is best understood as an architectural family rather than a canonical operator. The invariant element is the paired forward/backward state-space scan; the variable elements are weight sharing, scan scheduling, and fusion algebra.

## 3. Complexity, memory behavior, and efficiency engineering

A recurring claim in the literature is that BiMamba preserves the linear scaling of state-space models while avoiding the quadratic memory and compute of attention. BiTDiff states this comparison directly: a BiMamba layer has time complexity \(O(2\cdot N\cdot d^2)\) and memory complexity \(O(2\cdot N\cdot d)\), whereas a full Transformer layer has time complexity \(O(N^2\cdot d + N\cdot d^2)\) and memory complexity \(O(N^2 + N\cdot d)\) [2604.04395].

However, the practical cost of bidirectionality depends strongly on scan organization. Standard global bidirectional scan doubles the computational load, and LBMamba was proposed precisely because a global backward sweep erodes much of Mamba’s efficiency advantage in vision. LBMamba embeds a lightweight locally backward scan inside the forward selective scan and executes it entirely in per-thread registers; the reported overhead is approximately \(27\%\) more FLOPs inside the scan kernel but only a \(2\%\) runtime penalty, with zero additional HBM traffic or inter-thread communication [2506.15976].

TC-BiMamba addresses an analogous issue in unified streaming and non-streaming ASR. Without Trans-Chunk, chunking the backward pass as in LC-BiMamba requires physically reshaping \([B,T,D]\to[B\times N,T'/C,D]\), leading to a \(2\times\) slowdown relative to offline training and \(\sim2\times\) memory. With Trans-Chunk, TC-BiMamba simultaneously achieves 1.3 times training speedup, reduces training memory by 50%, and improves model performance since it can capture bidirectional context [2602.11546].

BiMamba’s efficiency arguments also extend outside sequence modeling in the narrow sense. In the Mamba message-passing decoder for long LDPC codes, local Tanner-graph aggregation is combined with bidirectional Mamba blocks, and on the \((1056,880)\) LDPC code MMPD achieves a \(0.45\) dB gain over CrossMPT at a target BER of \(10^{-4}\) while reducing memory consumption by a factor of \(1.5\); the reported training memory is \(23.9\) GB for CrossMPT versus \(15.8\) GB for MMPD [2605.10681]. At the opposite extreme of model scale, UltraLBM-UNet uses shared weights for its forward and backward Mamba branches, so the bidirectional extension incurs no extra SSM weights; the full model has only \(0.034\)M parameters and \(0.060\) GFLOPs, while the distilled UltraLBM-UNet-T has \(0.011\)M parameters and \(0.019\) GFLOPs [2512.21584].

## 4. Scan topology, locality, and deployment-specific variants

A major line of BiMamba research concerns the geometry of the scan itself. In ASR, TC-BiMamba treats bidirectionality under dynamic chunk-size training, where each batch samples a chunk size \(C\in[C_{\min},C_{\max}]\) with \(C_{\min}=2\) and \(C_{\max}=T\), enabling a single model to decode offline or at multiple streaming latencies; on LibriSpeech, chunk \(C=16\) corresponds to \(25\) ms \(\times 16 = 400\) ms look-ahead [2602.11546]. This directly counters the assumption that bidirectionality necessarily implies fixed-latency, full-context inference.

In computer vision, LBMamba localizes bidirectionality to short windows. The sequence is broken into contiguous tiles of size \(M\), the local backward scan is executed entirely in registers, and global receptive field is recovered by alternating the global scan direction every two layers in LBVim [2506.15976]. ABS-Mamba takes a different route for 2D medical image translation: it converts feature maps into 1D sequences via a spiral scan, processes the spiral sequence and its reverse with two independent Mamba modules, then concatenates the results with the original features before reprojection [2505.07687].

Three-dimensional segmentation introduces another family of scan topologies. BiSegMamba defines three orthogonal re-orientations \(\mathbf u^{(d)}, \mathbf u^{(w)}, \mathbf u^{(h)}\), forms both forward and reversed sequences for each orientation, concatenates all six sequences along the batch dimension into one large Mamba call, and then applies Adaptive Directional Fusion to re-weight the three orientation streams channel-wise [2605.30972]. In multi-task dense prediction, Bidirectional Interaction Mamba uses BI-Scan and MS-Scan: BI-Scan combines task-first and position-first sequence modeling, and MS-Scan introduces multiple window scales while retaining \(O(T\cdot H\cdot W)\) time and memory [2508.20376].

This diversity shows that bidirectionality is not limited to temporal reversal. It can be imposed over chunks, tiles, spirals, orthogonal volumetric orientations, task axes, and position-first/task-first serializations, depending on the structure of the underlying data.

## 5. Empirical record across domains

In speech recognition, TC-BiMamba provides a representative benchmark for unified offline and streaming ASR. On LibriSpeech clean/other test sets, TC-BiMamba(S) with \(47\)M parameters achieves offline WER \(3.12\% / 8.19\%\) and chunk16 WER \(3.66\% / 9.05\%\), compared with U2++ at \(3.32\% / 8.67\%\) offline and \(3.80\% / 9.50\%\) for chunk16. TC-BiMamba(L), with \(70\)M parameters, further reduces offline WER to \(2.97\% / 7.72\%\) and chunk16 WER to \(3.51\% / 8.62\%\) [2602.11546]. More broadly, “Mamba in Speech” reports that ConExtBiMamba reaches \(5.4\) dev-other and \(5.4\) test-other WER on LibriSpeech-960, slightly improving over Conformer at \(5.5\) and \(5.6\), while training approximately \(10\)–\(15\%\) faster per epoch and decoding at comparable or lower real-time factor [2405.12609].

In vision, LBMamba demonstrates that local bidirectionality can preserve throughput. On ImageNet-1K, LBVim-Ti runs at \(1\,621\) img/s with Top-1 \(73.7\%\), LBVim-300 at \(906\) img/s with Top-1 \(77.7\%\), and LBVim-S at \(663\) img/s with Top-1 \(79.6\%\); on ADE20K, LBVim-528 reaches \(45.5\%\) mIoU at \(15\) fps, and on COCO, LBVim-300 obtains \(46.6\) AP\(^b\) and \(40.3\) AP\(^m\) at \(7.3\) fps [2506.15976]. In multi-task dense prediction, BiMamba improves over MTMamba on NYUD-V2 from \(55.82\) to \(57.40\) Semseg mIoU and from \(0.5066\) to \(0.4733\) Depth RMSE, and on PASCAL-Context from \(72.62\) to \(73.15\) Parsing mIoU while increasing Boundary odsF from \(78.80\) to \(79.05\) [2508.20376].

In audio and speech security, the replacement of Conformer or MHSA by BiMamba has produced consistent gains. On DCASE2025 Task 3, the pretrained stereo BiMambaAC decoder reaches \(39.6\%\) \(F_{20^\circ}\), \(15.8^\circ\) DOAE, and \(33\%\) RDE with \(76\)M parameters, compared with the pretrained Conformer decoder at \(38.2\%\), \(15.9^\circ\), and \(33\%\) RDE with \(210\)M parameters [2507.09570]. In speech deepfake detection, Fake-Mamba reports EERs of \(0.97\%\) on ASVspoof 21 LA, \(1.74\%\) on 21 DF, and \(5.85\%\) on In-The-Wild, while maintaining real-time inference across utterance lengths [2508.09294].

Time-series and biomedical applications show the same pattern. Bi-Mamba+ achieves the lowest average MSE and MAE on eight real-world multivariate forecasting datasets, improving over S-Mamba by \(3.76\%\) MSE and \(2.67\%\) MAE and over iTransformer by \(4.72\%\) MSE and \(2.60\%\) MAE [2404.15772]. MSSC-BiMamba reaches ACC \(0.852\), F1 \(0.824\), and Kappa \(0.803\) on ISRUC-S3 after adding ECA, and ACC \(0.830\), F1 \(0.801\), and Kappa \(0.773\) on ISRUC-S1 [2405.20142]. BiT-MamSleep reports ACC \(85.18\%\), MF1 \(78.51\%\), and \(\kappa=79.59\%\) on Sleep-EDF-20 for the full bidirectional model [2411.01589]. BioMamba sets new SOTA on \(5/6\) biosignal datasets, with average Accuracy \(81.44\%\) versus \(75.54\%\) and average F1 \(80.15\%\) versus \(73.26\%\), while using approximately \(0.7\)–\(1.1\)M parameters versus Medformer at approximately \(3.5\)–\(7.4\)M [2503.11741].

Medical and scientific modeling further extend the empirical range. ABS-Mamba ablations on BraTS2019 show SSIM \(0.926\) with \(1\) BRMN, \(0.935\) with \(2\) BRMN, and \(0.931\) with \(3\) BRMN, indicating that two bidirectional spiral-Mamba blocks were optimal in that setting [2505.07687]. UltraLBM-UNet reports IoU \(79.82\) and DSC \(88.78\) for its preferred LMBP + \(2\) GLMBP configuration, and its ablation attributes the best result to the branch structure “1 DwConv + 2 BiMamba + Identity” [2512.21584]. BiSegMamba reaches Vessel Dice \(91.63\%\) versus SegMamba-V2’s \(88.32\%\), Plaque Dice \(60.99\%\) versus \(32.42\%\), and Foreground Dice \(96.38\%\) versus \(92.81\%\) on the carotid CTA dataset, while on BraTS2023 it reduces FLOPs from \(1853.19\) G to \(410.28\) G [2605.30972]. In anomalous diffusion, Bi-Mamba achieves MAE \(0.27\) for \(\alpha\) inference, MSLE \(0.05\) for \(K\) inference, F1 \(0.91\) for diffusion-type classification, and RMSE \(2.7\) frames for change-point detection on AnDi-2 test datasets [2412.07299].

## 6. Misconceptions, limitations, and research directions

A common misconception is that BiMamba is synonymous with a full reverse pass over the entire sequence. The literature contradicts this: dynamic chunked backward context in TC-BiMamba, thread-local backward scans in LBMamba, spiral serialization in ABS-Mamba, tri-oriented volumetric scans in BiSegMamba, and task-first/position-first BI-Scan in dense prediction all implement bidirectionality under different structural constraints [2602.11546] [2506.15976] [2505.07687] [2605.30972] [2508.20376].

Another misconception is that bidirectionality necessarily dominates unidirectional or attention-based alternatives. Several papers report more nuanced trade-offs. UltraLBM-UNet finds that “Bi-Mamba but unshared weights” yields IoU \(78.19\), worse than “Single-direction Mamba” at \(78.65\) and worse than the shared-weight design at \(79.82\) [2512.21584]. ABS-Mamba shows that increasing BMRN depth from \(2\) to \(3\) reduces SSIM from \(0.935\) to \(0.931\) [2505.07687]. LBMamba notes that class-token variants underperform simple global-average pooling because local backward windows dilute the class token’s global summary role [2506.15976].

Task-dependent limitations also remain explicit. TC-BiMamba was motivated partly because existing BiMamba-based streaming methods were limited to fixed chunk size decoding [2602.11546]. The anomalous-diffusion Bi-Mamba is architected for up to two change-points, and out-of-distribution biological processes may require fine-tuning or domain adaptation [2412.07299]. BioMamba identifies dataset-specific frequency resolution tuning as a burden and points to low-precision optimization and multimodal biosignal extensions as future directions [2503.11741]. UltraLBM-UNet likewise identifies further low-precision optimization as necessary for real-time on-device deployment [2512.21584].

A plausible synthesis is that the main research frontier has shifted from the basic question of whether bidirectionality helps to the more technical question of how bidirectionality should be scheduled, localized, fused, and regularized for a given deployment regime. The recent literature suggests that BiMamba’s enduring value lies less in a single canonical block than in its ability to express full-context state-space reasoning under strict latency, memory, and domain-structure constraints.

Source: https://www.emergentmind.com/topics/bidirectional-mamba-bimamba