---
title: 'PN-BiMamba: Bidirectional Audio Model'
url: https://www.emergentmind.com/topics/pn-bimamba
type: topic
---

# PN-BiMamba: Bidirectional Audio Model

Searching arXiv for PN-BiMamba and closely related papers to ground the article.
PN-BiMamba is a name used in 2025 audio research for bidirectional Mamba-based sequence models that replace or augment Transformer-style self-attention. In stereo sound event localization and detection (SELD), it denotes a two-stage architecture built from a pre-trained PSELDnet encoder and a BiMamba decoder with asymmetric convolutions [2507.09570, 2506.13455]. In speech deepfake detection, the same label denotes one of the three efficient bidirectional Mamba encoders proposed inside Fake-Mamba, alongside TransBiMamba and ConBiMamba [2508.09294]. Across these usages, the common design motif is the substitution of MHSA or Conformer-style sequence modeling with bidirectional structured state-space model (SSM) computation that preserves global context modeling while targeting near-linear sequence-length scaling.

## 1. Terminology and scope

The term PN-BiMamba is not used in a single, universal sense across the 2025 arXiv literature. In "Enhancing Stereo Sound Event Detection with BiMamba and Pretrained PSELDnet" [2507.09570], PN-BiMamba is explicitly short for a system that combines a pre-trained PSELDnet backbone ("PN") with a BiMamba decoder. In the closely related stereo SELD paper "Stereo sound event localization and detection based on PSELDnet pretraining and BiMamba sequence modeling" [2506.13455], the same architectural family is described as a stereo SELD system based on pre-trained PSELDnet and bidirectional Mamba sequence modeling. In "Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternative" [2508.09294], PN-BiMamba is instead one of three encoder variants inside a speech anti-spoofing framework.

A concise way to understand the term is as a family resemblance rather than a single immutable blueprint. The shared elements are bidirectional SSM streams, Mamba-based recurrence, and the replacement of quadratic-cost self-attention modules with near-linear alternatives. The differences lie in task framing, front-end representation, output heads, and the exact placement of convolutions, normalization, and pooling.

| Usage context | Meaning of PN-BiMamba | Primary task |
|---|---|---|
| [2507.09570] | Pretrained PSELDnet with BiMamba | Stereo SELD |
| [2506.13455] | PSELDnet pretraining with BiMamba sequence modeling | Stereo SELD |
| [2508.09294] | One Fake-Mamba encoder variant | Speech deepfake detection |

## 2. PN-BiMamba in stereo SELD

In the stereo SELD line of work, PN-BiMamba is a two-stage architecture. The encoder is a frozen, or optionally fine-tuned, PSELDnet backbone. The decoder is a BiMamba stack with asymmetric convolutions that separately model time and frequency [2507.09570]. The input is stereo audio, which is converted to pseudo-FOA and then represented by log-mel and intensity-vector features. One formulation specifies the pipeline as stereo \((L,R)\rightarrow [W,Y,X=0,Z=0]\), followed by extraction of 7-channel log-mel plus intensity-vector features, a CNN14 encoder, a BiMamba plus asymmetric-convolution decoder, and two output heads: ACCDOA and distance [2506.13455].

The dimensional conventions are explicit in the PN-BiMamba description. For 5 s clips at 20 ms hop, the feature sequence has \(T=250\) frames and \(F=64\) mel bins; with 4 FOA channels, the flattened input dimension is \(D_{\text{in}}=256\). The CNN14 encoder uses six VGG-style blocks and reduces temporal resolution by a factor of \(32\), yielding \(T_e=T/32\approx 8\) and an output channel count typically given as \(N_e=2048\) [2507.09570]. A temporal module then interpolates the encoder output back to frame rate and applies frame aggregation before the sequence enters the BiMamba decoder.

The decoder itself is described as two bidirectional Mamba2DAC blocks with state dimension \(64\), conv dimension \(4\), and expand \(2\) [2507.09570]. Each block splits processing into a frequency path and a time path, applies asymmetric convolution, performs Mamba state updates, and then fuses forward and backward recurrences. The related SELD description emphasizes alternating asymmetric convolutions over time and frequency with \(k_t=9\) and \(k_f=9\), interleaved with BiMamba layers, to decouple temporal and spectral processing in the 2D feature map [2506.13455]. This suggests an architectural emphasis on factorized spatiotemporal modeling rather than a monolithic sequence operator.

The output stage is task-specific. One head predicts multi-ACCDOA outputs with a linear layer followed by \(\tanh\); the other predicts distance with a linear layer followed by ReLU [2507.09570, 2506.13455]. The multi-head design preserves the conventional SELD decomposition into activity-localization and distance regression while replacing the sequence backbone.

## 3. Sequence-modeling formulation

The stereo SELD papers ground PN-BiMamba in the Mamba family of structured state-space models. One formulation starts from the continuous-time SSM
$$
\begin{aligned}
\mathbf{h}'(t) &= \mathbf{A}\,\mathbf{h}(t) + \mathbf{B}\,x(t),\\
y(t) &= \mathbf{C}^\top \mathbf{h}'(t) + \mathbf{D}\,x(t),
\end{aligned}
$$
followed by Zero-Order Hold discretization
$$
\bar{\mathbf{A}} = \exp(\Delta\,\mathbf{A}),\quad
\bar{\mathbf{B}} = (\Delta\,\mathbf{A})^{-1}\bigl(\exp(\Delta\,\mathbf{A})-\mathbf{I}\bigr)\,\mathbf{B},
$$
and the recurrence
$$
\mathbf{h}_k = \bar{\mathbf{A}}\,\mathbf{h}_{k-1} + \bar{\mathbf{B}}\,x_k,\quad
y_k = \mathbf{C}^\top\,\mathbf{h}_k
$$
[2506.13455].

The bidirectional construction applies the SSM twice, once forward and once backward:
$$
\mathbf{H}^{\mathrm{fwd}} = \mathrm{SSM}(\mathbf{E}),\quad
\mathbf{H}^{\mathrm{bwd}} = \mathrm{reverse}\bigl(\mathrm{SSM}(\mathrm{reverse}(\mathbf{E}))\bigr),
$$
after which the two streams are fused by concatenation and projection:
$$
\mathbf{H} = \mathbf{W}_p[\mathbf{H}^{\mathrm{fwd}};\mathbf{H}^{\mathrm{bwd}}] + \mathbf{b}_p
$$
[2506.13455]. The alternative PN-BiMamba summary states the bidirectional recurrence as
$$
h_k^f = \bar{A}\,h_{k-1}^f + \bar{B}\,x_k,\qquad
h_k^b = \bar{A}\,h_{k+1}^b + \bar{B}\,x_k,
$$
with the output
$$
y_k = \mathrm{Mix}([C^\top h_k^f;C^\top h_k^b]) + D\,x_k
$$
[2507.09570].

Asymmetric convolution is the second defining mathematical device. In the 2D formulation,
$$
\mathbf{X}_t = \mathrm{Conv}_{k_t\times 1}(\mathbf{X}),\quad
\mathbf{X}_f = \mathrm{Conv}_{1\times k_f}(\mathbf{X}),\quad
\mathbf{X}' = \mathbf{X}_t + \mathbf{X}_f
$$
[2506.13455]. In the 1D formulation over an intermediate feature map \(X\in\mathbb{R}^{T\times \rho}\), the temporal and spectral paths are written as separate convolutions over time and feature index [2507.09570]. In both cases, the formal objective is the same: to disentangle time-varying and frequency-varying patterns before or alongside bidirectional SSM propagation.

## 4. Pretraining, optimization, and performance in SELD

The encoder pretraining follows PSELDnet. One account specifies FSD50K clean clips plus room RIR generated with the mirror-source method, converted to FOA, with an approximately \(70\) h train, \(20\) h val, and \(20\) h test split. The pretraining model uses 4-channel FOA input, 64-bin log-mel plus intensity vectors, CNN14 plus 4 Conformer blocks, and a multi-ACCDOA objective trained with Adam, \(lr_{\text{init}}=10^{-4}\), \(weight\_decay=10^{-4}\), and ReduceLROnPlateau [2507.09570]. The related stereo SELD paper describes large-scale synthetic spatialized mixtures with ground-truth ACCDOA vectors and an MSE loss over time and classes [2506.13455].

For downstream stereo SELD, the development set is DCASE2025 Task 3, derived from STARSS23, resampled to \(24\) kHz, with STFT window \(40\) ms, hop \(20\) ms, 64-band Mel filterbank, and 250-frame input length corresponding to 5 s audio [2506.13455]. Data augmentation includes Audio Channel Swapping, which swaps \(L/R\) and rotates azimuth in \([-90^\circ,90^\circ]\), doubling the training set via front-back symmetry [2506.13455]. Training uses Adam with ReduceLROnPlateau, 120 epochs, and seed \(42\); reported optimizer settings vary by decoder family, with BiMambaAC given as \(lr=3\times 10^{-5}\), \(wd=5\times 10^{-6}\), batch \(32\) in one configuration [2506.13455].

The computational comparison centers on the claim that Conformer-style MHSA has \(O(T^2)\) memory and compute, whereas BiMamba has \(O(T)\) sequence scaling [2506.13455]. For fine-tuned PSELDnet-based models, the reported parameter and MAC counts are:

| Model | Params / MACs | Development-set result |
|---|---|---|
| Conformer | 210 M / 4.69 G | \(F_{20^\circ}=38.2\%\), DOAE \(=15.9^\circ\), RDE \(=33\%\) |
| BiMamba | 178 M / 7.57 G | \(36.2\%\), \(16.6^\circ\), \(33\%\) |
| ConBiMamba | 338 M / 7.98 G | \(36.2\%\), \(16.9^\circ\), \(53\%\) |
| BiMambaAC | 76 M / 4.63 G | **\(39.6\%\)**, **\(15.8^\circ\)**, **\(33\%\)** |

These numbers are reported in both stereo SELD papers, with BiMambaAC identified as the strongest configuration on the DCASE2025 Task 3 development set [2507.09570, 2506.13455]. One paper reports that the \(+1.4\) percentage-point gain in \(F_{20^\circ}\) over the Conformer baseline is significant by paired bootstrap at \(p<0.05\) across 4 CV folds [2507.09570]; the other reports paired \(t\)-tests on 5 runs with \(p<0.01\) for BiMambaAC versus Conformer on \(F_{20^\circ}\) [2506.13455]. On latency, one account gives approximately \(45\) ms per 250-frame batch for Conformer versus approximately \(28\) ms for BiMambaAC on an NVIDIA V100, corresponding to an approximately \(38\%\) speed-up [2506.13455].

## 5. PN-BiMamba inside Fake-Mamba for speech deepfake detection

In Fake-Mamba, PN-BiMamba is not a PSELDnet-based SELD model but one of three bidirectional Mamba encoders designed for speech anti-spoofing, the others being TransBiMamba and ConBiMamba [2508.09294]. The overall pipeline has four stages: frame-level features \(S_f\in\mathbb{R}^{T\times C}\) from XLSR, a PN-BiMamba backbone that replaces all MHSA blocks, utterance-level linear-attention pooling to \(S_u\in\mathbb{R}^D\), and a 2-layer MLP classification head producing real/fake logits [2508.09294].

The data flow is explicit. Raw audio waveform is framed into 25 ms frames and passed to XLSR; the fine-tuned XLSR front end outputs \(S_f\in\mathbb{R}^{T\times C}\); a learnable linear projection maps \(\mathbb{R}^C\rightarrow\mathbb{R}^D\) to produce \(S_f'\in\mathbb{R}^{T\times D}\); \(N\) PN-BiMamba blocks are stacked, with the paper reporting \(L=7\) or \(S=4\); the final hidden sequence \(H_N\in\mathbb{R}^{T\times D}\) is pooled by linear attention to an utterance embedding \(S_u\in\mathbb{R}^D\); and a 2-layer MLP outputs logits and real/fake scores [2508.09294].

A PN-BiMamba block takes \(h_{i-1}\in\mathbb{R}^{T\times D}\), applies Pre-LayerNorm,
$$
\tilde h_{i-1}=\mathrm{LayerNorm}(h_{i-1}),
$$
projects to \(x,z\in\mathbb{R}^{T\times E}\), performs local spectral mixing and SiLU,
$$
x'=\mathrm{SiLU}(\mathrm{Conv1D}(x)),
$$
applies a forward SSM and gating,
$$
y=\mathrm{SSM}(x')\odot \mathrm{SiLU}(z),
$$
projects back to model dimension, computes a backward path using \(\mathrm{Flip}(\cdot)\) and Mamba, fuses forward and backward streams by summation, adds a first residual, applies a second Pre-LayerNorm, and then applies an FFN with final residual [2508.09294]. The block therefore replaces a Conformer MHSA sublayer with two unidirectional Mamba SSM convolutions, one left-to-right and one right-to-left, together with gating under Pre-LayerNorm.

Training uses weighted cross-entropy to counter real/fake imbalance, with Adam, \(lr=10^{-6}\), \(wd=10^{-4}\), batch \(32\), early stopping with patience \(7\), and final weight averaging over the top 5 dev-EER epochs [2508.09294]. The key hyper-parameters reported for PN-BiMamba are 7 blocks for the large model or 4 for the small model, model dimension \(D=144\), and expansion \(E=16D\) [2508.09294].

The computational rationale mirrors the stereo SELD use case. Self-attention in Conformer is given as \(O(T^2\cdot D)\) per layer, whereas a PN-BiMamba layer is \(O(T\cdot E+T\cdot D)\), described as near-linear in \(T\) due to selective SSM convolution [2508.09294]. On a V100 GPU, the Real-Time Factor over 1–6 s utterances is reported as \([0.0334, 0.0158, 0.0107, 0.0083, 0.0076, 0.0059]\) for XLSR-Conformer and \([0.0279, 0.0140, 0.0090, 0.0068, 0.0064, 0.0056]\) for Fake-Mamba(L), which is summarized as an approximately \(10\)–\(20\%\) speed-up over MHSA at inference [2508.09294].

Performance is reported on three anti-spoofing benchmarks:

| Backbone | Cross-dataset EERs (%) |
|---|---|
| TransBiMamba | \(1.25\pm0.094\) on ASVspoof 21 LA, \(2.55\pm0.128\) on 21 DF, \(7.59\pm0.30\) on In-The-Wild |
| ConBiMamba | \(1.80\pm0.113\), \(2.50\pm0.127\), \(8.11\pm0.31\) |
| PN-BiMamba (L) | \(0.97\pm0.083\), \(1.74\pm0.107\), \(5.85\pm0.267\) |

The same report compares PN-BiMamba against XLSR-Conformer \((1.40, 2.89, 8.23)\) and XLSR-DuaBiMamba \((0.93, 1.88, 6.71)\), stating that PN-BiMamba reduces EER by up to \(28.9\%\) on LA, \(23.4\%\) on DF, and \(12.8\%\) on In-The-Wild [2508.09294].

## 6. Ablations, design rationale, and limitations

Ablation studies across both application areas emphasize that PN-BiMamba is not merely a label for bidirectionality; its reported performance depends on a specific combination of bidirectional SSM fusion, normalization, auxiliary convolutions, and task-specific pooling or heads.

In stereo SELD, removing bidirectional recurrence from the full BiMambaAC system reduces \(F_{20^\circ}\) from \(39.6\%\) to \(31.0\%\), removing asymmetric convolution yields \(33.2\%\), and doubling the state size from \(128\rightarrow 256\) changes performance to \(39.9\%\) with DOAE \(15.5^\circ\) and RDE \(35\%\) [2507.09570]. The related stereo SELD discussion states that BiMamba helps because linear-time SSM captures very long-range dependencies more efficiently than MHSA, bidirectionality removes causal delay and allows the decoder to see the entire 5 s context, and asymmetric convolutions disentangle temporal versus spectral patterns, benefiting overlapping sources [2506.13455].

In Fake-Mamba, the analogous ablations on a 7-block PN-BiMamba show full-model EERs of \((0.97,1.74,5.85)\); removing the three Pre-LayerNorm operations gives \((3.00,4.13,9.51)\); removing the FFN after SSM gives \((1.07,2.47,7.25)\); removing the bidirectional split gives \((1.56,3.02,7.91)\); and removing linear-attention pooling gives \((2.19,2.52,7.37)\) on ASVspoof 21 LA, 21 DF, and In-The-Wild, respectively [2508.09294]. The paper interprets the parallel SSM paths plus Pre-LN placement as better fusion of temporal and channel cues, increasing sensitivity to synthetic artifacts [2508.09294].

The published limitations are domain-specific. For stereo SELD, MACs remain approximately \(4.6\) G per 5 s clip, pseudo-FOA conversion from stereo limits spatial cues versus true FOA, memory usage of bidirectional SSM remains higher than purely causal SSM, SSM state matrices can become memory-intensive as hidden size increases, and the current stereo track lacks explicit elevation estimation [2507.09570, 2506.13455]. For speech deepfake detection, the main emphasis is instead on real-time inference and cross-dataset robustness, with fewer explicit limitations stated in the supplied summary [2508.09294].

Taken together, these reports suggest that PN-BiMamba is best understood as an audio-modeling pattern rather than a single canonical architecture: a bidirectional Mamba core, often paired

Source: https://www.emergentmind.com/topics/pn-bimamba