PN-BiMamba: Bidirectional Audio Model

• PN-BiMamba is a family of bidirectional sequence models that replace Transformer-style self-attention with structured state-space modeling and asymmetric convolutions.
• It enables efficient stereo sound event localization by integrating a pretrained PSELDnet encoder with a BiMamba decoder for near-linear sequence-length scaling.
• In speech deepfake detection, PN-BiMamba functions as one encoder variant within Fake-Mamba, offering improved inference speed and cross-dataset robustness.

Searching arXiv for PN-BiMamba and closely related papers to ground the article. PN-BiMamba is a name used in 2025 audio research for bidirectional Mamba-based sequence models that replace or augment Transformer-style self-attention. In stereo sound event localization and detection (SELD), it denotes a two-stage architecture built from a pre-trained PSELDnet encoder and a BiMamba decoder with asymmetric convolutions (Gao et al., 13 Jul 2025, Gao et al., 16 Jun 2025). In speech deepfake detection, the same label denotes one of the three efficient bidirectional Mamba encoders proposed inside Fake-Mamba, alongside TransBiMamba and ConBiMamba (Xuan et al., 12 Aug 2025). Across these usages, the common design motif is the substitution of MHSA or Conformer-style sequence modeling with bidirectional structured state-space model (SSM) computation that preserves global context modeling while targeting near-linear sequence-length scaling.

1. Terminology and scope

The term PN-BiMamba is not used in a single, universal sense across the 2025 arXiv literature. In "Enhancing Stereo Sound Event Detection with BiMamba and Pretrained PSELDnet" (Gao et al., 13 Jul 2025), PN-BiMamba is explicitly short for a system that combines a pre-trained PSELDnet backbone ("PN") with a BiMamba decoder. In the closely related stereo SELD paper "Stereo sound event localization and detection based on PSELDnet pretraining and BiMamba sequence modeling" (Gao et al., 16 Jun 2025), the same architectural family is described as a stereo SELD system based on pre-trained PSELDnet and bidirectional Mamba sequence modeling. In "Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternative" (Xuan et al., 12 Aug 2025), PN-BiMamba is instead one of three encoder variants inside a speech anti-spoofing framework.

A concise way to understand the term is as a family resemblance rather than a single immutable blueprint. The shared elements are bidirectional SSM streams, Mamba-based recurrence, and the replacement of quadratic-cost self-attention modules with near-linear alternatives. The differences lie in task framing, front-end representation, output heads, and the exact placement of convolutions, normalization, and pooling.

Usage context	Meaning of PN-BiMamba	Primary task
(Gao et al., 13 Jul 2025)	Pretrained PSELDnet with BiMamba	Stereo SELD
(Gao et al., 16 Jun 2025)	PSELDnet pretraining with BiMamba sequence modeling	Stereo SELD
(Xuan et al., 12 Aug 2025)	One Fake-Mamba encoder variant	Speech deepfake detection

2. PN-BiMamba in stereo SELD

In the stereo SELD line of work, PN-BiMamba is a two-stage architecture. The encoder is a frozen, or optionally fine-tuned, PSELDnet backbone. The decoder is a BiMamba stack with asymmetric convolutions that separately model time and frequency (Gao et al., 13 Jul 2025). The input is stereo audio, which is converted to pseudo-FOA and then represented by log-mel and intensity-vector features. One formulation specifies the pipeline as stereo $(L,R)\rightarrow [W,Y,X=0,Z=0]$, followed by extraction of 7-channel log-mel plus intensity-vector features, a CNN14 encoder, a BiMamba plus asymmetric-convolution decoder, and two output heads: ACCDOA and distance (Gao et al., 16 Jun 2025).

The dimensional conventions are explicit in the PN-BiMamba description. For 5 s clips at 20 ms hop, the feature sequence has $T=250$ frames and $F=64$ mel bins; with 4 FOA channels, the flattened input dimension is $D_{\text{in}}=256$. The CNN14 encoder uses six VGG-style blocks and reduces temporal resolution by a factor of $32$, yielding $T_e=T/32\approx 8$ and an output channel count typically given as $N_e=2048$ (Gao et al., 13 Jul 2025). A temporal module then interpolates the encoder output back to frame rate and applies frame aggregation before the sequence enters the BiMamba decoder.

The decoder itself is described as two bidirectional Mamba2DAC blocks with state dimension $64$, conv dimension $4$, and expand $2$ (Gao et al., 13 Jul 2025). Each block splits processing into a frequency path and a time path, applies asymmetric convolution, performs Mamba state updates, and then fuses forward and backward recurrences. The related SELD description emphasizes alternating asymmetric convolutions over time and frequency with $T=250$0 and $T=250$1, interleaved with BiMamba layers, to decouple temporal and spectral processing in the 2D feature map (Gao et al., 16 Jun 2025). This suggests an architectural emphasis on factorized spatiotemporal modeling rather than a monolithic sequence operator.

The output stage is task-specific. One head predicts multi-ACCDOA outputs with a linear layer followed by $T=250$2; the other predicts distance with a linear layer followed by ReLU (Gao et al., 13 Jul 2025, Gao et al., 16 Jun 2025). The multi-head design preserves the conventional SELD decomposition into activity-localization and distance regression while replacing the sequence backbone.

3. Sequence-modeling formulation

The stereo SELD papers ground PN-BiMamba in the Mamba family of structured state-space models. One formulation starts from the continuous-time SSM

$T=250$3

followed by Zero-Order Hold discretization

$T=250$4

and the recurrence

$T=250$5

(Gao et al., 16 Jun 2025).

The bidirectional construction applies the SSM twice, once forward and once backward:

$T=250$6

after which the two streams are fused by concatenation and projection:

$T=250$7

(Gao et al., 16 Jun 2025). The alternative PN-BiMamba summary states the bidirectional recurrence as

$T=250$8

with the output

$T=250$9

(Gao et al., 13 Jul 2025).

Asymmetric convolution is the second defining mathematical device. In the 2D formulation,

$F=64$0

(Gao et al., 16 Jun 2025). In the 1D formulation over an intermediate feature map $F=64$1, the temporal and spectral paths are written as separate convolutions over time and feature index (Gao et al., 13 Jul 2025). In both cases, the formal objective is the same: to disentangle time-varying and frequency-varying patterns before or alongside bidirectional SSM propagation.

4. Pretraining, optimization, and performance in SELD

The encoder pretraining follows PSELDnet. One account specifies FSD50K clean clips plus room RIR generated with the mirror-source method, converted to FOA, with an approximately $F=64$2 h train, $F=64$3 h val, and $F=64$4 h test split. The pretraining model uses 4-channel FOA input, 64-bin log-mel plus intensity vectors, CNN14 plus 4 Conformer blocks, and a multi-ACCDOA objective trained with Adam, $F=64$5, $F=64$6, and ReduceLROnPlateau (Gao et al., 13 Jul 2025). The related stereo SELD paper describes large-scale synthetic spatialized mixtures with ground-truth ACCDOA vectors and an MSE loss over time and classes (Gao et al., 16 Jun 2025).

For downstream stereo SELD, the development set is DCASE2025 Task 3, derived from STARSS23, resampled to $F=64$7 kHz, with STFT window $F=64$8 ms, hop $F=64$9 ms, 64-band Mel filterbank, and 250-frame input length corresponding to 5 s audio (Gao et al., 16 Jun 2025). Data augmentation includes Audio Channel Swapping, which swaps $D_{\text{in}}=256$0 and rotates azimuth in $D_{\text{in}}=256$1, doubling the training set via front-back symmetry (Gao et al., 16 Jun 2025). Training uses Adam with ReduceLROnPlateau, 120 epochs, and seed $D_{\text{in}}=256$2; reported optimizer settings vary by decoder family, with BiMambaAC given as $D_{\text{in}}=256$3, $D_{\text{in}}=256$4, batch $D_{\text{in}}=256$5 in one configuration (Gao et al., 16 Jun 2025).

The computational comparison centers on the claim that Conformer-style MHSA has $D_{\text{in}}=256$6 memory and compute, whereas BiMamba has $D_{\text{in}}=256$7 sequence scaling (Gao et al., 16 Jun 2025). For fine-tuned PSELDnet-based models, the reported parameter and MAC counts are:

Model	Params / MACs	Development-set result
Conformer	210 M / 4.69 G	$D_{\text{in}}=256$8, DOAE $D_{\text{in}}=256$9, RDE $32$0
BiMamba	178 M / 7.57 G	$32$1, $32$2, $32$3
ConBiMamba	338 M / 7.98 G	$32$4, $32$5, $32$6
BiMambaAC	76 M / 4.63 G	$32$7, $32$8, $32$9

These numbers are reported in both stereo SELD papers, with BiMambaAC identified as the strongest configuration on the DCASE2025 Task 3 development set (Gao et al., 13 Jul 2025, Gao et al., 16 Jun 2025). One paper reports that the $T_e=T/32\approx 8$0 percentage-point gain in $T_e=T/32\approx 8$1 over the Conformer baseline is significant by paired bootstrap at $T_e=T/32\approx 8$2 across 4 CV folds (Gao et al., 13 Jul 2025); the other reports paired $T_e=T/32\approx 8$3-tests on 5 runs with $T_e=T/32\approx 8$4 for BiMambaAC versus Conformer on $T_e=T/32\approx 8$5 (Gao et al., 16 Jun 2025). On latency, one account gives approximately $T_e=T/32\approx 8$6 ms per 250-frame batch for Conformer versus approximately $T_e=T/32\approx 8$7 ms for BiMambaAC on an NVIDIA V100, corresponding to an approximately $T_e=T/32\approx 8$8 speed-up (Gao et al., 16 Jun 2025).

5. PN-BiMamba inside Fake-Mamba for speech deepfake detection

In Fake-Mamba, PN-BiMamba is not a PSELDnet-based SELD model but one of three bidirectional Mamba encoders designed for speech anti-spoofing, the others being TransBiMamba and ConBiMamba (Xuan et al., 12 Aug 2025). The overall pipeline has four stages: frame-level features $T_e=T/32\approx 8$9 from XLSR, a PN-BiMamba backbone that replaces all MHSA blocks, utterance-level linear-attention pooling to $N_e=2048$0, and a 2-layer MLP classification head producing real/fake logits (Xuan et al., 12 Aug 2025).

The data flow is explicit. Raw audio waveform is framed into 25 ms frames and passed to XLSR; the fine-tuned XLSR front end outputs $N_e=2048$1; a learnable linear projection maps $N_e=2048$2 to produce $N_e=2048$3; $N_e=2048$4 PN-BiMamba blocks are stacked, with the paper reporting $N_e=2048$5 or $N_e=2048$6; the final hidden sequence $N_e=2048$7 is pooled by linear attention to an utterance embedding $N_e=2048$8; and a 2-layer MLP outputs logits and real/fake scores (Xuan et al., 12 Aug 2025).

A PN-BiMamba block takes $N_e=2048$9, applies Pre-LayerNorm,

$64$0

projects to $64$1, performs local spectral mixing and SiLU,

$64$2

applies a forward SSM and gating,

$64$3

projects back to model dimension, computes a backward path using $64$4 and Mamba, fuses forward and backward streams by summation, adds a first residual, applies a second Pre-LayerNorm, and then applies an FFN with final residual (Xuan et al., 12 Aug 2025). The block therefore replaces a Conformer MHSA sublayer with two unidirectional Mamba SSM convolutions, one left-to-right and one right-to-left, together with gating under Pre-LayerNorm.

Training uses weighted cross-entropy to counter real/fake imbalance, with Adam, $64$5, $64$6, batch $64$7, early stopping with patience $64$8, and final weight averaging over the top 5 dev-EER epochs (Xuan et al., 12 Aug 2025). The key hyper-parameters reported for PN-BiMamba are 7 blocks for the large model or 4 for the small model, model dimension $64$9, and expansion $4$0 (Xuan et al., 12 Aug 2025).

The computational rationale mirrors the stereo SELD use case. Self-attention in Conformer is given as $4$1 per layer, whereas a PN-BiMamba layer is $4$2, described as near-linear in $4$3 due to selective SSM convolution (Xuan et al., 12 Aug 2025). On a V100 GPU, the Real-Time Factor over 1–6 s utterances is reported as $4$4 for XLSR-Conformer and $4$5 for Fake-Mamba(L), which is summarized as an approximately $4$6–$4$7 speed-up over MHSA at inference (Xuan et al., 12 Aug 2025).

Performance is reported on three anti-spoofing benchmarks:

Backbone	Cross-dataset EERs (%)
TransBiMamba	$4$8 on ASVspoof 21 LA, $4$9 on 21 DF, $2$0 on In-The-Wild
ConBiMamba	$2$1, $2$2, $2$3
PN-BiMamba (L)	$2$4, $2$5, $2$6

The same report compares PN-BiMamba against XLSR-Conformer $2$7 and XLSR-DuaBiMamba $2$8, stating that PN-BiMamba reduces EER by up to $2$9 on LA, $T=250$00 on DF, and $T=250$01 on In-The-Wild (Xuan et al., 12 Aug 2025).

6. Ablations, design rationale, and limitations

Ablation studies across both application areas emphasize that PN-BiMamba is not merely a label for bidirectionality; its reported performance depends on a specific combination of bidirectional SSM fusion, normalization, auxiliary convolutions, and task-specific pooling or heads.

In stereo SELD, removing bidirectional recurrence from the full BiMambaAC system reduces $T=250$02 from $T=250$03 to $T=250$04, removing asymmetric convolution yields $T=250$05, and doubling the state size from $T=250$06 changes performance to $T=250$07 with DOAE $T=250$08 and RDE $T=250$09 (Gao et al., 13 Jul 2025). The related stereo SELD discussion states that BiMamba helps because linear-time SSM captures very long-range dependencies more efficiently than MHSA, bidirectionality removes causal delay and allows the decoder to see the entire 5 s context, and asymmetric convolutions disentangle temporal versus spectral patterns, benefiting overlapping sources (Gao et al., 16 Jun 2025).

In Fake-Mamba, the analogous ablations on a 7-block PN-BiMamba show full-model EERs of $T=250$10; removing the three Pre-LayerNorm operations gives $T=250$11; removing the FFN after SSM gives $T=250$12; removing the bidirectional split gives $T=250$13; and removing linear-attention pooling gives $T=250$14 on ASVspoof 21 LA, 21 DF, and In-The-Wild, respectively (Xuan et al., 12 Aug 2025). The paper interprets the parallel SSM paths plus Pre-LN placement as better fusion of temporal and channel cues, increasing sensitivity to synthetic artifacts (Xuan et al., 12 Aug 2025).

The published limitations are domain-specific. For stereo SELD, MACs remain approximately $T=250$15 G per 5 s clip, pseudo-FOA conversion from stereo limits spatial cues versus true FOA, memory usage of bidirectional SSM remains higher than purely causal SSM, SSM state matrices can become memory-intensive as hidden size increases, and the current stereo track lacks explicit elevation estimation (Gao et al., 13 Jul 2025, Gao et al., 16 Jun 2025). For speech deepfake detection, the main emphasis is instead on real-time inference and cross-dataset robustness, with fewer explicit limitations stated in the supplied summary (Xuan et al., 12 Aug 2025).

Taken together, these reports suggest that PN-BiMamba is best understood as an audio-modeling pattern rather than a single canonical architecture: a bidirectional Mamba core, often paired