Fake-Mamba: Real-Time Deepfake Speech Detection

• The paper introduces Fake-Mamba, a framework that replaces self-attention with bidirectional Mamba for efficient real-time speech deepfake detection.
• It integrates a fine-tuned XLSR front-end with state-space selective scanning and achieves EERs as low as 0.97% in cross-corpus evaluations.
• The approach leverages local convolutions and global cues to capture synthetic artifacts, ensuring robust and fast detection under streaming conditions.

Fake-Mamba is a speech deepfake detection framework that replaces self-attention with bidirectional Mamba for real-time anti-spoofing. It formulates speech deepfake detection as utterance-level binary classification, integrates a fine-tuned XLSR front-end with bidirectional Mamba encoders, performs utterance-level pooling, and predicts “real” or “fake” with an MLP classifier. In the reported cross-corpus setting—training on ASVspoof 2019 LA and testing on ASVspoof 2021 LA, ASVspoof 2021 DF, and In-The-Wild—its best configuration, PN-BiMamba, achieves 0.97%, 1.74%, and 5.85% EER, respectively, while maintaining real-time inference across utterance lengths (Xuan et al., 12 Aug 2025).

1. Problem setting and design objective

Fake-Mamba is motivated by the operational mismatch between contemporary speech synthesis and practical spoof detection. Modern TTS and VC systems can synthesize highly convincing speech, and the deployment setting of interest includes telephony channels, codecs, and compression that alter signals before detection. The target task is therefore not only accurate speech deepfake detection, but robust and fast detection under streaming or near-streaming conditions (Xuan et al., 12 Aug 2025).

Within this formulation, the input is an utterance and the output is a binary decision. The front-end operates at frame level, while the backbone learns temporal dependencies over the full utterance and the classifier acts on an utterance-level representation. This division is central to the framework’s rationale: frame-level representations capture rich acoustic and linguistic content, whereas the encoder and pooling stages aggregate evidence of synthetic artifacts that may be local, global, or jointly distributed across time (Xuan et al., 12 Aug 2025).

The evaluation protocol is explicitly cross-corpus. Training uses ASVspoof 2019 LA, described as approximately 25k utterances with 6 VC/TTS attacks. Testing uses ASVspoof 2021 LA, ASVspoof 2021 DF, and In-The-Wild, which respectively introduce telephony/channel effects, diverse codecs with more than 100 attacks, and non-controlled recording conditions. This suggests that Fake-Mamba is designed less as an in-domain benchmark optimizer than as a generalization-oriented detector (Xuan et al., 12 Aug 2025).

2. Architecture and encoder variants

The architecture consists of four stages: audio input, an XLSR front-end, a bidirectional Mamba backbone, utterance-level pooling, and a final MLP classifier. XLS-R is described as a wav2vec 2.0-based, self-supervised cross-lingual model pre-trained on 128 languages and approximately 436k hours. It is not frozen; instead, it is fine-tuned jointly with the Fake-Mamba backbone and classifier. XLSR produces frame-level features $S_f \in \mathbb{R}^{T \times C}$, after which a linear projection reduces the channel dimension to $D = 144$, yielding $S_f' \in \mathbb{R}^{T \times D}$ (Xuan et al., 12 Aug 2025).

The backbone is instantiated in three ways. TransBiMamba replaces the MHSA module in a standard Transformer block with a bidirectional Mamba module while preserving Transformer feed-forward and normalization structures. ConBiMamba replaces the MHSA module in a Conformer block with bidirectional Mamba while retaining convolution sub-blocks. PN-BiMamba, the best-performing variant, uses Pre-LayerNorm stabilization, parallel selective state-space paths, gating, bidirectional fusion, residual connections, and an FFN. After the backbone, Linear Attention Pooling aggregates across time into $S_u \in \mathbb{R}^{D}$, and an MLP maps $S_u$ to real/fake logits (Xuan et al., 12 Aug 2025).

The model is explicitly organized around local and global artifact capture. Local cues are modeled with 1D convolutions, especially in PN-BiMamba, to enhance short-range spectral-temporal patterns such as micro-spectral distortions, phase inconsistencies, and codec-induced banding. Global cues are modeled through Mamba’s state-space selective scan, which provides a global receptive field with near-linear complexity and is used to capture prosodic and long-range temporal or linguistic cues characteristic of synthetic speech. Bidirectionality is emphasized because some artifacts depend on both left and right temporal neighborhoods, including unnatural coarticulation and inconsistent long-span intonation (Xuan et al., 12 Aug 2025).

A concise comparison of the three encoder backbones is given below.

Backbone	Design summary	Reported EERs
TransBiMamba	Transformer MHSA replaced by bidirectional Mamba	21LA 1.25%, 21DF 2.55%, ITW 7.59%
ConBiMamba	Conformer MHSA replaced by bidirectional Mamba, conv retained	21LA 1.80%, 21DF 2.50%, ITW 8.11%
PN-BiMamba	Pre-LayerNorm, parallel SSM paths, gating, bidirectional fusion, FFN	21LA 0.97%, 21DF 1.74%, ITW 5.85%

PN-BiMamba is consistently the strongest of the three, while ConBiMamba is the weakest in the reported comparison. A plausible implication is that the specific combination of Pre-Norm stabilization, selective gating, and bidirectional fusion is more consequential than merely replacing attention inside a pre-existing Transformer or Conformer template (Xuan et al., 12 Aug 2025).

3. State-space formulation and bidirectional Mamba mechanism

Fake-Mamba is grounded in the standard continuous-time SSM formalization

$$\frac{d h(t)}{dt} = A h(t) + B x(t), \qquad y(t) = C h(t) + D x(t),$$

with $h(t) \in \mathbb{R}^{N_M}$, $x(t), y(t) \in \mathbb{R}$, $A \in \mathbb{R}^{N_M \times N_M}$, $B \in \mathbb{R}^{N_M \times 1}$, $D = 144$0, and $D = 144$1. Under Zero-Order Hold discretization with step $D = 144$2,

$D = 144$3

and the discrete dynamics become

$D = 144$4

The global convolution view is expressed as

$D = 144$5

These equations establish how a structured state evolution can yield a global receptive field without quadratic attention (Xuan et al., 12 Aug 2025).

Mamba introduces input-dependent parameterization so that $D = 144$6 and $D = 144$7 depend on $D = 144$8, allowing the model to selectively emphasize relevant content while suppressing irrelevant features. In the PN-BiMamba formulation, this is instantiated through local preconditioning and gating:

$D = 144$9

where $S_f' \in \mathbb{R}^{T \times D}$0 denotes the Hadamard product. The selective scan updates kernels and content on the fly, producing a content-aware global receptive field at near-linear time (Xuan et al., 12 Aug 2025).

Bidirectionality is implemented by a forward scan and a backward scan over the flipped sequence:

$S_f' \in \mathbb{R}^{T \times D}$1

In PN-BiMamba, the two streams are fused additively,

$S_f' \in \mathbb{R}^{T \times D}$2

and then passed through residual additions, LayerNorm, and an FFN. Self-attention is characterized in the paper as requiring $S_f' \in \mathbb{R}^{T \times D}$3 time and $S_f' \in \mathbb{R}^{T \times D}$4 memory, whereas Mamba or Bi-Mamba uses $S_f' \in \mathbb{R}^{T \times D}$5 time and $S_f' \in \mathbb{R}^{T \times D}$6 memory and supports streaming or online scanning. This computational distinction is not merely asymptotic; it is presented as a deployment-oriented argument for replacing MHSA in real-time speech anti-spoofing (Xuan et al., 12 Aug 2025).

4. Training protocol, optimization, and model scale

The training protocol uses 16 kHz audio and fixed 4.175 s training segments, corresponding to 66,800 samples. Augmentation is performed with RawBoost, but in dataset-specific forms: for 21LA, linear/nonlinear convolutive noise plus impulsive signal-dependent additive noise; for 21DF, stationary signal-independent additive noise with random coloration. This separation suggests an explicit attempt to match augmentation to test-domain nuisance factors rather than use a single universal corruption model (Xuan et al., 12 Aug 2025).

The loss is weighted cross-entropy,

$S_f' \in \mathbb{R}^{T \times D}$7

used to address class imbalance. EER is defined by the threshold $S_f' \in \mathbb{R}^{T \times D}$8 satisfying $S_f' \in \mathbb{R}^{T \times D}$9, and the confidence interval is reported as

$S_u \in \mathbb{R}^{D}$0

Optimization uses Adam with initial learning rate $S_u \in \mathbb{R}^{D}$1, weight decay $S_u \in \mathbb{R}^{D}$2, batch size 32, and a maximum of 100 epochs. Early stopping uses patience 7 on dev loss, and checkpoint averaging takes the top 5 epochs with the lowest dev EERs [2508