ProSDD: Robust Speech Deepfake Detection

• The paper introduces ProSDD, a two-stage framework that learns speaker-conditioned prosodic variability to enhance the detection of speech deepfakes.
• It employs a real-speech-only masked prediction stage followed by joint spoof classification to preserve natural prosodic patterns during fine-tuning.
• Empirical evaluations demonstrate significant reductions in equal-error rates, particularly on expressive and out-of-domain benchmarks, underscoring the value of prosodic supervision.

Searching arXiv for the primary paper and the referenced MPM-Pro work. ProSDD is a two-stage framework for speech deepfake detection (SDD) that is designed to improve robustness to expressive and emotional spoofing attacks by explicitly learning speaker-conditioned prosodic variability from bona fide speech and preserving that structure during spoof classification (Mahapatra et al., 14 Apr 2026). It is motivated by the observation that SDD systems can achieve near-perfect error rates on standard corpora such as ASVspoof 2019 yet suffer dramatic performance drops under emotionally expressive or out-of-distribution attacks. In contrast to approaches that rely on spoof-heavy training data and may learn dataset-specific artifacts, ProSDD structures model embeddings through supervised masked prediction of prosodic variation based on pitch, voice activity, and energy, using Stage I to internalize natural prosodic variability and Stage II to jointly optimize this objective with spoof classification.

1. Problem setting and design rationale

Speech deepfake detection systems described in the source material perform well on standard benchmark datasets but often fail to generalize to expressive and emotional spoofing attacks. The stated reason is that many methods rely on spoof-heavy training data and consequently learn dataset-specific artifacts rather than transferable cues of natural speech. ProSDD addresses this failure mode by focusing on prosodic regularities in real speech rather than solely on discriminative patterns between bona fide and spoofed examples (Mahapatra et al., 14 Apr 2026).

The framework is built around supervised, speaker-conditioned prosodic learning. The high-level design is explicitly two-stage. Stage I learns prosodic variability from real speech only, and Stage II preserves this structure through an auxiliary prosodic objective while fine-tuning for spoof classification. The backbone is XLS-R, treated as an SSL encoder whose embeddings are enriched by a prosodic prediction task.

This design is presented as an analogue to human perceptual behavior: humans internalize variability in real speech and detect fakes as deviations from it. A plausible implication is that ProSDD is intended to bias the representation space toward modeling lawful within-speaker and across-speaker prosodic variation before the classifier is exposed to spoof labels. The paper’s discussion attributes the resulting gains to increased sensitivity to subtle prosodic inconsistencies in modern expressive TTS and VC systems.

2. Stage I: learning speaker-conditioned prosodic variation

Stage I uses only real speech from LibriSpeech train-clean-100 and dev. Its purpose is to teach the XLS-R backbone to predict, in a masked-prediction fashion, frame-level prosodic embeddings conditioned on speaker identity (Mahapatra et al., 14 Apr 2026).

The speaker representation is denoted $spk \in \mathbb{R}^{D_s}$ with $D_s = 192$. It is extracted per utterance using a pretrained ECAPA-TDNN, then averaged and $L_2$-normalized across all utterances of that speaker. The frame-level prosodic embedding is denoted $f_t \in \mathbb{R}^{D_p}$ with $D_p = 256$. It is produced by a prosody encoder following MPM-Pro and integrates pitch ($F_0$), voice activity, and short-term energy. For each frame $t$, the target is the concatenation of the speaker and prosodic embeddings,

$P_t^{target} = spk \parallel f_t,$

giving $P^{target} \in \mathbb{R}^{T \times D}$ with $D = D_s + D_p = 448$.

The masked-prediction pipeline begins with latent features $D_s = 192$0 from XLS-R’s convolutional front end. Span masking is applied with probability $D_s = 192$1 and span length $D_s = 192$2, yielding $D_s = 192$3. The Transformer produces contextualized embeddings $D_s = 192$4, and a linear projection maps each $D_s = 192$5 into the target space,

$D_s = 192$6

The supervised masked prediction objective is an InfoNCE loss defined over masked frame indices $D_s = 192$7. For each $D_s = 192$8, the positive example is

$D_s = 192$9

whereas the negatives comprise $L_2$0 samples, half intra-speaker and half inter-speaker:

$L_2$1

$L_2$2

The loss is

$L_2$3

with cosine similarity

$L_2$4

and temperature $L_2$5 in Stage I.

Architecturally, Stage I uses a prosody encoder that absorbs raw $L_2$6, voice-activity flags, and energy contours and outputs $L_2$7, together with a masked-prediction head consisting of a single linear layer from $L_2$8 to $L_2$9 followed by the contrastive InfoNCE computation. This stage is explicitly real-speech-only, which the ablation results later identify as crucial for robustness beyond in-domain benchmarks.

3. Stage II: joint optimization with spoof classification

Stage II initializes from the Stage I checkpoint and trains on mixed bona fide and spoofed utterances from ASVspoof 2019 or 2024 logical access. Each training iteration uses two forward passes (Mahapatra et al., 14 Apr 2026).

The first is a masked pass, identical in structure to Stage I, with the same $f_t \in \mathbb{R}^{D_p}$0 computation but with reduced masking probability $f_t \in \mathbb{R}^{D_p}$1 and temperature $f_t \in \mathbb{R}^{D_p}$2. The second is a classification pass without masking. In that pass, the contextual embeddings $f_t \in \mathbb{R}^{D_p}$3 are mean-pooled over time to form $f_t \in \mathbb{R}^{D_p}$4, and a lightweight classifier head with the sequence Linear $f_t \in \mathbb{R}^{D_p}$5 ReLU $f_t \in \mathbb{R}^{D_p}$6 Dropout $f_t \in \mathbb{R}^{D_p}$7 Linear produces logits for two classes, real versus fake. The classification objective $f_t \in \mathbb{R}^{D_p}$8 is the weighted cross-entropy over these two labels.

The total objective is

$f_t \in \mathbb{R}^{D_p}$9

with $D_p = 256$0. In the authors’ notation, $D_p = 256$1 is set to $D_p = 256$2 for the first $D_p = 256$3 epochs in order to emphasize prosodic regularization and then reduced to $D_p = 256$4 for the remaining training so that spoof discrimination dominates.

This training regime preserves the auxiliary prosodic supervision instead of treating it as a purely pretraining-stage signal. The paper’s interpretation is that Stage II prevents the learned representation from collapsing into a purely classification-driven embedding space. The ablation study supports that interpretation by showing degradation when either masked prediction and Stage I are removed or when Stage I is omitted and only auxiliary masked prediction is retained during Stage II.

4. Prosodic representation, feature extraction, and optimization details

The prosodic representation is continuous and frame-level. Following MPM-Pro’s design, $D_p = 256$5 contours are extracted at $D_p = 256$6 Hz frame rate, voice activity is flagged per frame as binary voiced or unvoiced, and short-term energy is computed from the waveform magnitude (Mahapatra et al., 14 Apr 2026). These signals are concatenated and fed into the prosody encoder to yield continuous $D_p = 256$7-dimensional embeddings $D_p = 256$8. No explicit binning or discrete quantization is applied; normalization is handled internally by the prosody encoder’s learned parameters.

The training configuration is specified separately for the two stages. Stage I uses LibriSpeech train-clean-100 and dev, runs for $D_p = 256$9 epochs, and uses batch size $F_0$0 with $F_0$1 s segments, $F_0$2, span length $F_0$3, $F_0$4, learning rates $F_0$5 for the backbone, $F_0$6 for the projection, $F_0$7 for the classification head$F_0$8, and weight decay $F_0$9. Stage II uses ASVspoof 2019 LA or ASVspoof 2024 LA train/dev, the same $t$0 epochs and batch size, $t$1, $t$2, joint loss weights $t$3, RawBoost data augmentation, and the same layer-wise learning-rate schedule. Model selection is based on training loss for ASVspoof 2019 and validation accuracy for ASVspoof 2024.

These implementation choices indicate that ProSDD is not only a representation-learning proposal but also a specific training protocol in which the auxiliary task remains active during supervised spoof learning. A plausible implication is that the method’s reported gains depend not merely on pretraining data selection but on the continued regularizing effect of the prosodic objective.

5. Empirical performance across standard and emotional benchmarks

The reported results are expressed in equal-error rate (EER). Under the ASVspoof 2019 LA training regime, RawNet2 records EERs of $t$4 on ASV19, $t$5 on ASV21, $t$6 on ASV24, $t$7 on EmoFake, and $t$8 on EmoSpoof-TTS. AASIST records $t$9, $P_t^{target} = spk \parallel f_t,$0, $P_t^{target} = spk \parallel f_t,$1, $P_t^{target} = spk \parallel f_t,$2, and $P_t^{target} = spk \parallel f_t,$3 on the same benchmarks. XLSR-SLS records $P_t^{target} = spk \parallel f_t,$4, $P_t^{target} = spk \parallel f_t,$5, $P_t^{target} = spk \parallel f_t,$6, $P_t^{target} = spk \parallel f_t,$7, and $P_t^{target} = spk \parallel f_t,$8. ProSDD records $P_t^{target} = spk \parallel f_t,$9 on ASV19, $P^{target} \in \mathbb{R}^{T \times D}$0 on ASV21, $P^{target} \in \mathbb{R}^{T \times D}$1 on ASV24, $P^{target} \in \mathbb{R}^{T \times D}$2 on EmoFake, and $P^{target} \in \mathbb{R}^{T \times D}$3 on EmoSpoof-TTS (Mahapatra et al., 14 Apr 2026).

Under the ASVspoof 2024 LA training regime, RawNet2 records $P^{target} \in \mathbb{R}^{T \times D}$4 on ASV19, $P^{target} \in \mathbb{R}^{T \times D}$5 on ASV21, $P^{target} \in \mathbb{R}^{T \times D}$6 on ASV24, $P^{target} \in \mathbb{R}^{T \times D}$7 on EmoFake, and $P^{target} \in \mathbb{R}^{T \times D}$8 on EmoSpoof-TTS. AASIST records $P^{target} \in \mathbb{R}^{T \times D}$9, $D = D_s + D_p = 448$0, $D = D_s + D_p = 448$1, $D = D_s + D_p = 448$2, and $D = D_s + D_p = 448$3. XLSR-SLS records $D = D_s + D_p = 448$4, $D = D_s + D_p = 448$5, $D = D_s + D_p = 448$6, $D = D_s + D_p = 448$7, and $D = D_s + D_p = 448$8. ProSDD records $D = D_s + D_p = 448$9 on ASV19, $D_s = 192$00 on ASV21, $D_s = 192$01 on ASV24, $D_s = 192$02 on EmoFake, and $D_s = 192$03 on EmoSpoof-TTS.

The headline results stated in the abstract are reductions of ASVspoof 2024 EER from $D_s = 192$04 to $D_s = 192$05 for the 2019-trained setting and from $D_s = 192$06 to $D_s = 192$07 for the 2024-trained setting, together with $D_s = 192$08 relative reductions on EmoFake and EmoSpoof-TTS. The high-level overview characterizes the effect as reducing EER on emotionally rich benchmarks by roughly $D_s = 192$09 while maintaining or improving upon state-of-the-art performance on standard datasets.

Taken together, these results distinguish two regimes. First, when trained on ASVspoof 2019, ProSDD retains very low EER on ASV19 and materially improves ASV24 and emotional benchmarks relative to XLSR-SLS. Second, when trained on ASVspoof 2024, ProSDD strongly improves ASV24 and emotional benchmarks relative to all listed baselines. This suggests that the framework is particularly targeted at cross-domain robustness rather than only incremental in-domain optimization.

6. Ablation evidence, interpretation, and limitations

The ablation on ASVspoof 2019 LA evaluates three variants. The configuration without masked prediction and Stage I yields EERs of $D_s = 192$10 on ASV19, $D_s = 192$11 on ASV21, $D_s = 192$12 on ASV24, $D_s = 192$13 on EmoFake, and $D_s = 192$14 on EmoSpoof. The configuration without Stage I but with only auxiliary masked prediction in Stage II yields $D_s = 192$15, $D_s = 192$16, $D_s = 192$17, $D_s = 192$18, and $D_s = 192$19. Full ProSDD yields $D_s = 192$20, $D_s = 192$21, $D_s = 192$22, $D_s = 192$23, and $D_s = 192$24 (Mahapatra et al., 14 Apr 2026).

The accompanying discussion attributes ProSDD’s gains to forcing the SSL backbone to internalize speaker-conditioned prosodic variability prior to spoof discrimination. By learning genuine $D_s = 192$25, energy, and voice-activity patterns through contrastive supervision, the model is described as becoming highly sensitive to subtle prosodic inconsistencies that modern expressive TTS and VC systems still exhibit, cues that purely classification-driven fine-tuning overlooks. The ablation study is used to support the claim that both real-only prosodic pretraining in Stage I and continued auxiliary supervision in Stage II are crucial for cross-domain robustness.

The stated limitations are also specific. ProSDD is demonstrated on English, logical-access tasks, and its reliance on a pretrained prosody encoder may limit direct portability to other languages or dialects without retraining that encoder. Future work identified in the source includes extending prosodic supervision to multilingual and cross-lingual corpora, incorporating additional expressive cues such as spectral dynamics or rhythm, and exploring alternative contrastive architectures for prosody modeling.

A common misconception in this area is that improvements on standard ASVspoof benchmarks necessarily imply robustness to expressive or emotional attacks. The paper’s framing directly rejects that assumption: near-perfect performance on standard corpora can coexist with dramatic degradation under emotionally expressive or out-of-distribution attacks. Within that problem setting, ProSDD is best understood not merely as a new classifier, but as a structured auxiliary-learning scheme for encoding natural prosodic variation and using that structure to regularize spoof detection.