WSPT-XLSR for Speech Deepfake Detection

• WSPT-XLSR is a parameter-efficient front-end that applies learnable wavelet prompt tuning on a frozen XLSR encoder to capture both coarse and fine synthetic speech artifacts.
• It combines learnable wavelet decomposition, sparsification, and reconstruction to inject time-frequency cues into transformer layers for improved speech deepfake detection.
• It achieves competitive performance with only around 1.3% trainable parameters compared to full fine-tuning, as shown on Deepfake-Eval-2024 and SpoofCeleb benchmarks.

WSPT-XLSR, short for Wavelet Sparse Prompt Tuning for XLSR, is a parameter-efficient front-end for speech deepfake detection that combines prompt tuning with the wavelet transform on top of a frozen XLSR backbone. It was introduced as part of a family of XLSR-based front-ends that also includes FourierPT-XLSR and Partial-WSPT-XLSR, and is used within WaveSP-Net, a system that couples a wavelet-enhanced prompt front-end with a bidirectional Mamba-based back-end. The method inherits its backbone from XLSR—the multilingual self-supervised speech representation framework derived from wav2vec 2.0 and pretrained across many languages—while shifting adaptation from full backbone fine-tuning to prompt and transform layers tailored for synthetic-speech artifacts (Xuan et al., 6 Oct 2025, Conneau et al., 2020).

1. Definition and architectural placement

WSPT-XLSR is defined as a parameter-efficient front-end for speech deepfake detection that operates with a frozen XLSR model and augments each transformer layer with prompt tokens. In the formulation reported for WaveSP-Net, audio is preprocessed into 4-second segments at 16 kHz, and the XLSR-300M feature extractor produces embeddings of shape (201, 1024). At each transformer layer $k$, the model adds $p$ prompt tokens $\mathbf{P}_k \in \mathbb{R}^{p \times d}$, with the reported setting typically using $p=10$ and $d=1024$. The last $m$ prompt tokens are subjected to wavelet-domain enhancement, while the remaining tokens are kept as standard learnable prompts (Xuan et al., 6 Oct 2025).

This places WSPT-XLSR between two established adaptation regimes. On one side are fully fine-tuned XLSR systems, where all backbone parameters are updated for the target task; on the other are lightweight prompt-only methods such as PT-XLSR and transform-based variants such as FourierPT-XLSR. WSPT-XLSR preserves the frozen self-supervised encoder and shifts task specialization into prompt construction and downstream classification. In the reported implementation, the final representations are passed to a Mamba-based classifier, and only the prompts, learnable wavelet filters, and classifier are trained (Xuan et al., 6 Oct 2025).

2. Wavelet sparse prompt-tuning mechanism

The distinctive component of WSPT-XLSR is its wavelet-domain processing of prompt tokens. The method applies three operations to the last $m$ prompt tokens at each transformer layer: Learnable Wavelet Decomposition (LWD), Wavelet Domain Sparsification (WDS), and Learnable Wavelet Reconstruction (LWR). During decomposition, each prompt token vector is transformed with learnable low-pass and high-pass filters, denoted $F_0$ and $F_1$. This differs from a fixed discrete wavelet transform because the filters are optimized jointly with the task objective (Xuan et al., 6 Oct 2025).

After decomposition, WSPT-XLSR performs sparsification in the wavelet domain. Only a subset of coefficient positions is retained for update; the reported sparsity ratio is $\rho=0.1$, meaning that 10% of wavelet features are randomly selected and updated per forward pass. Reconstruction is then carried out with learned synthesis filters $p$0 and $p$1, producing a set of wavelet-enhanced prompt tokens $p$2. Prompt construction is written as

$p$3

and the layer computation is

$p$4

The role of this mechanism is to inject multi-resolution structure into the prompts. The reported interpretation is that wavelet processing enables the prompts to capture both coarse and fine artifacts by joint time-frequency localization, which is presented as critical for detecting subtle synthetic cues that are temporally localized and spectrally heterogeneous (Xuan et al., 6 Oct 2025).

3. Parameter efficiency and training behavior

A central property of WSPT-XLSR is that the XLSR backbone is kept entirely frozen. The trainable components are restricted to the prompt tokens, the learnable wavelet filters, and the back-end classifier. This is the key reason the method is described as parameter-efficient: adaptation is expressed through auxiliary structures rather than through updating the large self-supervised backbone (Xuan et al., 6 Oct 2025).

The reported parameter counts make this efficiency explicit. In the comparison provided for WaveSP-Net, PT-XLSR uses 4.145M (~1.3%) trainable parameters, WSPT-XLSR also uses 4.145M (~1.3%), and Partial-WSPT-XLSR uses 4.146M (~1.3%). These figures are contrasted with a Full XLS-R-1B system using 965M trainable parameters. The design intent is not only to reduce training cost, but also to avoid overfitting and catastrophic forgetting associated with full fine-tuning of a large multilingual encoder. The sparsification stage is additionally described as a stochastic regularizer that reduces redundancy, prevents overfitting, and strengthens resistance to noise and artifacts (Xuan et al., 6 Oct 2025).

This parameter-efficient strategy is notable in the broader XLSR lineage. The original XLSR program emphasized multilingual self-supervised pretraining and CTC fine-tuning for ASR, with strong gains for low-resource languages (Conneau et al., 2020). WSPT-XLSR repurposes the same family of encoders for a different target problem—synthetic speech detection—while changing the adaptation interface from encoder fine-tuning to structured prompt injection (Xuan et al., 6 Oct 2025).

4. Reported empirical performance

The primary reported evaluations for WSPT-XLSR are on Deepfake-Eval-2024 and SpoofCeleb. In both benchmarks, WSPT-XLSR improves substantially over FourierPT-XLSR, while Partial-WSPT-XLSR delivers the strongest results among the prompt-based front-ends (Xuan et al., 6 Oct 2025).

Model	Deepfake-Eval-2024 EER (%)	SpoofCeleb EER (%)
FourierPT-XLSR	16.58	0.23
WSPT-XLSR	13.15	0.19
Partial-WSPT-XLSR	10.58	0.13

Beyond EER, WSPT-XLSR reports 86.85 ACC, 83.84 F1, and 93.33 AUC on Deepfake-Eval-2024, and 99.89 ACC, 99.92 F1, and 99.91 AUC on SpoofCeleb. Partial-WSPT-XLSR improves these to 89.42 ACC, 86.35 F1, and 94.26 AUC on Deepfake-Eval-2024, and to 99.87 ACC, 99.93 F1, and 99.99 AUC on SpoofCeleb. The WaveSP-Net system, which incorporates Partial-WSPT-XLSR, is reported to achieve 10.58% EER on Deepfake-Eval-2024, outperforming XLS-R-1B with full fine-tuning at 11.85% EER, while using only 1.3% of the total XLSR parameters—approximately 4.1M trainable parameters (Xuan et al., 6 Oct 2025).

The paper’s characterization of these results is specific: WSPT-XLSR is described as a close second to Partial-WSPT-XLSR and as substantially better than FourierPT-XLSR on the reported benchmarks. This positions wavelet-enhanced prompt tuning as the stronger transform-based prompt strategy within the evaluated design space (Xuan et al., 6 Oct 2025).

5. Comparative interpretation and ablation evidence

The main empirical comparison in the originating study is against FourierPT-XLSR and against fully fine-tuned XLSR. The stated limitation of FourierPT-XLSR is that it lacks time-frequency localization, which constrains discriminability when synthetic artifacts are localized in time. WSPT-XLSR is reported to outperform FourierPT-XLSR on all benchmarks, while remaining far more parameter-efficient than fully fine-tuned models (Xuan et al., 6 Oct 2025).

Ablation findings assign substantial importance to the wavelet components themselves. The study reports that removing sparsification or using fixed (not learnable) wavelet filters significantly worsens performance, with up to 35%–56% EER increase. It also reports that t-SNE visualization yields better separation between real and fake speech for WSPT-XLSR and especially for Partial-WSPT-XLSR, which is presented as evidence that the wavelet-enhanced prompts improve discriminative structure in the embedding space (Xuan et al., 6 Oct 2025).

In the wider XLSR-based deepfake detection literature, architectural changes have also been applied at other points in the pipeline. XLSR-GRKAN-Conformer replaces the MLP projector with a Group Rational KAN layer and reports 0.70% EER on ASVspoof 2021 LA (Var) (Phuong et al., 17 Jun 2025). Fake-Mamba retains an XLSR front-end and uses bidirectional Mamba variants as the temporal backbone, reporting 0.97%, 1.74%, and 5.85% EER on 21LA, 21DF, and In-The-Wild (Xuan et al., 12 Aug 2025). These systems are evaluated on different benchmarks, but they clarify WSPT-XLSR’s role: rather than replacing the projector or fine-tuning the encoder end-to-end, it modifies the front-end prompting interface while keeping the pretrained XLSR parameters frozen (Xuan et al., 6 Oct 2025).

6. Terminology, scope, and related uses of the acronym

A plausible source of confusion is that “WSPT” is used in unrelated areas and, within XLSR research, is not uniquely associated with wavelet prompt tuning. In the speech deepfake detection setting, WSPT-XLSR denotes Wavelet Sparse Prompt Tuning for XLSR (Xuan et al., 6 Oct 2025). However, in multilingual ASR research, the same string is used in a different sense for language-specific adaptive weights (LSAW, WSPT-XLSR), where adaptive low-rank scale and bias terms are attached to the encoder. In that setting, the method reports 22.09 Avg. WER, a 14.3% relative gain over the standard XLSR model, and only 0.5% parameter increase over the base multilingual encoder (Ding et al., 2022).

This suggests that WSPT-XLSR should be interpreted through its immediate paper context rather than through the acronym alone. A second, entirely unrelated use appears in scheduling theory, where WSPT denotes the Weighted Shortest Processing Time rule for minimizing weighted completion time on identical parallel machines (Jäger et al., 2018). The speech-model use of WSPT-XLSR is therefore unrelated both to scheduling theory and to the multilingual-ASR adaptive-weights formulation, despite the acronym overlap.

Within the broader XLSR adaptation landscape, WSPT-XLSR belongs to a family of methods that try to improve specialization without abandoning a strong multilingual self-supervised backbone. Related examples include S3Net, which uses sparse sharing sub-networks to mitigate language interference in multilingual speech recognition (Lu et al., 2022), and SAPT, which uses continual self-supervised adaptation of XLSR-128 to target domains and languages for spoken language and dialect identification (Shaik et al., 2023). WSPT-XLSR shares the parameter-efficiency objective of these methods, but applies it to speech deepfake detection through structured wavelet-domain prompt construction rather than through sub-network masking or continued pretraining.