Partial-WSPT-XLSR: Efficient Wavelet Prompting

• The paper introduces a selective prompt tuning method that applies learnable wavelet-domain processing to only part of the prompt tokens, reducing computational load and enhancing detection accuracy.
• Partial-WSPT-XLSR leverages wavelet transforms to capture localized time-frequency features, outperforming Fourier-based and full-wavelet prompting variants in key evaluation metrics.
• The architecture combines a frozen XLSR backbone with a bidirectional Mamba classifier to achieve high performance with only 1.298% trainable parameters.

Partial-WSPT-XLSR is a parameter-efficient front-end for speech deepfake detection that adapts a frozen XLSR encoder through prompt tuning augmented with wavelet-domain processing. It is introduced as the most refined prompt-tuning variant in the WaveSP-Net family, alongside FourierPT-XLSR and WSPT-XLSR, and serves as the core front-end in WaveSP-Net. Its defining characteristic is selective wavelet enhancement: instead of applying wavelet processing to all prompt tokens, it applies learnable wavelet-domain transformation and sparsification only to a subset of them, while the remaining prompt tokens stay as ordinary trainable parameters. This design is intended to preserve prompt flexibility while injecting compact, multi-resolution, artifact-sensitive structure into the adaptation mechanism, without altering the frozen XLSR backbone (Xuan et al., 6 Oct 2025).

1. Position within prompt-based XLSR adaptation

Partial-WSPT-XLSR is motivated by limitations attributed to standard XLSR-based speech deepfake detectors that rely on full fine-tuning of a very large self-supervised learning model. The stated concerns are that full fine-tuning is parameter-heavy, compute-intensive, and potentially less robust for in-the-wild generalization because it can overfit to the source domain (Xuan et al., 6 Oct 2025).

The method adopts prompt tuning as a parameter-efficient fine-tuning strategy. Rather than modifying XLSR weights, it prepends trainable prompt tokens to the internal Transformer layers and updates only those prompts. In this formulation, the large pre-trained XLSR remains frozen while the prompts serve as the adaptation interface for the spoofing task.

Within the paper’s front-end progression, Partial-WSPT-XLSR follows two earlier transform-enhanced variants. FourierPT-XLSR injects FFT features into prompt embeddings. WSPT-XLSR replaces Fourier features with wavelet-domain features. Partial-WSPT-XLSR then further restricts wavelet enhancement to only part of the prompt set. This sequencing is conceptually important because it frames Partial-WSPT-XLSR not as an isolated design, but as the selective culmination of a broader attempt to fuse prompt tuning with classical signal processing transforms.

A plausible implication is that the method treats prompt tokens not merely as free latent parameters, but as a structured interface where some capacity remains unconstrained and some capacity is explicitly biased toward multi-resolution signal structure.

2. Motivation for wavelet-domain prompting

The paper contrasts Fourier-based and wavelet-based prompt enrichment in terms of the signal structures they emphasize (Xuan et al., 6 Oct 2025). FourierPT-XLSR is characterized as using global sinusoidal basis features. The paper notes three limitations of this representation for speech deepfake detection: Fourier bases are non-localized in time, they use uniform time-frequency tiling, and they are less suited to abrupt, localized artifacts.

WSPT-XLSR and Partial-WSPT-XLSR are motivated by the properties of the wavelet transform. The stated advantages are joint time-frequency localization, adaptive multi-resolution analysis, and better robustness to transient or abrupt changes. These properties are presented as better aligned with speech deepfake artifacts, which are described as subtle and localized.

Partial-WSPT-XLSR differs from full WSPT-XLSR in its selectivity. Instead of wavelet-enhancing all prompt tokens, it applies wavelet processing only to the last $m$ tokens out of the $p$ prompt tokens in each layer. The remaining tokens are retained as regular prompt parameters. In the reported implementation, $p=10$, and the partial wavelet variant uses 4 wavelet-based tokens and 6 regular tokens.

That selectivity is central to the method’s identity. The prompt remains length-preserving, but only part of it is structurally transformed. A common misunderstanding would be to treat the method as reducing prompt dimensionality; in the given formulation, prompt length stays fixed at $p$, and “partial” refers to which tokens are wavelet-enhanced, not to removal of prompt capacity.

3. Formal architecture and trainable components

Let the XLSR backbone have $\ell$ Transformer layers. For each layer $k \in \{1,\dots,\ell\}$, the method introduces a layer-specific prompt matrix

$\mathbf{P}_k \in \mathbb{R}^{p \times d},$

where $p$ is the number of prompt tokens and $d$ is the hidden size (Xuan et al., 6 Oct 2025).

The model keeps all parameters of the XLSR encoder frozen. The trainable components are the prompt embeddings $\mathbf{P}_k$, the learnable wavelet filters, and, in the full WaveSP-Net system, the Mamba-based classifier. The learnable wavelet filters comprise the analysis filters $p$0 and the synthesis filters $p$1.

The wavelet-domain transformation is applied only to the last $p$2 prompt tokens in each layer,

$p$3

These selected tokens are converted into wavelet sparse prompt tokens,

$p$4

After wavelet enhancement, the final prompt representation is formed by replacing the last $p$5 tokens with their transformed versions: $p$6

The modified prompt is then injected into each XLSR layer through concatenation with the incoming sequence embeddings: $p$7 Here, $p$8 are the transformed prompt outputs, $p$9 are the updated sequence embeddings, and $p=10$0 denotes the $p=10$1-th XLSR Transformer layer. At the end of the encoder, the final output is

$p=10$2

This formulation makes Partial-WSPT-XLSR a frozen-backbone adaptation scheme rather than a reparameterization of XLSR itself. The prompts constitute the learnable interface, and only a designated subset undergoes wavelet-domain processing.

4. Wavelet decomposition, sparsification, and reconstruction

The wavelet-specific processing pipeline consists of three components: Learnable Wavelet Decomposition (LWD), Wavelet Domain Sparsification (WDS), and Learnable Wavelet Reconstruction (LWR) (Xuan et al., 6 Oct 2025).

In LWD, the selected prompt tokens are passed through a 1D discrete wavelet transform using learnable analysis filters. $p=10$3 is the low-pass filter and $p=10$4 is the high-pass filter. The decomposition separates the signal into coarse and detail components, and the paper emphasizes that these filters are learned during training rather than fixed as in Haar or Daubechies wavelets. The conceptual decomposition is written as

$p=10$5

where $p=10$6 denotes filtering or convolution, and $p=10$7 is the token sequence being transformed.

In WDS, the low- and high-frequency coefficients are stacked, but only a fraction of positions is randomly selected according to a sparsity ratio $p=10$8. The stated purposes of this sparsification are to reduce redundancy, suppress noise, regularize training, and improve generalization. The main setup uses $p=10$9.

In LWR, the selected and sparsified coefficients are recombined using synthesis filters $p$0 and $p$1, which are the low-pass and high-pass synthesis filters, respectively. This reconstructs the compact wavelet-enhanced prompt representation $p$2. Because the analysis and synthesis filters are jointly learned, the method is designed to preserve important structure while emphasizing discriminative coefficients.

A plausible implication is that WDS is not only a compression step but also a task-specific inductive bias: the prompt pathway is encouraged to focus on sparse, localized structures rather than dense coefficient updates.

5. Function inside WaveSP-Net

WaveSP-Net combines a Partial-WSPT-XLSR front-end with a bidirectional Mamba-based back-end (Xuan et al., 6 Oct 2025). In this system, the front-end provides frozen-SSL features plus prompt adaptation, and the back-end performs sequence classification.

The classifier is based on the Mamba state-space model. The paper states that it is chosen because it captures long-range temporal dependencies efficiently, has linear-time complexity, and is suitable for high-dimensional sequence features. The Mamba back-end is further described as well-suited to the transformed wavelet-enhanced representations because it can model temporal structure effectively without the cost of attention. In WaveSP-Net, the back-end receives the final encoder output $p$3 and outputs the spoofing decision.

During training, the updated parameters are the prompt embeddings, the learnable wavelet filters, and the Mamba classifier parameters, while XLSR remains frozen. This keeps the overall system parameter-efficient.

The combination of Partial-WSPT-XLSR with bidirectional Mamba is therefore not merely a juxtaposition of front-end and classifier. The paper presents it as an architecture in which selective wavelet prompting supplies artifact-sensitive representations and the sequence model consumes them with an efficient long-range temporal mechanism.

6. Empirical performance, ablations, and interpretation

The reported experiments compare FourierPT-XLSR, WSPT-XLSR, and Partial-WSPT-XLSR using the same Mamba back-end. On Deepfake-Eval-2024, the EER values are 16.58% for FourierPT-XLSR, 13.15% for WSPT-XLSR, and 10.58% for Partial-WSPT-XLSR. On SpoofCeleb, the corresponding EER values are 0.23%, 0.19%, and 0.13%. Partial-WSPT-XLSR is reported as best on both datasets across EER, ACC, F1, and AUC (Xuan et al., 6 Oct 2025).

Front-end variant	Deepfake-Eval-2024 EER	SpoofCeleb EER
FourierPT-XLSR	16.58%	0.23%
WSPT-XLSR	13.15%	0.19%
Partial-WSPT-XLSR	10.58%	0.13%

Against stronger baselines, WaveSP-Net reaches 10.58% EER, 89.42% ACC, 86.35% F1, and 94.26% AUC on Deepfake-Eval-2024. These results are reported to surpass PT-XLSR at 20.40% EER, WPT-XLSR at 14.39% EER, and the XLS-R-1B baseline at 11.85% EER. On SpoofCeleb, WaveSP-Net achieves 0.13% EER, 99.87% ACC, 99.93% F1, and 99.99% AUC, and is reported as the best among the compared systems.

The parameter-efficiency claim is explicit: WaveSP-Net has 4.146M trainable parameters, which is only 1.298% of total parameters. This situates Partial-WSPT-XLSR within a PEFT regime rather than a full-model adaptation regime.

The ablation study isolates the contribution of each wavelet-processing component. On Deepfake-Eval-2024, removing LWD raises EER to 12.97%, removing WDS raises it to 14.34%, and removing LWR raises it to 11.33%. The paper identifies the largest degradation as coming from removal of WDS, indicating that sparsity is especially important. Replacing learnable wavelet filters with fixed ones yields 16.55% EER, which is much worse than the full model. The reported best configuration uses learnable wavelet filters, sparsity ratio $p$4, and 4 wavelet sparse prompt tokens.

The paper also reports a t-SNE visualization in which FourierPT-XLSR shows more overlap between real and fake samples, WSPT-XLSR improves separation, and Partial-WSPT-XLSR produces the cleanest clustering. This suggests that selective wavelet prompting yields more discriminative embeddings.

Taken together, the reported results support a specific interpretation of Partial-WSPT-XLSR: its advantage does not arise solely from adding wavelet structure, nor solely from PEFT, but from combining frozen-backbone prompt tuning with learnable, sparse, partial wavelet-domain processing.