Spoofing-Aware Speaker Verification

• The paper introduces the Wavelet Prompt-Tuned XLSR-AASIST module that significantly reduces error rates in spoof detection while requiring minimal trainable parameters.
• It employs a two-stage cascade architecture combining a frozen SSL backbone with dedicated spoof countermeasure and multi-model ensemble for enhanced speaker verification.
• Comprehensive evaluations using EER and Macro a-DCF metrics demonstrate both the robustness in in-domain settings and challenges in cross-domain generalization.

A Spoofing-Aware Speaker Verification (SASV) framework is designed to simultaneously assess speaker identity and audio authenticity, forming an integrated defense against generative spoofing attacks such as text-to-speech (TTS), voice conversion (VC), and adversarial voice synthesis. Recent systems, notably the UZH-CL submission to the WildSpoof 2026 challenge, achieve robust cascaded verification by leveraging prompt-tuned self-supervised learning, multi-model ensemble verification, and state-of-the-art spectral countermeasures. Central to this approach is the Wavelet Prompt-Tuned XLSR-AASIST (WPT-XLSR-AASIST) module, which injects multi-resolution spectral cues derived from discrete wavelet transforms into a frozen SSL backbone, providing substantial gains in spoof detection with minimal trainable parameters (Farhadipour et al., 24 Jan 2026, Xie et al., 9 Apr 2025).

1. Architectural Components of Spoofing-Aware Speaker Verification

The SASV framework implements a two-stage cascade, integrating a dedicated spoof countermeasure with ensemble automatic speaker verification (ASV):

• Front-end Countermeasure: The XLSR-AASIST architecture consists of a frozen XLSR-53 wav2vec-2.0 backbone ($\approx$300M parameters), augmented by a Wavelet Prompt Tuning (WPT) module that inserts 10 learnable tokens (6 global, 4 wavelet) into selected transformer layers.
• Wavelet Prompt Tuning Module: Employs spectral prompts constructed by encoding multi-scale discrete wavelet coefficients; these inject time-frequency artifact sensitivity into the latent representations.
• AASIST Back-end: Processes the prompt-augmented embeddings using frame–channel graphs and graph-attention networks (GATs) for binary spoof vs. bona fide discrimination.
• ASV Ensemble: A score-level pooling of ResNet34, ResNet293, and WavLM-ECAPA-TDNN models is performed with Z-score normalization and averaging. This ensemble jointly evaluates speaker identity for authenticated audio.
• Interaction Flow: $x(t)\rightarrow$ XLSR + WPT $\rightarrow$ AASIST $\rightarrow$ spoof scores; accepted bona fide utterances are forwarded to the ASV ensemble for identity verification.

2. Wavelet Prompt Tuning Mechanism and Mathematical Formalism

Wavelet Prompt Tuning (WPT) is a parameter-efficient approach enabling spectral artifact detection without fine-tuning the entire SSL backbone:

• Discrete Wavelet Transform (DWT):
  • Given a signal $x(t)$ and a mother wavelet $\psi(t)$, 1D DWT coefficients are computed as

  $$W_\psi[x](j,k) = \int_{-\infty}^\infty x(t)\,\psi_{j,k}(t)\,dt$$

  where $\psi_{j,k}(t) = 2^{-j/2}\psi(2^{-j}t-k)$. - For prompt feature extraction, coefficient vectors $w^{(j)}\in\mathbb{R}^{T/2^j}$, $j=1,...,J$, are concatenated and embedded using a small MLP.

• Prompt Token Construction:

  • Global tokens: $P_0 = \{p_0^{(1)},\ldots,p_0^{(6)}\}$.
  • Wavelet tokens: $P_w = \{p_w^{(1)},...,p_w^{(4)}\}$, with

  $p_w^{(n)} = W_E\cdot[w^{(s_n)}; w^{(s_n+1)}] + b_E$

• Insertion into Transformer Layers:

  • At layer $\ell$, prepend prompt tokens:

  $$\tilde{Z}^{(\ell-1)} = [p_0^{(1)},...,p_0^{(6)},p_w^{(1)},...,p_w^{(4)}; Z^{(\ell-1)}]$$ - Multi-head attention updates the frame representations with spectral bias. After feed-forward, prompt positions are discarded; frame embeddings continue to layer $\ell+1$.

This suggests that spectral discontinuities caused by generative attacks (e.g., vocoder artifacts or spectrum smoothing) become more salient for the countermeasure.

3. Training Strategies and Parameter Efficiency

• Objective Function: Weighted binary cross-entropy:

$$L_{CE} = -\alpha\cdot y\cdot\log(\sigma(s)) - (1-\alpha)\cdot(1-y)\cdot\log(1-\sigma(s))$$

where $y\in\{0,1\}$ and $s$ is the AASIST output logit.

• Data Protocols:

  • SASV: VoxCeleb2 and SpoofCeleb (≈50K bona fide and 50K spoof utterances).
  • All-type ADD: Combined sets from ASVspoof2019, Codecfake, CtrSVDD, and FakeMusicCaps (Xie et al., 9 Apr 2025).
• Optimization Details:
  • Mini-batch size: 64 with 1:1 class balance.
  • Augmentations: MUSAN noise, RIR reverberation, codec simulation.
  • AdamW optimizer, cosine annealing from $1\times10^{-4}$ to $1\times10^{-6}$ over 15 epochs.
• Parameter Economy:
  • WPT requires ≈1M trainable parameters compared to >300M if the backbone were fine-tuned.
  • For cross-type verification, PT/WPT modules match or surpass fine-tuning, needing only 458× fewer trainable parameters.

4. Evaluation Metrics and Comparative Performance

Performance for spoof detection and speaker-and-spoof joint verification is measured using EER and Macro a-DCF. Table A presents results on SASV and ADD tasks.

Model	ASVspoof-like Dev	Eval	WildSpoof EER	Macro a-DCF	ADD Avg EER
XLSR-AASIST (baseline)	0.45%	0.50%	2.35%	0.0469	10.50%
WPT-XLSR-AASIST	0.23%	0.16%	2.08%	0.0375	3.58%
Full Fine-tuning (ADD)	-	-	-	-	4.98%

The WPT-XLSR-AASIST countermeasure achieves a 0.16% EER on in-domain spoof detection, lowering end-to-end SASV EER to 2.08% and Macro a-DCF to 0.0375 (Farhadipour et al., 24 Jan 2026). In the all-type ADD benchmark, WPT-XLSR-AASIST yields an average EER of 3.58%, outperforming full fine-tuning and standard prompt-tuning (Xie et al., 9 Apr 2025). Evaluation on unseen datasets reveals cross-domain generalization challenges, with a-DCF worsening to ≈0.41 (ASVspoof 2025 F5) and ≈0.32 (ASV 2022).

5. Model Robustness, Cross-Domain Generalization, and Limitations

• Artifact Sensitivity: Wavelet prompts enhance focus on time–frequency irregularities characteristic of many TTS/VC spoofing pipelines—such as spectral smoothing, phase mismatches, and unnatural transitions.
• Generalization Gap: Despite low in-domain EERs, performance drops substantially on novel vocoder attacks. Macro a-DCF on SpoofCeleb is 0.0457, whereas out-of-domain sets see degradations to 0.32–0.41 (Farhadipour et al., 24 Jan 2026).
• Prompt Token Design: Empirically, combining 4 wavelet tokens with 6 standard tokens per inserted layer (4 layers evenly selected) yields optimal trade-offs for both spoofing detection and computational overhead.
• Type-Invariant Detection: In all-type ADD scenarios, the fixed DWT integration in WPT supports universal CM capability for cross-modal attacks (speech, music, singing, generic sounds) without adding new trainable parameters (Xie et al., 9 Apr 2025).

A plausible implication is that transferability of spectral features is restricted by the diversity of attack pipelines, motivating expanded prompt design for future research.

6. Future Directions in Spoofing-Aware Speaker Verification

Envisioned directions to strengthen spoofing awareness and generalization include:

• Dynamic Prompt Generation: Adaptive prompt synthesis per utterance may enable countermeasures to “seek out” novel waveform distortions.
• Multi-domain Prompt Banks: Training prompt tokens on a breadth of spoof corpora could provide broader coverage of attack spaces.
• Contrastive Cross-domain Alignment: Incorporating bona fide data from disparate datasets with contrastive objectives may regularize embeddings for consistent decision boundaries.
• Extended Audio Modalities: Universal countermeasure training on diverse material (speech, music, sound effects, singing) addresses the increasing prevalence of cross-type deepfake threats.

Wavelet Prompt Tuning, as implemented in prompt-tuned SASV systems and in universal countermeasures for all-type deepfake audio detection, injects compact, multi-scale spectral side-information to the self-supervised backbone, providing robust artifact modeling capabilities and significant error rate reductions with only minimal parameter increases (Farhadipour et al., 24 Jan 2026, Xie et al., 9 Apr 2025).