WildSpoof Challenge

• WildSpoof Challenge is a speech research initiative that advances spoofed speech generation and detection by leveraging in-the-wild, unconstrained datasets.
• It features two independent tracks: a TTS track for synthesizing spoofed speech under varied text and speaker conditions, and a SASV track for robust speaker verification.
• The challenge fosters interdisciplinary collaboration, encouraging advancements in realistic data processing and security-sensitive voice biometrics.

The WildSpoof Challenge is a speech research initiative focused on advancing the robustness and realism of both automatic spoofed speech generation and detection capabilities by leveraging in-the-wild datasets. The challenge features two parallel tracks—Text-to-Speech (TTS) synthesis for spoofed speech generation and Spoofing-robust Automatic Speaker Verification (SASV) for the detection of spoofed speech. The core objective is to promote the use of unconstrained, naturalistic data and to strengthen interdisciplinary collaboration between the communities working on spoofing generation and detection (Wu et al., 23 Aug 2025).

1. Core Objectives and Scope

The WildSpoof Challenge is constructed with two primary goals:

• Promote in-the-wild data use for TTS generation and SASV: Traditional research relies heavily on clean, controlled datasets. WildSpoof explicitly focuses on noisy, variable, and truly unconstrained acoustic environments, enabling more realistic modeling and evaluation.
• Encourage interdisciplinary development: By running intertwined but strictly independent tracks (TTS for spoofing generation; SASV for spoofing detection), the challenge fosters knowledge transfer and methodological innovation across both communities. Organizer coordination ensures consistent protocols, but each team participates in only one track.

Systems developed within this challenge are expected to function robustly in realistic environments, moving the field closer to practical, deployable solutions for secure biometric speech processing.

2. Challenge Structure: Tracks and Task Definitions

Track 1: Text-to-Speech Synthesis (TTS)

• Task: Given reference utterances and text, generate speech samples that convincingly spoof the voice identity of specified speakers.
• Protocol: Two levels of difficulty—TITW-KSKT (Known Speaker, Known Text) and TITW-KSUT (Known Speaker, Unknown Text)—test the ability of the TTS system to generalize to novel text and, where relevant, unseen speakers.

Track 2: Spoofing-robust Automatic Speaker Verification (SASV)

• Task: Given an unlabeled test utterance and reference enroLLMent utterances, determine whether the test is:
  • A bona fide target
  • A bona fide non-target
  • A spoofed target
• Protocol: The SpoofCeleb dataset and its evaluation design include these trial types, simulating real-world verification challenges.

Task Independence: Teams participate in either spoofing generation or spoofing detection; cross-track competition is not permitted though methods from each domain can inform the other (Wu et al., 23 Aug 2025).

3. Data Protocols and Evaluation Design

Datasets

• TTS Track: Uses TITW-Easy and TITW-Hard for training, and TITW-KSKT / TITW-KSUT for evaluation. Data diversity is realized via unconstrained recording conditions and variable speaker/text pairings.
• SASV Track: Employs the SpoofCeleb dataset for both training and test trials, including genuine and spoofed utterances mapped to specific trial types.

Trial Structure

• For TTS, each submission consists of thousands of .wav files generated from reference text/utterance pairs.
• For SASV, participants submit a table of trial scores assigning confidence to each enroLLMent/test pair.

Evaluation Metrics

TTS Track

Evaluated on:

• Mel-Cepstral Distortion (MCD)
• UTMOS
• DNSMOS
• Word Error Rate (WER)
• Speaker Similarity (SPK-sim)

All metrics implemented in the Versa toolkit; there is no system ranking, but aggregate challenge results are reported.

SASV Track

• Uses agnostic Detection Cost Function (a-DCF):
  • Incorporates miss rate and false alarm rates for both non-target and spoof trials.
  • Priors: $\pi_{tar} = 0.9405$, $\pi_{non} = 0.0095$, $\pi_{spf} = 0.05$
  • Costs: $C_{miss} = 1$, $C_{fa} = 10$, $C_{fa,spoof} = 10$
• Submissions require trial name, enroLLMent target speaker ID, and a continuous-valued score (higher score = greater likelihood of a correct target trial).

Reference implementations and protocols are distributed via GitHub (Wu et al., 23 Aug 2025).

4. Data Realism and Protocol Significance

A distinguishing feature of WildSpoof is its explicit commitment to unconstrained data:

• Training and evaluation data are sourced from real-world environments (not lab settings), reflecting genuine noise, device variability, and speaker diversity.
• Data protocols ensure evaluation on both familiar and unfamiliar text/speaker conditions (for TTS) and genuine versus spoofed trials in the presence of spontaneous variability (for SASV).
• This design supports comparative research on model generalization, overfitting mitigation, and robustness against real-world speech variabilities.

Such realism eliminates the artificial performance inflation seen in controlled settings and drives innovation toward deployable security-sensitive voice and speech technologies.

5. Collaboration Between Generation and Detection Communities

The challenge actively stimulates exchange between often siloed communities:

• Spoof Generation Insights: Advances in TTS modeling produce more convincing voice spoofs, which can inform detection strategies regarding new artifact types and failure modes.
• Spoof Detection Advances: Improved SASV approaches highlight areas where generation methods succeed or fail, enabling feedback that can enhance future TTS system designs.
• Cross-domain Methodology: Integrated and robust systems are more likely when knowledge is shared regarding feature selection, adversarial vulnerability, and naturalistic evaluation.

This structure is intended to lay the foundation for holistic, ecosystem-wide security in biometric speech systems.

6. Evaluation Plan and Technical Criteria

Precisely-defined submission and evaluation criteria ensure fair and reproducible system benchmarking:

• TTS submissions: Single ZIP archive with required .wav files, strict format compliance (sample rate, bit depth), and file count.
• SASV submissions: TSV file listing trial information and continuous scores, with reference implementation for score computation.
• Metrics: All holders of performance metrics are made public and computed automatically; for SASV, the cost function is explicitly defined in LaTeX, enabling easy auditing and reproducibility.

No ranking is used in the TTS track; rather, metric distributions across submissions serve as the comparative result summary.

7. Impact and Future Directions

By utilizing in-the-wild data and rigorous protocols, the WildSpoof Challenge is positioned to:

• Exemplify best practices in benchmarking for both speech synthesis and speaker verification under genuinely non-ideal conditions.
• Illuminate the limits and strengths of current deep learning architectures and classical methods in unconstrained environments.
• Enable realistic assessment of security vulnerabilities and detection capabilities in biometric systems.
• Catalyze sustained interdisciplinary collaborations leading to more robust, integrated systems for voice biometrics.

A plausible implication is that research outcomes and engineering practices developed through WildSpoof will directly influence future standards for voice-based authentication and adversarial attack resilience in production systems.

---

WildSpoof stands as a prominent benchmarking initiative explicitly bridging the gap between clean-lab benchmarks and the highly variable realities of in-the-wild speech data. Its twin focus on generation and detection, rigorous evaluation metrics, and coordinated protocol design make it a reference point for security-sensitive and robust voice technology development (Wu et al., 23 Aug 2025).