SpoofCeleb: Speech Deepfake & SASV Dataset

• SpoofCeleb is a large-scale dataset of bona fide and spoofed audio recordings that captures real-world channel, noise, and speaker variability.
• It utilizes a fully automated pipeline with state-of-the-art TTS and ASR systems to generate 23 distinct spoof attack types from in-the-wild data.
• The dataset provides rigorous benchmarking for Speech Deepfake Detection and SASV using comprehensive metrics and standardized protocols.

SpoofCeleb is a large-scale dataset designed to facilitate research in Speech Deepfake Detection (SDD) and Spoofing-robust Automatic Speaker Verification (SASV), with a focus on realistic, in-the-wild acoustic variability and high speaker diversity. All bona fide and spoofed samples are sourced or generated from real-world data, addressing the limitations of previous SDD/SASV corpora that often rely on studio-quality, read speech and present restricted variety in speakers or spoofing techniques. SpoofCeleb introduces over 2.5 million utterances from 1,251 speakers, with 23 distinct spoof attack types generated using contemporary TTS systems trained directly on bona fide data exhibiting natural channel and noise conditions. The dataset is partitioned with strict speaker separation across train, validation, and evaluation, and is released with standardized protocols and baselines to support rigorous benchmarking (Jung et al., 2024).

1. Corpus Construction Pipeline

SpoofCeleb's fully automated construction process ensures both breadth and quality in bona fide and spoof classes:

• Source Data and Segmentation: Starting from VoxCeleb1, WhisperX ASR (Whisper Large v2) and phoneme alignment generate transcripts. Segmentation occurs at pauses > 500 ms, yielding segments of 1–8 seconds.
• Filtering and Quality Control: Non-English, too-short or too-long segments, utterances with long individual words, and empty transcriptions are discarded ("TITW-Hard"). DEMUCS is applied for denoising, and DNSMOS BAK score (< 3.0) is used as a quality filter, resulting in the "TITW-Easy" set (~248,000 bona fide segments with genuine noise/reverberation).
• Spoof Generation: Twenty-three spoof-generation systems (TTS models) are trained on TITW-Easy. These include 4 acoustic models (TransformerTTS, GradTTS, Matcha-TTS, BVAE-TTS), 6 waveform/vocoder models (DiffWave, HiFiGAN, Parallel-WaveGAN, NSF-HiFiGAN, BigVGAN, WaveGlow), and 5 end-to-end or codec models (VALL-E, Multi-Scale Transformer, Delay, MQTTS, VITS). Every spoofed utterance is synthesized using the original bona fide transcript and speaker.

This pipeline guarantees that both bona fide and spoofed data reflect real-world variability in channel, noise, and speaker.

2. Dataset Characteristics

SpoofCeleb provides a uniquely large and diverse resource for SDD and SASV:

Statistic	Value	Details
Total utterances	2,687,292 (~2.5M)	A00: 248K bona fide; remainder spoofed
Speakers	1,251	Same as VoxCeleb1
Spoof attack types	23 (A01–A23)	Each attack is a unique TTS system/configuration
Bona fide set	A00 (248K segments)	Used to train all TTS systems
Spoof set	A01–A23	Each comprises the segmented transcript set

The attack taxonomy covers a wide architectural range (autoregressive, diffusion, flow-matching, variational, adversarial, codec-based) and includes both “fully known,” “partially known,” and “unknown” attack scenarios in evaluation partitions. Quality metrics (SPF-EER, MCD, UTMOS, DNSMOS, WER) for each attack demonstrate that most attacks retain high naturalness and intelligibility, while reliably challenging state-of-the-art ASV systems (many with SPF-EER > 20%).

3. Data Partitioning and Experimental Protocols

SpoofCeleb's split strategy is designed to prevent overfitting and to support both SDD and SASV task designs:

• Speaker Split: No speakers appear in more than one partition.
  • Train: 1,171 speakers
  • Validation: 40 speakers
  • Evaluation: 40 speakers
• Attack Assignment:
  • Train: A01–A10 (covering main system classes)
  • Validation: Mix of known/unknown attacks (A06, A07, A11, A12, A13, A14)
  • Evaluation: A15–A23 (includes "white-box" and fully/partially unknown attacks)
• Counts per Partition:
  • Train: 2,540,421 utterances (no SASV trials)
  • Validation: 55,741 utterances; 39,353 SASV trials
  • Evaluation: 91,130 utterances; 133,448 SASV trials

Protocol lists are provided for both SDD (file-level bona fide/spoof labeling) and SASV ((enroll, test) pairs with target or impostor labels).

4. Evaluation Metrics and Baselines

SpoofCeleb supports precise benchmarking of SDD and SASV with the following metrics:

• SDD Metrics:
  • Equal Error Rate (EER): Operating point where false acceptance equals false rejection.
  • minDCF: Minimum Detection Cost Function; $$DCF(t) = C_{miss}P_{tar}P_{miss}(t) + C_{fa}(1-P_{tar})P_{fa}(t)$$, with $C_{miss}=C_{fa}=1$ and $P_{tar}=0.05$.
• SASV Metrics:
  • SV-EER (target vs bona fide non-target)
  • SPF-EER (target vs spoof non-target)
  • a-DCF: Architecture-agnostic DCF, combining all three trial types.
• Baseline Performance:
  • SDD (RawNet2, AASIST; zero-shot ASVspoof2019 LA):
  • RawNet2: Validation EER=56.33%, Eval EER=58.79%
  • AASIST: Validation EER=26.64%, Eval EER=23.51%
  • SDD (in-domain, SpoofCeleb train):
  • RawNet2: Validation EER=8.63%, Eval EER=1.12%
  • AASIST: Validation EER=0.61%, Eval EER=2.37%
  • SASV (SKA-TDNN, various training regimes, see paper for full details):
  • Conventional ASV: Eval SV-EER=3.84%, SPF-EER=23.44%, a-DCF=0.4923
  • SASV trained on SpoofCeleb: Eval SV-EER=12.78%, SPF-EER=5.00%, a-DCF=0.2902

These results confirm the necessity of in-domain training for robust SDD/SASV and highlight attack-specific vulnerabilities (e.g., A11 remains hardest for RawNet2 in SDD).

5. Dataset Access and Structure

SpoofCeleb is publicly available with explicit directions for usage:

• Download: https://jungjee.github.io/spoofceleb
• Directory Structure: Top-level folders A00 (bona fide) and A01–A23 (spoofed), each subdivided by speaker, containing WAV files.
• Protocols: JSON/CSV files for train/val/eval splits (SDD) and trial lists (SASV).
• Recommended Practices:
  • Always train on SpoofCeleb train fold to ensure in-domain robustness.
  • Use provided evaluation scripts and follow metric definitions strictly.

6. Research Significance and Design Insights

SpoofCeleb addresses specific deficiencies in previous SDD/SASV datasets:

• Retains full real-world variability (backgrounds, channel, noise) in both bona fide and spoofed classes.
• Supports evaluation under known, partially known, and fully unknown attack settings in a controlled fashion.
• Its speaker diversity (1,251) and nuanced attack taxonomy foster generalization beyond previous protocols that are limited in scale or diversity.
• Provides baseline implementations, split protocols, and benchmark scripts to encourage reproducible research and robust comparison across methods.

A plausible implication is that models evaluated only on studio-quality or limited-vocabulary datasets will significantly overestimate their generalization capacity under true in-the-wild attack conditions, as demonstrated by the marked performance drops observed in zero-shot baselines. SpoofCeleb’s coverage of attack-model, channel, and speaker axes makes it central for future research in both SDD and SASV (Jung et al., 2024).