SingFake: Singing Voice Deepfake Detection

• SingFake is a benchmark for singing voice deepfake detection that differentiates synthetic songs from bonafide tracks by addressing melody, rhythm, and accompaniment challenges.
• The dataset includes 28.93 hours of authentic and 29.40 hours of deepfake audio from 40 singers across five languages, curated from user-generated content.
• Empirical results demonstrate that traditional speech anti-spoofing methods fail in musical contexts, highlighting the need for singing-specific architectures and robust evaluation.

SingFake is the task and benchmark of singing voice deepfake detection: determining whether a singing clip is bonafide or deepfake under realistic musical conditions. It was introduced as the first curated in-the-wild dataset for this problem, with 28.93 hours of bonafide audio and 29.40 hours of deepfake audio from 40 singers in five languages. The benchmark was motivated by the observation that singing is not merely speech with background music added: songs contain strong accompaniment, melody and rhythm constraints, wider timbral variation, and substantial post-processing, all of which change both the signal structure and the failure modes of conventional speech countermeasures (Zang et al., 2023).

1. Task definition and problem characteristics

SingFake formalizes singing voice deepfake detection as a distinct problem from standard speech anti-spoofing. The central claim of the original benchmark is that synthesized singing differs from synthesized speech in several ways that materially affect detection: songs are usually released with strong background music; singing is constrained by melody, rhythm, note duration, pitch control, and vibrato; lyrics may be elongated or repeated; and musical production adds effects and mastering that can hide or distort the artifacts on which speech-trained detectors often rely (Zang et al., 2023).

This framing was later reinforced by follow-up work. “Singing Voice Graph Modeling for SingFake Detection” states that singing must be modeled jointly through vocal/linguistic information, musical information such as pitch and rhythm, and spectral-temporal spoofing artifacts, while the SVDD 2024 challenge paper describes SingFake as the groundwork for the broader area of singing voice deepfake detection (SVDD) and as the template for an in-the-wild challenge track with background music and web-scale variability (Chen et al., 2024, Zhang et al., 2024).

A recurrent misconception in the literature is that SingFake can be handled by reusing speech countermeasures with minimal adaptation. The benchmark results and later analyses reject that view. In the original paper, background accompaniment is treated as a major obstacle; in later work, accompaniment is shown to help some systems mainly through regularization and robustness rather than through direct use of intrinsic musical cues such as rhythm or harmony (Zang et al., 2023, Chen et al., 18 Sep 2025).

2. Dataset composition, curation, and protocol design

The original SingFake corpus was collected from popular user-generated content platforms and pairs deepfake songs with corresponding bonafide songs from the original singers. The dataset contains 634 bonafide songs and 671 deepfake songs, with an average clip length of 13.75 seconds. The curation pipeline includes manual metadata annotation for singer identity, language, website source, deepfake label, and generation model when available; GPT-4 was used to help verify metadata against song titles and descriptions, followed by manual review of discrepancies (Zang et al., 2023).

Audio preprocessing was designed around singing rather than generic audio spoofing. Demucs was used for source separation, PyAnnote VAD for active singing-region segmentation, clips were resampled to 16 kHz, and stereo audio was preserved with one channel randomly selected during training. The benchmark provides both song mixtures and separated vocals, allowing direct study of accompaniment interference versus vocal-only evidence (Zang et al., 2023).

The split structure is central to the benchmark because it probes distinct forms of generalization rather than only in-domain accuracy.

Split	# Singers	Clips (Real / Fake)
Train	12	5251 / 4519
Validation	4	1089 / 543
T01	1	370 / 1208
T02	6	1685 / 1006
T03	6	6740 / 4024
T04	17	353 / 166

The test scenarios are deliberately heterogeneous. T01 is a seen-singer test set for Stefanie Sun; T02 evaluates unseen singers; T03 applies 4 codecs—MP3 128 Kbps, AAC 64 Kbps, OPUS 64 Kbps, and Vorbis 64 Kbps—to T02; and T04 is a Persian musical context set intended to probe language and style shift (Zang et al., 2023).

3. Baseline behavior and empirical difficulty

The original benchmark evaluates four state-of-the-art speech countermeasure systems: AASIST, Spectrogram + ResNet, LFCC + ResNet, and Wav2Vec2 + AASIST. When these systems are trained on ASVspoof2019 LA and then tested on SingFake, their performance collapses. On ASVspoof2019 LA Eval, EER ranges from 0.83 to 7.03; on SingFake-T02 Mixture, the same systems reach 58.12, 51.87, 45.12, and 56.75 EER; on SingFake-T02 Vocals, they reach 37.91, 37.65, 54.88, and 57.26. The result is that systems strong on speech become near-random or severely degraded on singing (Zang et al., 2023).

Training the same architectures on SingFake substantially improves performance, but the difficulty remains highly nonuniform across test conditions. Wav2Vec2 + AASIST is the strongest baseline overall. Under Mixture training it reports 1.57 EER on Train, 4.62 on T01, 8.23 on T02, 13.62 on T03, and 42.77 on T04. Under Vocals training it reports 1.70, 5.39, 9.10, 10.03, and 42.19 on the same splits. The consistent pattern is that T02 is harder than T01, codec shift in T03 is challenging but not dominant, and T04 is the hardest setting overall (Zang et al., 2023).

These results established the main research agenda for SingFake. First, speech-trained countermeasures do not transfer well to singing. Second, background music severely degrades performance. Third, training on singing data helps, but does not solve robustness to new singers, new languages, or new musical contexts. The sharp T04 degradation also suggests that current systems are not genre-agnostic and remain vulnerable to culturally or stylistically shifted material (Zang et al., 2023).

4. Architecture development and the role of accompaniment

A major line of work on SingFake has been the design of singing-specific architectures. SingGraph combines MERT-v1-330M as an instrumental backbone, wav2vec 2.0 XLS-R as a vocal backbone, and a graph-based backend built from a RawNet2 encoder, heterogeneous stacking graph attention layers (HS-GAL), a stack node, and a Max Graph Operation (MGO). The training framework separates each song with Demucs into instrumental and vocal streams, applies RawBoost to vocals and beat matching to instrumentals, and operates in an IV setting with separate instrumental and vocal input. In this setting, SingGraph reports 0.54% EER on Train, 4.01% on T01, 6.23% on T02, and 6.30% on T03, corresponding to relative EER reductions of 65.6%, 13.2%, 24.3%, and 37.1% over the prior best W2V2-AASIST baseline (Chen et al., 2024).

The role of accompaniment was examined more directly in “How Does Instrumental Music Help SingFake Detection?” That study evaluates Spec-ResNet, AASIST, W2V2-AASIST, and SingGraph on the T01 split, comparing Vocal-Only and Vocal-Inst. inputs after separation with Demucs and waveform augmentation with RawBoost. The reported EER changes are 11.78% → 10.82% for Spec-ResNet, 6.78% → 6.28% for AASIST, 5.24% → 5.51% for W2V2-AASIST, and 5.24% → 3.77% for SingGraph. The gain is therefore architecture-dependent rather than universal (Chen et al., 18 Sep 2025).

The same paper shows that unpaired music or even noise can match or sometimes outperform the original paired accompaniment, including under test-time perturbations using music, speech, noise, or reverberation from MUSAN. A subband study with 4th-order Butterworth band-stop filters identifies 0–0.8 kHz as the most important band: masking that range causes a large EER increase, especially for SingGraph and W2V2-AASIST, while masking the same bands in the instrumental track has little effect. Representational probes using SUPERB and MARBLE further show that fine-tuning increases reliance on SID and ASV, reduces PR and ASR by up to 40%, and degrades music tasks such as Key Detection (KD). The paper’s central conclusion is that instrumental accompaniment acts mainly as data augmentation rather than as a direct source of rhythm-, harmony-, or melody-based authenticity evidence (Chen et al., 18 Sep 2025).

5. Expansion into challenges and successor benchmarks

SingFake became the basis for the WildSVDD track in the SVDD 2024 challenge. That paper states that WildSVDD “follows the same approach as our SingFake project,” is expanded to approximately double the original size of SingFake, and contains 97 singers, 2007 deepfake songs, and 1216 bonafide songs. It preserves the in-the-wild setting with background music, social-media sourcing, unknown platform codecs, and realistic web variability. The track defines Test A as a newly formed test set and Test B as the hardest inherited split, corresponding to T04 from the original SingFake work (Zhang et al., 2024).

The challenge also standardized several procedural points. Evaluation is conducted at the segment level rather than at the song level, participants receive training and test partitions and may carve out their own validation data, and the main metric is Equal Error Rate (EER). No external submissions were received for WildSVDD, so the paper reports baseline systems only. Among those baselines, Wav2vec2 XLS-R achieves 6.09% EER on Test A vocals, while the Raw Waveform baseline reports 8.48% on Test A vocals and 14.91% on Test B vocals; classical spectral front ends such as spectrograms, mel-spectrograms, MFCC, and LFCC are markedly weaker (Zhang et al., 2024).

Later work broadened the benchmark regime further. SingFox introduces a multilingual singfake corpus with 113,802 audio clips across 20 languages, 126.32 hours of audio, and 1,150 singers, organized into six tracks (T1–T6) for multilingual detection, genre and instrument diversity, alternative fakes, and source tracing / source verification. The dataset includes 54,911 real clips, 37,953 fake clips, and 10,469 alternative-fake clips. In cross-dataset evaluation, the highest reported accuracy on SingFox (T4) is 77.84%, obtained by a model trained on FMC (Shah et al., 17 Jun 2026).

This progression suggests that SingFake has functioned as the foundational in-the-wild benchmark from which later SVDD evaluations inherited their emphasis on unseen singers, codecs, language shifts, background accompaniment, and increasingly, open-set and source-attribution settings.

6. SingFake as a transfer, robustness, and forensic testbed

SingFake is now used not only for in-domain binary detection but also for cross-domain robustness studies. The paper “Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speech” evaluates SIGNAL—a framework combining speech foundation models, a GNN branch, and a KNN branch—by training on DiffSSD and testing on SingFake in a zero-shot setting, without any fine-tuning. In that paper, SingFake is explicitly treated as a different domain from speech TTS forensics, with 28.93 hours of bonafide and 29.40 hours of deepfake clips, five languages, and 40 singers (Akhtar et al., 11 Jan 2026).

Under this transfer setting, the best reported SingFake results come from MAMBA-B + KNN+GNN (SIGNAL), with DEV performance of 92.28% ACC, 90.99% F1, and 3.81% EER, and ID/OOD performance of 79.66% ACC, 78.55% F1, and 7.92% EER. The paper interprets this as evidence that Mamba-based embeddings, graph reasoning over generator prototypes, and instance-level similarity can jointly improve cross-domain singfake robustness (Akhtar et al., 11 Jan 2026).

The same broadening appears in high-resolution SVDD work. “Joint Fullband-Subband Modeling for High-Resolution SingFake Detection” argues that conventional 16 kHz detectors discard high-frequency information important for singing and proposes Sing-HiResNet on 44.1 kHz audio for WildSVDD. Its reported best variants achieve relative EER reductions of 31.6% on Test A and 30.9% on Test B over a reimplemented UNIBS baseline, supporting the claim that high-frequency subbands provide complementary cues for in-the-wild singing detection (Chen et al., 6 Apr 2026).

Taken together, these developments place SingFake at the center of several research threads: dedicated singing-specific detection architectures, explicit analysis of accompaniment and vocal evidence, cross-dataset robustness, open-set detection, source attribution, and high-resolution acoustic modeling. The benchmark’s original observation remains intact throughout this literature: singing voice deepfake detection is a separate forensic problem whose difficulty is driven not only by synthesis quality, but also by music mixing, speaker novelty, language and genre shift, and the evolving diversity of singing generation systems (Zang et al., 2023).