---
title: 'SingFake: Singing Voice Deepfake Detection'
url: https://www.emergentmind.com/topics/singfake
type: topic
---

# SingFake: Singing Voice Deepfake Detection

SingFake is the task and benchmark of **singing voice deepfake detection**: determining whether a singing clip is **bonafide** or **deepfake** under realistic musical conditions. It was introduced as the first curated **in-the-wild** dataset for this problem, with **28.93 hours** of bonafide audio and **29.40 hours** of deepfake audio from **40 singers** in **five languages**. The benchmark was motivated by the observation that singing is not merely speech with background music added: songs contain strong accompaniment, melody and rhythm constraints, wider timbral variation, and substantial post-processing, all of which change both the signal structure and the failure modes of conventional speech countermeasures [2309.07525].

## 1. Task definition and problem characteristics

SingFake formalizes singing voice deepfake detection as a distinct problem from standard speech anti-spoofing. The central claim of the original benchmark is that synthesized singing differs from synthesized speech in several ways that materially affect detection: songs are usually released with **strong background music**; singing is constrained by **melody, rhythm, note duration, pitch control, and vibrato**; lyrics may be elongated or repeated; and musical production adds effects and mastering that can hide or distort the artifacts on which speech-trained detectors often rely [2309.07525].

This framing was later reinforced by follow-up work. “Singing Voice Graph Modeling for SingFake Detection” states that singing must be modeled jointly through **vocal/linguistic information**, **musical information** such as **pitch and rhythm**, and **spectral-temporal spoofing artifacts**, while the SVDD 2024 challenge paper describes SingFake as the groundwork for the broader area of **singing voice deepfake detection (SVDD)** and as the template for an in-the-wild challenge track with background music and web-scale variability [2406.03111] [2408.16132].

A recurrent misconception in the literature is that SingFake can be handled by reusing speech countermeasures with minimal adaptation. The benchmark results and later analyses reject that view. In the original paper, background accompaniment is treated as a major obstacle; in later work, accompaniment is shown to help some systems mainly through **regularization and robustness** rather than through direct use of intrinsic musical cues such as rhythm or harmony [2309.07525] [2509.14675].

## 2. Dataset composition, curation, and protocol design

The original SingFake corpus was collected from **popular user-generated content platforms** and pairs deepfake songs with corresponding bonafide songs from the original singers. The dataset contains **634 bonafide songs** and **671 deepfake songs**, with an average clip length of **13.75 seconds**. The curation pipeline includes manual metadata annotation for singer identity, language, website source, deepfake label, and generation model when available; **GPT-4** was used to help verify metadata against song titles and descriptions, followed by manual review of discrepancies [2309.07525].

Audio preprocessing was designed around singing rather than generic audio spoofing. **Demucs** was used for source separation, **PyAnnote VAD** for active singing-region segmentation, clips were resampled to **16 kHz**, and stereo audio was preserved with one channel randomly selected during training. The benchmark provides both **song mixtures** and **separated vocals**, allowing direct study of accompaniment interference versus vocal-only evidence [2309.07525].

The split structure is central to the benchmark because it probes distinct forms of generalization rather than only in-domain accuracy.

| Split | # Singers | Clips (Real / Fake) |
|---|---:|---:|
| Train | 12 | 5251 / 4519 |
| Validation | 4 | 1089 / 543 |
| T01 | 1 | 370 / 1208 |
| T02 | 6 | 1685 / 1006 |
| T03 | 6 | 6740 / 4024 |
| T04 | 17 | 353 / 166 |

The test scenarios are deliberately heterogeneous. **T01** is a seen-singer test set for **Stefanie Sun**; **T02** evaluates **unseen singers**; **T03** applies **4 codecs**—**MP3 128 Kbps**, **AAC 64 Kbps**, **OPUS 64 Kbps**, and **Vorbis 64 Kbps**—to T02; and **T04** is a **Persian musical context** set intended to probe language and style shift [2309.07525].

## 3. Baseline behavior and empirical difficulty

The original benchmark evaluates four state-of-the-art speech countermeasure systems: **AASIST**, **Spectrogram + ResNet**, **LFCC + ResNet**, and **Wav2Vec2 + AASIST**. When these systems are trained on **ASVspoof2019 LA** and then tested on SingFake, their performance collapses. On **ASVspoof2019 LA Eval**, EER ranges from **0.83** to **7.03**; on **SingFake-T02 Mixture**, the same systems reach **58.12**, **51.87**, **45.12**, and **56.75** EER; on **SingFake-T02 Vocals**, they reach **37.91**, **37.65**, **54.88**, and **57.26**. The result is that systems strong on speech become near-random or severely degraded on singing [2309.07525].

Training the same architectures on SingFake substantially improves performance, but the difficulty remains highly nonuniform across test conditions. **Wav2Vec2 + AASIST** is the strongest baseline overall. Under **Mixture** training it reports **1.57** EER on Train, **4.62** on T01, **8.23** on T02, **13.62** on T03, and **42.77** on T04. Under **Vocals** training it reports **1.70**, **5.39**, **9.10**, **10.03**, and **42.19** on the same splits. The consistent pattern is that **T02 is harder than T01**, codec shift in **T03** is challenging but not dominant, and **T04 is the hardest setting overall** [2309.07525].

These results established the main research agenda for SingFake. First, **speech-trained countermeasures do not transfer well to singing**. Second, **background music severely degrades performance**. Third, **training on singing data helps**, but does not solve robustness to new singers, new languages, or new musical contexts. The sharp T04 degradation also suggests that current systems are not genre-agnostic and remain vulnerable to culturally or stylistically shifted material [2309.07525].

## 4. Architecture development and the role of accompaniment

A major line of work on SingFake has been the design of singing-specific architectures. **SingGraph** combines **MERT-v1-330M** as an instrumental backbone, **wav2vec 2.0 XLS-R** as a vocal backbone, and a graph-based backend built from a **RawNet2 encoder**, **heterogeneous stacking graph attention layers (HS-GAL)**, a **stack node**, and a **Max Graph Operation (MGO)**. The training framework separates each song with **Demucs** into instrumental and vocal streams, applies **RawBoost** to vocals and **beat matching** to instrumentals, and operates in an **IV** setting with separate instrumental and vocal input. In this setting, SingGraph reports **0.54%** EER on Train, **4.01%** on **T01**, **6.23%** on **T02**, and **6.30%** on **T03**, corresponding to relative EER reductions of **65.6%**, **13.2%**, **24.3%**, and **37.1%** over the prior best **W2V2-AASIST** baseline [2406.03111].

The role of accompaniment was examined more directly in “How Does Instrumental Music Help SingFake Detection?” That study evaluates **Spec-ResNet**, **AASIST**, **W2V2-AASIST**, and **SingGraph** on the **T01** split, comparing **Vocal-Only** and **Vocal-Inst.** inputs after separation with **Demucs** and waveform augmentation with **RawBoost**. The reported EER changes are **11.78% → 10.82%** for Spec-ResNet, **6.78% → 6.28%** for AASIST, **5.24% → 5.51%** for W2V2-AASIST, and **5.24% → 3.77%** for SingGraph. The gain is therefore architecture-dependent rather than universal [2509.14675].

The same paper shows that **unpaired music or even noise can match or sometimes outperform the original paired accompaniment**, including under test-time perturbations using **music**, **speech**, **noise**, or **reverberation** from **MUSAN**. A subband study with **4th-order Butterworth band-stop filters** identifies **0–0.8 kHz** as the most important band: masking that range causes a large EER increase, especially for **SingGraph** and **W2V2-AASIST**, while masking the same bands in the instrumental track has little effect. Representational probes using **SUPERB** and **MARBLE** further show that fine-tuning increases reliance on **SID** and **ASV**, reduces **PR** and **ASR** by **up to 40%**, and degrades music tasks such as **Key Detection (KD)**. The paper’s central conclusion is that instrumental accompaniment acts mainly as **data augmentation** rather than as a direct source of rhythm-, harmony-, or melody-based authenticity evidence [2509.14675].

## 5. Expansion into challenges and successor benchmarks

SingFake became the basis for the **WildSVDD** track in the **SVDD 2024** challenge. That paper states that WildSVDD “follows the same approach as our SingFake project,” is expanded to **approximately double the original size of SingFake**, and contains **97 singers**, **2007 deepfake songs**, and **1216 bonafide songs**. It preserves the in-the-wild setting with background music, social-media sourcing, unknown platform codecs, and realistic web variability. The track defines **Test A** as a newly formed test set and **Test B** as the hardest inherited split, corresponding to **T04** from the original SingFake work [2408.16132].

The challenge also standardized several procedural points. Evaluation is conducted at the **segment level rather than at the song level**, participants receive training and test partitions and may carve out their own validation data, and the main metric is **Equal Error Rate (EER)**. No external submissions were received for WildSVDD, so the paper reports baseline systems only. Among those baselines, **Wav2vec2 XLS-R** achieves **6.09% EER** on **Test A vocals**, while the **Raw Waveform** baseline reports **8.48%** on Test A vocals and **14.91%** on **Test B vocals**; classical spectral front ends such as spectrograms, mel-spectrograms, MFCC, and LFCC are markedly weaker [2408.16132].

Later work broadened the benchmark regime further. **SingFox** introduces a multilingual singfake corpus with **113,802 audio clips** across **20 languages**, **126.32 hours** of audio, and **1,150 singers**, organized into **six tracks (T1–T6)** for multilingual detection, genre and instrument diversity, alternative fakes, and **source tracing / source verification**. The dataset includes **54,911 real clips**, **37,953 fake clips**, and **10,469 alternative-fake clips**. In cross-dataset evaluation, the highest reported accuracy on **SingFox (T4)** is **77.84%**, obtained by a model trained on **FMC** [2606.18985].

This progression suggests that SingFake has functioned as the foundational in-the-wild benchmark from which later SVDD evaluations inherited their emphasis on **unseen singers**, **codecs**, **language shifts**, **background accompaniment**, and increasingly, **open-set** and **source-attribution** settings.

## 6. SingFake as a transfer, robustness, and forensic testbed

SingFake is now used not only for in-domain binary detection but also for cross-domain robustness studies. The paper “Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speech” evaluates **SIGNAL**—a framework combining speech foundation models, a **GNN** branch, and a **KNN** branch—by training on **DiffSSD** and testing on SingFake in a **zero-shot setting**, **without any fine-tuning**. In that paper, SingFake is explicitly treated as a different domain from speech TTS forensics, with **28.93 hours of bonafide** and **29.40 hours of deepfake** clips, **five languages**, and **40 singers** [2601.07064].

Under this transfer setting, the best reported SingFake results come from **MAMBA-B + KNN+GNN (SIGNAL)**, with **DEV** performance of **92.28% ACC**, **90.99% F1**, and **3.81% EER**, and **ID/OOD** performance of **79.66% ACC**, **78.55% F1**, and **7.92% EER**. The paper interprets this as evidence that **Mamba-based embeddings**, graph reasoning over generator prototypes, and instance-level similarity can jointly improve cross-domain singfake robustness [2601.07064].

The same broadening appears in high-resolution SVDD work. “Joint Fullband-Subband Modeling for High-Resolution SingFake Detection” argues that conventional **16 kHz** detectors discard high-frequency information important for singing and proposes **Sing-HiResNet** on **44.1 kHz** audio for **WildSVDD**. Its reported best variants achieve relative EER reductions of **31.6%** on **Test A** and **30.9%** on **Test B** over a reimplemented **UNIBS** baseline, supporting the claim that high-frequency subbands provide complementary cues for in-the-wild singing detection [2604.04841].

Taken together, these developments place SingFake at the center of several research threads: dedicated singing-specific detection architectures, explicit analysis of accompaniment and vocal evidence, cross-dataset robustness, open-set detection, source attribution, and high-resolution acoustic modeling. The benchmark’s original observation remains intact throughout this literature: singing voice deepfake detection is a separate forensic problem whose difficulty is driven not only by synthesis quality, but also by **music mixing**, **speaker novelty**, **language and genre shift**, and the evolving diversity of singing generation systems [2309.07525].

Source: https://www.emergentmind.com/topics/singfake