FakeMusicCaps: AI Music Forensics Dataset

• FakeMusicCaps is a dataset that pairs real music clips with AI-generated counterparts using standardized MusicCaps captions for forensic analysis.
• It employs prompt-aligned 10-second audio tracks to benchmark both closed-set and open-set detection and attribution methods across multiple TTM models.
• Empirical results reveal near-perfect closed-set performance yet significant challenges in open-set detection, prompting further research in music forensics.

FakeMusicCaps is a dataset for the detection and attribution of synthetic music generated via Text-to-Music (TTM) models. Introduced as a resource built by re-generating the music-caption pairs of MusicCaps with several state-of-the-art TTM techniques, it formalizes two central forensic tasks: distinguishing real music from generated music, and attributing a generated clip to its source model. In subsequent work, it has also become a benchmark for AI-generated music detection, especially for 10-second clip classification, cross-generator evaluation, and zero-shot detection settings (Comanducci et al., 2024).

1. Origin, scope, and forensic motivation

The FakeMusicCaps dataset was proposed in response to the rapid diffusion of TTM systems and the associated concerns regarding copyright violation and rightful attribution. The original study frames the problem in explicitly forensic terms: TTM models had reached superior performances to previous state-of-the-art systems, had lowered the technical proficiency needed for music generation, and had already started to be adopted for commercial uses and music production practices (Comanducci et al., 2024).

Its construction is anchored in MusicCaps. The original paper describes MusicCaps as containing 5,500 pairs of 10-second real music clips from AudioSet and professional musician-written captions. FakeMusicCaps reuses those captions as prompts for TTM generation, thereby creating a paired benchmark in which the textual conditioning is held constant while the audio provenance varies. This design makes the dataset suitable not only for binary “real versus fake” detection, but also for source-model attribution, since different generators can be compared under matched prompt conditions (Comanducci et al., 2024).

Later benchmark papers describe FakeMusicCaps as containing 10-second audio clips generated by five different TTM models together with real music recordings; one such description reports 5,373 real tracks and 27,605 AI-generated tracks. Those later descriptions emphasize the dataset’s diversity across generative models and conditioning artifacts, which is one reason it has been repeatedly used for generalization studies rather than only for closed-set classification (Kim et al., 10 Sep 2025).

2. Dataset construction and content

The original FakeMusicCaps paper states that, for every MusicCaps caption, five state-of-the-art TTM models were used to generate a corresponding synthetic audio clip. All generated tracks and original tracks are standardized as mono, 16 kHz, float32 PCM WAV files. The paper also characterizes the dataset as extensible: new TTM models can be added in a plug-and-play fashion using the same prompt set (Comanducci et al., 2024).

The five TTM systems used in the initial release are described as follows:

TTM model	Description
MusicGen	Autoregressive LLM using Encodec tokens
MusicLDM	Latent diffusion conditioned on CLAP
AudioLDM2	Latent diffusion with VAE + HiFiGAN
Stable Audio Open	Latent diffusion with T5 text encoder
Mustango	Diffusion model with music-knowledge-informed UNet

This composition is important methodologically. Because each synthetic clip is generated from a caption drawn from the same caption distribution as the real reference set, FakeMusicCaps is not merely a collection of AI music clips; it is a prompt-aligned benchmark for studying how generator-specific artifacts appear under common textual prompts. A plausible implication is that the dataset is particularly well suited to attribution settings in which the classifier must discriminate not just “real” from “fake,” but also among multiple synthetic provenance classes (Comanducci et al., 2024).

3. Detection, attribution, and evaluation protocols

The original paper defines two tasks. Detection is binary classification between real and TTM-generated music. Attribution is multi-class classification among real music and the known TTM generators. It further distinguishes closed-set evaluation, in which all classes are seen during training and test, from open-set evaluation, in which some generators are unseen during training and appear only at test time. For the open-set case, Suno-generated tracks from the SunoCaps dataset are used as “unknown” examples (Comanducci et al., 2024).

Three baseline neural architectures are reported: M5, a CNN on raw waveforms; RawNet2, an end-to-end anti-spoofing model operating on raw audio; and ResNet18 + Spec, a ResNet18 adapted to single-channel log-spectrogram input. Training uses cross-entropy loss, the Adam optimizer with learning rate $1 \times 10^{-4}$, and early stopping after 10 epochs without loss improvement, up to a maximum of 100 epochs. The closed-set formulation uses standard softmax classification, while the open-set experiments evaluate two strategies: a thresholding method based on the ratio of the top two softmax scores with $\theta = 2$, and a one-class SVM on softmax vectors (Comanducci et al., 2024).

The evaluation metrics include balanced accuracy, precision, recall, and F1 score. The balanced accuracy is given as

$$\text{ACC}_B = \frac{1}{C}\sum_{i=1}^C \frac{TP_i}{TP_i + FN_i},$$

where $C$ is the number of classes. In the later MusicDET work, Equal Error Rate (EER) becomes the primary metric for cross-generator detection experiments, reflecting a shift from attribution-style multiclass reporting toward detector robustness under unseen generators (Comanducci et al., 2024, Han et al., 18 May 2026).

4. Empirical results and subsequent benchmark use

In the original closed-set attribution experiments, all three baselines perform strongly, with ResNet18 + Spec reaching 1.00 balanced accuracy, 1.00 precision, 1.00 recall, and 1.00 F1 score. The open-set setting is substantially harder. Under thresholding, ResNet18 + Spec reaches 0.85 balanced accuracy, 0.78 precision, 0.85 recall, and 0.8 F1 score; under the SVM approach, the same model drops to 0.48 balanced accuracy, 0.80 precision, 0.48 recall, and 0.56 F1 score. The paper reports that commercial models not included during training are particularly challenging, and that highly realistic unknown syntheses are often confused with real music or with the closest known TTM class (Comanducci et al., 2024).

Subsequent papers have used FakeMusicCaps as a benchmark for increasingly different detector designs:

Paper	Use of FakeMusicCaps	Reported result
"Segment Transformer: AI-Generated Music Detection via Music Structural Analysis" (Kim et al., 10 Sep 2025)	Segment-level short-audio detection	MERT: Accuracy 0.994, F1 0.996, AUC 0.998
"MusicDET: Zero-Shot AI-Generated Music Detection" (Han et al., 18 May 2026)	Zero-shot, real-only training with cross-generator evaluation	MusicDET average EER 4.51; class-conditional variant 0.89
"Detecting Musical Deepfakes" (Sunday, 3 May 2025)	Mel-spectrogram CNN under pitch and tempo manipulation	Baseline Accuracy 0.885, F1 0.878

These later studies shift the emphasis from closed-set attribution toward generator-agnostic detection. The Segment Transformer paper uses an 80/10/10 split and evaluates segment-level detectors on 10-second clips, reporting that MERT outperforms Wav2vec 2.0, Music2vec, FXencoder, and a MobileNet baseline on FakeMusicCaps (Kim et al., 10 Sep 2025). MusicDET goes further by formulating a zero-shot setting in which the detector is trained exclusively on real music and evaluated on music produced by all five generators; its reported average EER of 4.51% markedly improves over conventional discriminative baselines in that setting (Han et al., 18 May 2026). A separate ResNet18 study applies pitch shifting in $[-2, +2]$ semitones and tempo scaling in $[0.8, 1.2]$, finding that performance degrades modestly rather than catastrophically under these manipulations (Sunday, 3 May 2025).

5. Relation to adjacent datasets and terminological ambiguities

FakeMusicCaps occupies a specific niche within audio forensics. A useful contrast is with FSD, which is described as a dataset for fake song detection in the singing-voice domain. That comparison states explicitly that FakeMusicCaps focuses on text-to-music generation for music deepfake detection and is music, not singing, oriented, whereas FSD targets singing voice deepfakes mixed with real accompaniments and is therefore more directly aligned with singer impersonation (Xie et al., 2023).

The dataset has also been absorbed into broader mixed-audio benchmarks. MixFake includes music backgrounds drawn from FMA-Medium, Sonics, and FakeMusicCaps, and recombines foreground speech and background music or noise in all authentic/fake permutations at SNR levels sampled from $\{-5, 0, 5, 10, 15, 20\}$ dB. In that setting, FakeMusicCaps is no longer the sole object of study, but a component of a more realistic mixed-source benchmark for background authenticity detection (Li et al., 22 May 2026).

The term “FakeMusicCaps” has not always been used exclusively to denote the 2024 dataset. In the caption-poisoning literature, attacker-injected, LLM-generated adversarial caption entries are described as yielding highly plausible “FakeMusicCaps” data (Wen et al., 18 May 2026). In multimodal image-to-music work, the phrase “FakeMusicCaps” risk is used to denote the possibility that pseudo-aligned or spurious pairs can mislead models and evaluations into overstating cross-modal feeling alignment (Hong et al., 27 Nov 2025). These usages are conceptually distinct from the dataset introduced for detection and attribution of synthetic music generated via TTM models.

6. Limitations and open research directions

The original FakeMusicCaps paper presents the dataset and experiments explicitly as a baseline rather than as a solved benchmark. The reported limitations are concrete: only open-source TTM models are included for paired generation; commercial systems are used only as “unknown” in open-set tests; the baselines are simple off-the-shelf networks; the paired data setting may not represent the full diversity of real-world music or TTM behavior; and the study does not yet examine how caption–music dependence, musicology, or music theory might improve detection (Comanducci et al., 2024).

Its empirical results already indicate the main unresolved problem. Closed-set attribution appears straightforward with current models and data, but open-set detection and attribution remain substantially more difficult, especially for realistic commercial generators. This has motivated later work to emphasize cross-generator generalization and detector designs that do not rely on seeing fake examples from the target generator during training (Comanducci et al., 2024, Han et al., 18 May 2026).

The future directions stated in the original work are correspondingly broad: extending the dataset to additional TTM models, studying the effect of textual descriptions on detection accuracy, leveraging caption–music pairing in attribution, incorporating musicology or music theory, exploring partial fakes and splicing, and benchmarking stronger forensic architectures (Comanducci et al., 2024). Taken together with later segment-level and zero-shot studies, this suggests that FakeMusicCaps functions less as a finished benchmark than as an initial standardized substrate for research on music-domain forensic generalization, model attribution, and authenticity verification under rapidly changing generative conditions (Kim et al., 10 Sep 2025, Han et al., 18 May 2026).