Melody or Machine (MoM): Human vs. Synthetic Music

• MoM is a research focus that distinguishes human-composed music from synthetic outputs using binary classification and robust benchmarks.
• Benchmark datasets include over 130,000 tracks from diverse sources, tested with both symbolic and waveform detection architectures.
• Advanced methods like CLAM use dual-stream encoders with contrastive triplet loss to detect subtle misalignments, achieving over 92% F1 accuracy.

Melody or Machine (MoM) refers to the task, benchmarks, and methods for distinguishing human-composed music from synthetic (machine-generated) music, as well as the broader scientific frameworks for quantifying and operationalizing the boundary between “melody” (understood as a proxy for human musical composition) and “machine” (automated music synthesis). MoM has emerged as a central problem at the intersection of music information retrieval, artificial intelligence, and computational creativity, driven by the rapid advancements in deep generative models for music and the associated challenges in detection, evaluation, and authentic authorship attribution.

1. MoM Task Definition and Motivations

The core MoM task is binary classification. Given an audio signal or symbolic melody excerpt $x$, determine the label $y \in \{0,1\}$, where $y=1$ denotes “human-composed” and $y=0$ denotes “machine-generated.” Input formats include both full-length songs (waveforms) and symbolic representations (MIDI sequences), depending on the experimental setting (Batra et al., 29 Nov 2025, Li et al., 2020).

Early MoM work focused on the need for robust, scalable detection systems as generative models—including state-of-the-art text-to-music and waveform synthesis pipelines—began producing outputs virtually indistinguishable from human compositions. Assessing plagiarism risk, establishing artistic authenticity, and enforcing copyright hinge on this detection capability. However, there is no standardized, widely adopted metric, and prior models often failed to generalize beyond the training generator’s artifacts (Batra et al., 29 Nov 2025).

2. MoM Benchmarks and Datasets

The introduction of the Melody or Machine (MoM) Benchmark addresses the lack of large, diverse, and challenging testbeds for detection research. The MoM dataset (Batra et al., 29 Nov 2025) comprises:

• $130,435$ full-length tracks (65,475 real/human, 64,960 synthetic/machine)
• $6,665$ hours total, average 196 sec per track
• Mixed sources: both open-source (Diffrythm, Yue) and closed/proprietary (Suno v2-v4, Udio, Riffusion, commercial voice-cloning) generation pipelines
• Explicitly constructed out-of-distribution (OOD) test splits, featuring generators and manipulations not present in training
• English-language bias (82% English content); limitations in coverage of other linguistic and instrumental traditions

Preprocessing for detection models standardizes audio to 90 seconds (trimmed/padded, 24 kHz SR). Symbolic datasets, as in “Melody Classifier with Stacked-LSTM” (Li et al., 2020), focus on monophonic, quantized MIDI representations, with moderated genre and stylistic coverage.

3. Detection Architectures and Methodologies

MoM detection models can be grouped into two lineages: event-level symbolic sequence classifiers, and waveform-level multimodal forensics.

Symbolic MoM Classifiers

The stacked-LSTM classifier (Li et al., 2020) models a melody as a sequence of note events $x = \{(pitch_t, pos_t, dur_t)\}_{t=1}^T$, mapping to label $y$ via

$$\hat{y} = \sigma(\mathrm{FC}(\mathrm{LSTM}_2(\mathrm{LSTM}_1(\mathrm{Encode}(x)))))$$

where Encode$(x)$ produces concatenated one-hot vectors over pitch, position, and duration. Regularization employs dropout (rate $0.4$) after each LSTM layer. Loss is binary cross-entropy.

Dual-Stream Contrastive Learning

CLAM (Cross-modality Alignment Model) (Batra et al., 29 Nov 2025) exemplifies current SOTA waveform forensics:

• Parallel pre-trained encoders:
  • Music-centric (MERT)
  • Vocal/timbral-centric (Wav2Vec2)
• Weighted cross-aggregation (learnable 1D convolution) collapses per-layer encoder outputs.
• Intra-stream self-attention models temporal and contextual dependencies.
• Cross-aggregation fuses modality-specific embeddings to a joint latent $\mathbf{h}$:

$$\mathbf{h} = \sigma\left( W_m \mathbf{d}_m + W_v \mathbf{d}_v + b \right)$$

• Binary cross-entropy loss and contrastive triplet loss: encourages intra-song coherence between vocal and instrumental streams, exposes machine-generated subtle misalignments.
• No explicit source separation; operates on the stereo mixture.

4. Evaluation Metrics and Results

Primary evaluation metric is the F1 score (harmonic mean of precision and recall). Statistical significance is established via two-sided McNemar’s tests (Batra et al., 29 Nov 2025). Key results include:

Model	Accuracy (%)	F1 (%)
SpecTTTra-α (Prev. SOTA)	87.2	86.9
MiO	88.4	87.2
Poin-HierNet	90.3	89.6
CLAM (Triplet Loss)	93.1	92.5

CLAM with triplet alignment loss outperforms all competitors, including major previous baselines, with improvements confirmed significant at $p<0.001$.

Out-of-distribution (OOD) evaluation demonstrates the crucial MoM dataset property: while earlier models degrade to F1 50–68% on OOD generators, CLAM maintains $>$90% F1. This is attributable to architectural sensitivity to cross-source dependencies and robust dataset design.

In symbolic classification, quantitative results for small-scale LSTM-based detection models are not explicitly reported, but effective discrimination is claimed (Li et al., 2020).

5. Underlying Machine-Human Discriminants

CLAM empirically validates that real music exhibits subtle but persistent dependencies between vocal timbre, instrumental context, pitch, and harmony—modeled as a joint distribution $P(V,M)$. Machine-generated audio often suffers from independent modeling of vocal and instrumental elements, resulting in detectable timing, rhythmic, or timbral misalignments ($\hat{P}(V,M) \approx P(V)P(M)$). Contrastive training is effective for exposing and exploiting these synthetic artifacts, making dual-branch architectures with learned alignment losses the preferred current strategy (Batra et al., 29 Nov 2025).

Symbolic classifiers target note-level statistics and learned sequential patterns, but are limited by feature scope (e.g., absence of dynamics or articulation information), data set size, and input domain (monophonic vs. polyphonic).

6. Broader Context: MoM in Generative Evaluation and Music Creation

MoM arises in generative model evaluation as both a test (“musical Turing test”) and a tool for understanding the limits of machine composition. Human–machine indistinguishability is central to the design and public acceptance of text-to-music pipelines (see (Wei et al., 30 Sep 2024, Dai et al., 2021)). Recent generative models, such as MG², explicitly incorporate both “implicit” and “explicit” melodic guidance, showing that the musical boundary between “melody” and “machine” is technically permeable: generated tracks are often rated as indistinguishable from human compositions by listeners (Wei et al., 30 Sep 2024).

MoM detection systems provide tools for:

• Forensics (copyright enforcement, authenticity)
• Human–machine collaboration (automatic flagging of generative outputs for human review)
• Iterative improvement of generator architectures (informing which musical dimensions current models fail to capture)

7. Open Challenges and Future Directions

MoM research must contend with:

• Rapid generator evolution, causing MoM benchmarks and models to lose efficacy (“dataset and model rot”)
• Language and style bias in datasets (MoM-2025 corpus is 82% English)
• High computational costs of dual-stream detection architectures
• Limited annotation and coverage of musical genres, linguistic traditions, and polyphonic or instrumental-only music

Future research aims include:

• Unsupervised anomaly detection for unknown generator detection
• Expansion of modalities (e.g., lyric-transcript alignment)
• Lightweight model variants for real-time/commercial deployment
• Benchmarking for purely instrumental and cross-cultural music (Batra et al., 29 Nov 2025)

In summary, Melody or Machine defines a research axis focused on the scientific, technical, and practical delineation of human and synthetic musicality. The evolving MoM benchmarks and architectures set the foundation for robust, generalizable, and interpretable music forensics, as well as a deeper understanding of musical authorship in the era of powerful generative models.