Probing Token Spaces under Generator Shift in AI-Generated Music Detection

Abstract: AI-generated music detectors can appear robust on standard benchmark splits, yet their deployments require transfer to generator sources absent during training. We study this problem with source-restricted evaluation on \textsc{MoM-open}, an open reconstruction of MoM-CLAM that replaces the non-redistributable real corpus with FMA and MTG-Jamendo while preserving the fake-generator protocol. To isolate the role of representation, we introduce \textsc{CoMoE}, a compact fixed classifier for comparing heterogeneous audio token spaces while keeping the downstream architecture and training recipe unchanged. Experiments show that standard and real-source-restricted splits are nearly saturated, whereas fake-source restriction exposes large differences between token spaces: X-Codec tokens are strongest when training on Udio alone, while MERT-derived tokens are stronger when training on Suno-v3.5 alone. These results suggest that codec-style discrete token spaces should be treated as a primary experimental axis under generator shift in AI-generated music detection. Our code and data are available at https://github.com/MAAP-LAB/CoMoE.

• The paper demonstrates that token space identity significantly affects detection robustness, with music-trained codecs like X-Codec showing enhanced cross-generator performance.
• The study uses the MoM-open dataset and compares discrete token streams from various codecs and self-supervised units to evaluate AI-generated music detection.
• Results imply that refining token representations can improve threshold stability and forensic detection in real-world scenarios under generator shift.

Probing Token Spaces under Generator Shift in AI-Generated Music Detection: An Expert Summary

Introduction

The proliferation of generative music systems capable of producing highly realistic tracks has intensified the challenge of distinguishing AI-generated music from human-created works, particularly under generator shift—a condition where the deployed detector encounters outputs from sources absent during training. This paper presents a controlled study on the robustness of AI-generated music detectors, focusing on the role of discrete audio token representations, codec-style tokens, and generator-specific shifts. It introduces MoM-open, a redistributable benchmark dataset, and the Codec-Mixture-of-Experts (CoMoE) classifier for evaluating token space effects with fixed downstream architecture. The research scrutinizes whether the choice of token space fundamentally impacts cross-generator detection performance, as opposed to treating tokenization as a preprocessing detail.

MoM-open Dataset and Evaluation Splits

The study builds MoM-open—an open reproduction of MoM-CLAM—using FMA-medium and MTG-Jamendo as real music sources to ensure redistribution compliance, and replicating the generator protocols for fake audio. This enables evaluation across diverse generator sources: Suno-v2/v3.5/v4, Udio, DiffRhythm, Riffusion, YuE, permitting rigorous analysis of generator shift.

Source-restricted evaluation splits are critical:

• Real-source restriction: Tests reliance on real corpus-specific cues (FMA or Jamendo).
• Fake-source restriction: Evaluates transfer to unseen fake generator sources, the challenging deployment scenario.

A total of 146,309 clips are standardized for duration, channel, sampling rate, codec, and metadata, affording robust comparison.

Model Architectures and Token Spaces

The core methodological contribution is CoMoE—a compact, fixed classifier variant parameterized solely by the input token space. Its design features four discrete streams: two lower-level and two higher-level token inputs, processed via identical 4-layer Transformer encoders (hidden size 256, four heads), pooled and averaged for binary classification.

Figure 1: Architecture of CoMoE, highlighting four-token-stream input structure for controlled tokenizer comparison.

Token streams are instantiated from various frontends:

• Neural audio codecs: EnCodec (acoustic RVQ, 24 kHz), DAC (acoustic RVQ, 44 kHz), X-Codec mini (music-trained, semantic-aware RVQ). Streams are selected by codebook index: early (lower-level), late (higher-level).
• Self-supervised units: MERT-v0-public layers clustered via MiniBatch k-means ($k=1024$). Early layers are lower-level, late layers are higher-level.
• Continuous baseline: Same Transformer backbone, but with linear projection of continuous MERT frame features.

Baseline detectors include mean-pooled MERT-MLP and CLAM (dual-rate contrastive).

Results: Generator Shift and Token Space Effects

Performance on MoM-open reveals:

• Saturation on base and real-source-restricted splits: CLAM, MLP (MERT), and most CoMoE variants achieve AUC near 99.8–99.9%, indicating minimal real-source bias.
• Fake-source restriction exposes substantial model variance:
  • On Fake-Suno3.5, CLAM leads (AUC 97.72%), but drops sharply in Fake-Udio (AUC 66.51%).
  • CoMoE (X-Codec) exhibits a contradictory strength: high robustness under Fake-Udio (AUC 89.04%) but not under Fake-Suno3.5—even surpassing CLAM and all other variants.
  • CoMoE (MERT $k$-means) is strongest on Fake-Suno3.5 (AUC 92.22%).
  • EnCodec is consistently inferior under generator shift, with catastrophic degradation under Fake-Udio (AUC 58.64%).
• Held-out-fake detection rate diverges from AUC: For CLAM, the validation-selected threshold results in negligible detection (2.6% for Fake-Udio), while CoMoE (X-Codec) maintains more consistent detection stability across generator-shift splits.

These findings robustly support the claim that token space identity is dominant: fixed-architecture CoMoE variants only differ by input tokens, yet exhibit large shifts in robustness. Sequential token structure thus confers significant cross-generator generalization, likely due to exposed codebook usage and quantization artifacts specific to generator outputs.

Analysis and Implications

Forensic Cues in Token Representations

Codec-style tokens encapsulate discrete, hierarchical, and residual quantization artifacts. The comparative breakdown reveals that music-trained semantic-aware codecs (X-Codec mini) provide superior cross-generator forensic cues for unseen generators (Fake-Udio), confirming the hypothesis that generator-specific artifacts are embedded in token distributions.

Limitations of Continuous Self-Supervision

Mean-pooled continuous SSL features (MLP/MERT) and even discrete MERT tokens lag in held-out detection under strong generator shift, indicating that semantic representations do not capture the low-level idiosyncrasies exploited by generator-specific codecs.

Discretization and Operating-Point Calibration

Ablation with continuous MERT inputs reveals that discretization alone does not suffice for AUC gains, but affects threshold stability. The operating point (real-world deployment threshold) is sensitive to tokenization, especially under severe generator shifts.

Practical and Theoretical Impact

Deployment Recommendations

Detectors for AI-generated music should treat codec-style token space as a primary axis—not merely preprocessing—especially for cross-generator generalization. Music-trained codecs such as X-Codec should be favored in applications with highly variable generator sources.

Research Directions

• Calibration and fusion: Robust calibration strategies are needed as ranking metrics diverge from operating-point detection under generator shift.
• Generator pool expansion: Future benchmarks should scale in number and diversity of generators, controlling for training pool size and lineage.
• Hybrid architectures: Integration of semantic and artifact-sensitive branches may improve both ranking and operating-point robustness.

Conclusion

This study firmly establishes that codec-style discrete token spaces, particularly music-trained semantic-aware codecs, are pivotal for robust AI-generated music detection under generator shift. The evidence from MoM-open, with nearly saturated benchmark splits but highly variable performance under fake-source restriction, indicates token space identity should be central in future detection architectures. The implications are vital for both forensic application and theoretical understandings of generative music artifacts. Continued evaluation of broader generator sets, calibration methodologies, and hybrid token-semantic detectors will further consolidate deployment-ready detection under adversarial generator evolution.