Subjective Quality Evaluation of Text-to-Music Generation

Establish reliable methodologies for evaluating the subjective quality of text-to-music generation models, including both acoustic quality and musical quality, that accurately reflect human judgments.

Background

Text-to-Music models (e.g., MusicGen, MusicLM, MusicLDM) are often assessed with objective metrics like FD, FAD, and KL; however, these may not adequately reflect human-perceived acoustic quality (noise/artifacts) and musical quality (composition/performance).

The paper compares PAM with FAD variants and human MOS for acoustic and musical quality, but explicitly notes that, despite objective metrics, evaluating subjective quality remains an open research question—highlighting the need for standardized, perceptually valid evaluation frameworks for music generation.