Subjective Quality Evaluation of Text-to-Music Generation

Establish reliable methodologies for evaluating the subjective quality of text-to-music generation models, including both acoustic quality and musical quality, that accurately reflect human judgments.

Background

Text-to-Music models (e.g., MusicGen, MusicLM, MusicLDM) are often assessed with objective metrics like FD, FAD, and KL; however, these may not adequately reflect human-perceived acoustic quality (noise/artifacts) and musical quality (composition/performance).

The paper compares PAM with FAD variants and human MOS for acoustic and musical quality, but explicitly notes that, despite objective metrics, evaluating subjective quality remains an open research question—highlighting the need for standardized, perceptually valid evaluation frameworks for music generation.

References

Although objective performance metrics exist, evaluating the subjective quality of these models remains an open research question.

— PAM: Prompting Audio-Language Models for Audio Quality Assessment  (2402.00282 - Deshmukh et al., 2024) in Section 5.2 (Text-to-Music generation)

Further, capturing relative quality across a broader candidate pool and reducing dependence on large-scale preference datasets remain open challenges, as existing approaches rely on pairwise comparisons or fixed datasets.

— VIBE: Video Instruction-aligned Background music gEneration  (2608.30125 - Bhosale et al., 31 Aug 2026) in Section 2, Related Work, subsection “Video-to-Music Generation”