Quantitative evaluation of music quality in symbolic generation

Determine reliable quantitative evaluation methodologies for music quality in symbolic music generation that can be used for objective assessment and comparison of generated music across models and datasets.

Background

In their experimental evaluation, the authors need objective measures to compare the quality of music produced by different generative models and configurations. They note that, despite various proxy metrics, establishing a robust quantitative evaluation framework for music quality remains unresolved.

As a practical workaround, the paper employs overlapping area (OA) metrics across several musical attributes as a necessary condition for good quality, and supplements this with listening tests. However, the authors explicitly state that quantitative evaluation of music quality is still an open problem, underscoring the need for more principled, validated metrics.

References

It is worth mentioning that quantitative evaluation of music quality remains an open problem .

— Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion  (2402.14285 - Huang et al., 2024) in Section 5.2 Unconditional Generation, Objective Metrics

Nevertheless, a more holistic assessment of expressive piano performance remains an open problem. Future research may explore richer evaluation metrics that jointly capture note accuracy, timing flexibility, and dynamic shaping, all of which contribute to perceptual expressivity.

— Expressive Robotic Pianist: Mastering Complex Piano Repertoire with Graph-Mimic and Musical Dynamics  (2609.10844 - Liang et al., 9 Sep 2026) in Section “Limitations and future work,” third limitation