Objective evaluation of automatic music mixing systems

Develop robust and standardized objective evaluation methodologies for automatic music mixing systems to address the current challenge that existing metrics (e.g., stereo-invariant loss and low-level audio feature measures such as spectral, panning, dynamic, and loudness descriptors) can yield mismatched conclusions about system performance, thereby providing reliable and comparable assessments of mixing quality.

Background

In the music mixing experiments, the authors evaluated systems using a stereo-invariant loss and a suite of low-level audio features related to spectral content, panning, dynamics, and loudness. They observed that these metrics can disagree, producing a mismatch between the loss and feature-based assessments.

The authors highlight that, due to the subjective nature of mixing and the absence of standardized objective protocols, objectively evaluating automatic mixing remains difficult. They note that this evaluation challenge persists and identify it as an open research direction, suggesting that more reliable objective methodologies are needed to complement or replace current metric combinations.