CMI-Pref: Multimodal Music Preference Dataset

• CMI-Pref is a human-annotated pairwise preference dataset that distinguishes musicality from instruction alignment in multimodal music generation.
• It integrates heterogeneous prompts—including text, lyrics, and audio—to evaluate both music quality and adherence to diverse conditions.
• The dataset addresses gaps in prior evaluation methods by providing high-fidelity supervision that enhances reward-model generalization.

CMI-Pref is a human-annotated pairwise preference dataset for music reward modeling under Compositional Multimodal Instruction (CMI), introduced in the context of music-generation evaluation and alignment. In this setting, a generation condition may include an optional text description $t$, optional lyrics $l$, and optional reference audio $a_{\text{ref}}$, and the supervision target is not only generic music quality but also faithfulness to whichever subset of conditions is present. CMI-Pref therefore defines a preference-learning problem over generated audio conditioned on heterogeneous prompt structures, and serves simultaneously as a high-fidelity supervision source and as a benchmark component for multimodal music reward models (Ma et al., 28 Feb 2026).

1. Definition and conceptual scope

In the paper that introduces it, CMI-Pref is the high-quality human-annotated component of a two-part dataset ecosystem built for reward modeling in music generation. Its underlying task formulation uses a prompt

$\mathcal{P}=(t,l,a_{\text{ref}})$

and asks a model to assess an evaluated audio $a_{\text{eval}}$ along two separate axes, musicality (MUS) and alignment (ALI), with outputs

$(s_{\text{MUS}}, s_{\text{ALI}})\in\mathbb{R}^2$

(Ma et al., 28 Feb 2026).

The dataset fills a gap left by earlier music-evaluation resources. The paper argues that recommendation data measures user-item affinity rather than generated-sample alignment, that distributional metrics such as FAD are unsuitable for sample-level reward modeling, and that prior sample-level evaluators often focus narrowly on text–music alignment while neglecting lyrics and reference-audio conditioning. CMI-Pref is designed to cover that missing regime: comparative human preferences over generated music under optional, compositional multimodal conditions (Ma et al., 28 Feb 2026).

A central design choice is the separation between musicality and instruction alignment. The annotation protocol explicitly allows a sample to win on alignment while losing on musicality. This separation is not incidental; it is structurally tied to the reward-model design used in the same work, which maintains separate MUS and ALI heads. A plausible implication is that CMI-Pref is intended less as a generic music-likeness corpus than as supervision for heterogeneous alignment under mixed conditioning.

2. Dataset composition and source material

CMI-Pref is constructed from a large pool of generated music drawn from a broad generator set. The paper states that the authors “distilled audio from a diverse set of 12 models and 11 commercial APIs,” and Table 1 reports 23 models/APIs in total across CMI-Pref and CMI-Pref-Pseudo. Commercial systems included multiple versions of Suno, Stable Audio 2.0, Minimax-Music-2.0, Mureka, and Loudly. Open-source systems included MusicGen, Stable Audio Open, YUE, SongGen, AudioLDM, AudioLDM 2, DiffRhythm, Levo, Magenta Lyria-RealTime, Jamify, MusicLDM, and ACE-step (Ma et al., 28 Feb 2026).

The generation pool was balanced by conditioning type where model capabilities allowed it. For models supporting lyrics, the authors created equal splits of instrumental and vocal tracks. For models supporting audio prompting, they created equal splits with and without audio prompts. Overall, 35.6\% of samples are conditioned on audio prompts for style transfer or continuation in addition to text and lyrics. If a model could not directly ingest an audio prompt, Qwen3-Omni was used to produce an audio caption that was then supplied as an additional text condition (Ma et al., 28 Feb 2026).

The human-annotated subset itself has the following reported profile:

Property	Value
Preference samples	4,027
Annotators	31
Models/APIs represented	23
Generated audio	133.80 hours
Reference audio	48.56 hours
Balanced test split	500 pairs
Rationale feedbacks	2,574

The paper reports 2,632 unique prompts in Table 1, while the appendix reports 2,788 unique prompts for the human-annotated split. The discrepancy is acknowledged in the source text but not resolved there (Ma et al., 28 Feb 2026).

CMI-Pref is explicitly balanced across four condition types: audio + lyrics (15.2\%), audio-only (20.4\%), lyrics-only (14.9\%), and text-only (49.5\%). The appendix further states that the human-annotated split contains 840 examples with non-empty lyrics. The prompt space spans genres such as popular, electronic, rock, jazz, classical, ambient, folk, and orchestral, and also covers mood, tempo, instrumentation, production characteristics, non-English and mixed-language prompts, and both short prompts and long compositional instructions (Ma et al., 28 Feb 2026).

3. Annotation protocol and label semantics

CMI-Pref uses a pairwise preference protocol. Annotators compare two generated music clips produced from the same prompt context and provide three outputs: a Preference Label (A/B), a Confidence Score (1–5), and Free-text Feedback. The protocol is organized around two deliberately isolated judgment dimensions: Instruction Following / Alignment and Music Quality / Musicality (Ma et al., 28 Feb 2026).

For alignment, annotators were instructed to assess factors such as instrumentation, mood or atmosphere, genre or style or era, and rhythm or tempo. For musicality, they were told to judge melodic memorability, structural progression, rhythmic stability, and production clarity or absence of artifacts. Confidence was calibrated explicitly: 5 meant very certain with a strong difference, 3 was moderate or default, and 1 indicated uncertainty or a near tie. Free-text rationales were required to be concise and based on “audible evidence,” ideally with timestamps (Ma et al., 28 Feb 2026).

The annotation interface was a dedicated Music-Arena-style annotation platform providing pairwise audio comparison, confidence scoring, and free-text feedback collection. Annotators were instructed to read the instruction before listening, with the stated aim of avoiding post-hoc rationalization. The resulting rationale corpus contains 2,574 feedbacks, with 68\% Chinese and 32\% English (Ma et al., 28 Feb 2026).

Agreement analysis was based on overlapping annotations. Among 492 comparison pairs with multiple annotations, there were 1,056 votes, yielding 644 vote pairs for agreement analysis. For instruction following, the reported agreement rate is 0.691 and Krippendorff’s $\alpha$ is 0.382. For music quality, the reported agreement rate is 0.724 and Krippendorff’s $\alpha$ is 0.447. The authors describe this as moderate but reasonable given the subjectivity of music evaluation, and note that music quality is slightly more consistent than instruction following. They also report that musicality and instruction-alignment preferences agree 81\% of the time in human data, compared with 91\% in pseudo-labels (Ma et al., 28 Feb 2026).

At the learning-objective level, pairwise labels are used through a Bradley–Terry formulation. Given prompt $\mathcal P$ and candidate audios $A$ and $l$0,

$l$1

where $l$2 is either the MUS score or the ALI score, depending on the label type (Ma et al., 28 Feb 2026).

4. Position within the larger CMI ecosystem

CMI-Pref is paired with CMI-Pref-Pseudo, a much larger pseudo-labeled dataset intended for scale. The pseudo set began from 129,545 pairwise comparisons and was labeled by Qwen3-Omni using a Position-Consistency protocol: the judge was queried on both $l$3 and $l$4, and a pseudo-label was retained only if the same underlying clip was preferred regardless of position. After filtering, the paper reports 114,694 valid musicality labels and 117,828 valid alignment labels, with the intersection yielding the final ~110k pseudo-labeled pairs (Ma et al., 28 Feb 2026).

The distinction between the two resources is explicit. CMI-Pref-Pseudo provides scale for large-scale preference pretraining; CMI-Pref provides high-fidelity human supervision for fine-tuning and evaluation. The paper presents CMI-Pref as the expert-annotated preference component within CMI-RewardBench, a benchmark that combines PAM, MusicEval, Music Arena, and the CMI-Pref test split (Ma et al., 28 Feb 2026).

Within the same ecosystem, the proposed reward-model family CMI-RM uses frozen MuQ-MuLan encoders and a parameter-efficient multimodal fusion stack. The architecture proceeds by encoding text $l$5 and lyrics $l$6 with the text encoder, encoding reference audio $l$7 and evaluated audio $l$8 with the audio encoder, fusing prompt embeddings with a 4-layer Prompt Transformer, combining prompt and evaluated-audio embeddings in a Joint Transformer, and then pooling evaluation-audio hidden states before an MLP outputs

$l$9

The formulas given are

$a_{\text{ref}}$0

$a_{\text{ref}}$1

$a_{\text{ref}}$2

Empty modalities are represented as zero tensors (Ma et al., 28 Feb 2026).

Training is two-stage. Stage 1 pretrains on CMI-Pref-Pseudo for 2k steps with batch size 48, using the pairwise Bradley–Terry objective and the multitask loss

$a_{\text{ref}}$3

To reduce overconfidence from noisy pseudo-labels, label smoothing with ratio 0.2 is applied. Stage 2 fine-tunes on the training split of CMI-Pref plus MusicEval, for a total of 6,647 training samples, again with batch size 48 and early stopping; the selected checkpoint is after 250 optimization steps. In practice, CMI-Pref supplies the pairwise human-preference supervision, while scalar ratings come from MusicEval (Ma et al., 28 Feb 2026).

CMI-Pref should also be distinguished from CMI-Bench, which uses the same expansion of CMI—Compositional Multimodal Instruction—but is a benchmark for music instruction following rather than a preference dataset. CMI-Bench reformulates MIR tasks into instruction-following format and uses standardized MIR metrics, whereas CMI-Pref centers on pairwise human preferences for reward modeling (Ma et al., 14 Jun 2025).

5. Empirical behavior and benchmark significance

The main empirical significance of CMI-Pref lies in its effect on reward-model performance. On the CMI-Pref musicality test set, the reported pairwise preference accuracies are 65.40\% for PAM, 73.80\% for Audiobox-PQ, 72.40\% for SongEval-RM, 70.00\% for Gemini 2.5 Pro, 65.80\% for Gemini 3 Pro, 60.40\% for Qwen3-Omni, 70.80\% for CMI-RM Distill only, 77.80\% for CMI-RM w/ f.t.: CMI-Pref, and 78.20\% for CMI-RM w/ f.t.: CMI + MusicEval. On compositional alignment, the reported scores include 70.20\% on CMI-Pref w/o Audio for the model fine-tuned with CMI + MusicEval, 79.20\% on CMI-Pref w/ Audio for the model fine-tuned with CMI-Pref, and 82.40\% on the fully compositional Text + Lyrics + Audio subset for CMI-RM fine-tuned on CMI-Pref (Ma et al., 28 Feb 2026).

The ablation evidence presented in the paper makes CMI-Pref particularly consequential. The reported mean CMI-Pref accuracies are 71.10\% for Distill only, 75.90\% for Distill + CMI-Pref, 69.00\% for Distill + MusicEval, 76.05\% for Distill + Both, and 72.15\% for Scratch + Both. The authors summarize this result with the statement that “CMI-Pref is the primary driver for cross-benchmark generalization.” They further report that pseudo-pretraining is useful but most effective when regularized by label smoothing and followed by fine-tuning on human-labeled CMI-Pref (Ma et al., 28 Feb 2026).

Confidence-stratified analysis supports the internal consistency of the labels. For musicality, CMI-RM fine-tuned on CMI-Pref achieves 66.67\% when confidence is $a_{\text{ref}}$4, 76.56\% when confidence $a_{\text{ref}}$5, and 80.72\% when confidence $a_{\text{ref}}$6. For audio-music alignment, the CMI-Pref-finetuned model achieves 66.67\%, 72.34\%, and 84.47\% in the same confidence bins. This monotonic increase with annotator confidence suggests that the task noise captured by the confidence field is informative rather than incidental (Ma et al., 28 Feb 2026).

Several caveats are also explicit. Music preference remains subjective, with only moderate agreement. There is a distribution shift between pseudo-labeled and human-labeled data; without smoothing, pseudo-pretraining can yield overconfident boundaries. The appendix notes an anomalous drop when using prompt conditioning for musicality in the Text + Lyrics w/o Audio case, which the authors hypothesize may reflect a bottleneck in the text encoder’s handling of raw lyric structure without acoustic grounding. The release policy is TOS-aware, with only redistributable components public and restricted parts potentially requiring application-based access; the dataset and benchmark are released under CC-BY-NC-SA and accompanied by a datasheet or data card documenting source and compatibility information (Ma et al., 28 Feb 2026).

6. Terminological ambiguity and related usages

The label CMI-Pref is highly specific to the music-generation context just described, where CMI means Compositional Multimodal Instruction. This usage should not be conflated with the much older and broader use of CMI for conditional mutual information in machine learning theory and privacy-preserving feature valuation. Papers on Shapley-CMI for vertical federated learning and on unified or sharpened CMI-based generalization bounds use “CMI” in the information-theoretic sense, not in the multimodal music sense (Laskurain et al., 16 Dec 2025, Lu et al., 20 May 2026). The same distinction applies to evaluated-CMI frameworks in supervised and meta-learning, where CMI denotes a train/test selector information quantity rather than a compositional instruction format (Hellström et al., 2022, Hellström et al., 2022).

A second source of ambiguity is astronomical nomenclature. In astrophysical literature, $a_{\text{ref}}$7 CMi refers to Beta Canis Minoris, a Be star studied in work on viscous decretion discs and spectroscopic binarity, and BG CMi names an intermediate polar cataclysmic variable (Wheelwright et al., 2012, Dulaney et al., 2017, Shaw et al., 20 Jan 2026). Those usages are orthographically close but scientifically unrelated to CMI-Pref.

Within music AI itself, the closest adjacent term is CMI-Bench, which shares the same expansion of CMI and evaluates music instruction following across MIR-style tasks, but it is not a preference corpus. CMI-Bench is concerned with task-specific instruction-following metrics such as accuracy, $a_{\text{ref}}$8, WER, CER, and MIR event metrics, whereas CMI-Pref centers on human pairwise preference supervision for reward modeling (Ma et al., 14 Jun 2025). This suggests that, in the contemporary music-AI literature, CMI-Bench and CMI-Pref occupy complementary rather than interchangeable roles: one organizes instruction-following evaluation, and the other organizes preference-based alignment.