---
title: 'CMI-Pref: Multimodal Music Preference Dataset'
url: https://www.emergentmind.com/topics/cmi-pref
type: topic
---

# CMI-Pref: Multimodal Music Preference Dataset

CMI-Pref is a human-annotated pairwise preference dataset for music reward modeling under **Compositional Multimodal Instruction (CMI)**, introduced in the context of music-generation evaluation and alignment. In this setting, a generation condition may include an optional text description \(t\), optional lyrics \(l\), and optional reference audio \(a_{\text{ref}}\), and the supervision target is not only generic music quality but also faithfulness to whichever subset of conditions is present. CMI-Pref therefore defines a preference-learning problem over generated audio conditioned on heterogeneous prompt structures, and serves simultaneously as a high-fidelity supervision source and as a benchmark component for multimodal music reward models [2603.00610].

## 1. Definition and conceptual scope

In the paper that introduces it, CMI-Pref is the high-quality human-annotated component of a two-part dataset ecosystem built for reward modeling in music generation. Its underlying task formulation uses a prompt
\[
\mathcal{P}=(t,l,a_{\text{ref}})
\]
and asks a model to assess an evaluated audio \(a_{\text{eval}}\) along two separate axes, **musicality (MUS)** and **alignment (ALI)**, with outputs
\[
(s_{\text{MUS}}, s_{\text{ALI}})\in\mathbb{R}^2
\]
[2603.00610].

The dataset fills a gap left by earlier music-evaluation resources. The paper argues that recommendation data measures user-item affinity rather than generated-sample alignment, that distributional metrics such as FAD are unsuitable for sample-level reward modeling, and that prior sample-level evaluators often focus narrowly on text–music alignment while neglecting lyrics and reference-audio conditioning. CMI-Pref is designed to cover that missing regime: comparative human preferences over generated music under optional, compositional multimodal conditions [2603.00610].

A central design choice is the separation between **musicality** and **instruction alignment**. The annotation protocol explicitly allows a sample to win on alignment while losing on musicality. This separation is not incidental; it is structurally tied to the reward-model design used in the same work, which maintains separate MUS and ALI heads. A plausible implication is that CMI-Pref is intended less as a generic music-likeness corpus than as supervision for heterogeneous alignment under mixed conditioning.

## 2. Dataset composition and source material

CMI-Pref is constructed from a large pool of generated music drawn from a broad generator set. The paper states that the authors “distilled audio from a diverse set of **12 models and 11 commercial APIs**,” and Table 1 reports **23 models/APIs** in total across CMI-Pref and CMI-Pref-Pseudo. Commercial systems included multiple versions of **Suno**, **Stable Audio 2.0**, **Minimax-Music-2.0**, **Mureka**, and **Loudly**. Open-source systems included **MusicGen, Stable Audio Open, YUE, SongGen, AudioLDM, AudioLDM 2, DiffRhythm, Levo, Magenta Lyria-RealTime, Jamify, MusicLDM, and ACE-step** [2603.00610].

The generation pool was balanced by conditioning type where model capabilities allowed it. For models supporting lyrics, the authors created equal splits of **instrumental** and **vocal** tracks. For models supporting audio prompting, they created equal splits with and without audio prompts. Overall, **35.6\% of samples are conditioned on audio prompts for style transfer or continuation in addition to text and lyrics**. If a model could not directly ingest an audio prompt, **Qwen3-Omni** was used to produce an audio caption that was then supplied as an additional text condition [2603.00610].

The human-annotated subset itself has the following reported profile:

| Property | Value |
|---|---:|
| Preference samples | 4,027 |
| Annotators | 31 |
| Models/APIs represented | 23 |
| Generated audio | 133.80 hours |
| Reference audio | 48.56 hours |
| Balanced test split | 500 pairs |
| Rationale feedbacks | 2,574 |

The paper reports **2,632 unique prompts** in Table 1, while the appendix reports **2,788 unique prompts** for the human-annotated split. The discrepancy is acknowledged in the source text but not resolved there [2603.00610].

CMI-Pref is explicitly balanced across four condition types: **audio + lyrics** (**15.2\%**), **audio-only** (**20.4\%**), **lyrics-only** (**14.9\%**), and **text-only** (**49.5\%**). The appendix further states that the human-annotated split contains **840 examples with non-empty lyrics**. The prompt space spans genres such as popular, electronic, rock, jazz, classical, ambient, folk, and orchestral, and also covers mood, tempo, instrumentation, production characteristics, non-English and mixed-language prompts, and both short prompts and long compositional instructions [2603.00610].

## 3. Annotation protocol and label semantics

CMI-Pref uses a **pairwise preference** protocol. Annotators compare two generated music clips produced from the same prompt context and provide three outputs: a **Preference Label (A/B)**, a **Confidence Score (1–5)**, and **Free-text Feedback**. The protocol is organized around two deliberately isolated judgment dimensions: **Instruction Following / Alignment** and **Music Quality / Musicality** [2603.00610].

For alignment, annotators were instructed to assess factors such as instrumentation, mood or atmosphere, genre or style or era, and rhythm or tempo. For musicality, they were told to judge melodic memorability, structural progression, rhythmic stability, and production clarity or absence of artifacts. Confidence was calibrated explicitly: **5** meant very certain with a strong difference, **3** was moderate or default, and **1** indicated uncertainty or a near tie. Free-text rationales were required to be concise and based on “audible evidence,” ideally with timestamps [2603.00610].

The annotation interface was a dedicated **Music-Arena-style annotation platform** providing pairwise audio comparison, confidence scoring, and free-text feedback collection. Annotators were instructed to read the instruction before listening, with the stated aim of avoiding post-hoc rationalization. The resulting rationale corpus contains **2,574 feedbacks**, with **68\% Chinese** and **32\% English** [2603.00610].

Agreement analysis was based on overlapping annotations. Among **492 comparison pairs** with multiple annotations, there were **1,056 votes**, yielding **644 vote pairs** for agreement analysis. For **instruction following**, the reported **agreement rate** is **0.691** and **Krippendorff’s \(\alpha\)** is **0.382**. For **music quality**, the reported **agreement rate** is **0.724** and **Krippendorff’s \(\alpha\)** is **0.447**. The authors describe this as moderate but reasonable given the subjectivity of music evaluation, and note that music quality is slightly more consistent than instruction following. They also report that **musicality and instruction-alignment preferences agree 81\% of the time** in human data, compared with **91\%** in pseudo-labels [2603.00610].

At the learning-objective level, pairwise labels are used through a Bradley–Terry formulation. Given prompt \(\mathcal P\) and candidate audios \(A\) and \(B\),
\[
P(A > B) = \sigma\big(s_{\theta}(\mathcal{P}, A) - s_{\theta}(\mathcal{P}, B)\big),
\]
where \(s_\theta\) is either the MUS score or the ALI score, depending on the label type [2603.00610].

## 4. Position within the larger CMI ecosystem

CMI-Pref is paired with **CMI-Pref-Pseudo**, a much larger pseudo-labeled dataset intended for scale. The pseudo set began from **129,545 pairwise comparisons** and was labeled by **Qwen3-Omni** using a **Position-Consistency** protocol: the judge was queried on both \((A,B)\) and \((B,A)\), and a pseudo-label was retained only if the same underlying clip was preferred regardless of position. After filtering, the paper reports **114,694 valid musicality labels** and **117,828 valid alignment labels**, with the intersection yielding the final **~110k** pseudo-labeled pairs [2603.00610].

The distinction between the two resources is explicit. CMI-Pref-Pseudo provides scale for large-scale preference pretraining; CMI-Pref provides high-fidelity human supervision for fine-tuning and evaluation. The paper presents CMI-Pref as the expert-annotated preference component within **CMI-RewardBench**, a benchmark that combines **PAM**, **MusicEval**, **Music Arena**, and the **CMI-Pref test split** [2603.00610].

Within the same ecosystem, the proposed reward-model family **CMI-RM** uses frozen **MuQ-MuLan** encoders and a parameter-efficient multimodal fusion stack. The architecture proceeds by encoding text \(t\) and lyrics \(l\) with the text encoder, encoding reference audio \(a_{\text{ref}}\) and evaluated audio \(a_{\text{eval}}\) with the audio encoder, fusing prompt embeddings with a **4-layer Prompt Transformer**, combining prompt and evaluated-audio embeddings in a **Joint Transformer**, and then pooling evaluation-audio hidden states before an MLP outputs
\[
(s_{\text{ALI}}, s_{\text{MUS}}).
\]
The formulas given are
\[
\mathbf{h}_{\text{prompt}} = \text{PromptTF}([\mathbf{E}_t; \mathbf{E}_l; \mathbf{E}_{a_{\text{ref}}}]),
\]
\[
\mathbf{h}_{\text{prompt}}; \mathbf{h}_{\text{eval}} = \text{JointTF}([\mathbf{h}_{\text{prompt}}; \mathbf{E}_{a_{\text{eval}}}]),
\]
\[
(s_{\text{ALI}}, s_{\text{MUS}}) = \text{MLP}(\text{Pool}(\mathbf{h}_{\text{eval}})).
\]
Empty modalities are represented as **zero tensors** [2603.00610].

Training is two-stage. Stage 1 pretrains on CMI-Pref-Pseudo for **2k steps** with batch size **48**, using the pairwise Bradley–Terry objective and the multitask loss
\[
\mathcal{L}_{\text{total}} = 0.5 \cdot \mathcal{L}_{\text{MUS}} + 0.5 \cdot \mathcal{L}_{\text{ALI}}.
\]
To reduce overconfidence from noisy pseudo-labels, label smoothing with ratio **0.2** is applied. Stage 2 fine-tunes on the **training split of CMI-Pref** plus **MusicEval**, for a total of **6,647 training samples**, again with batch size **48** and early stopping; the selected checkpoint is after **250 optimization steps**. In practice, CMI-Pref supplies the pairwise human-preference supervision, while scalar ratings come from MusicEval [2603.00610].

CMI-Pref should also be distinguished from **CMI-Bench**, which uses the same expansion of CMI—**Compositional Multimodal Instruction**—but is a benchmark for music instruction following rather than a preference dataset. CMI-Bench reformulates MIR tasks into instruction-following format and uses standardized MIR metrics, whereas CMI-Pref centers on pairwise human preferences for reward modeling [2506.12285].

## 5. Empirical behavior and benchmark significance

The main empirical significance of CMI-Pref lies in its effect on reward-model performance. On the **CMI-Pref musicality** test set, the reported pairwise preference accuracies are **65.40\%** for **PAM**, **73.80\%** for **Audiobox-PQ**, **72.40\%** for **SongEval-RM**, **70.00\%** for **Gemini 2.5 Pro**, **65.80\%** for **Gemini 3 Pro**, **60.40\%** for **Qwen3-Omni**, **70.80\%** for **CMI-RM Distill only**, **77.80\%** for **CMI-RM w/ f.t.: CMI-Pref**, and **78.20\%** for **CMI-RM w/ f.t.: CMI + MusicEval**. On compositional alignment, the reported scores include **70.20\%** on **CMI-Pref w/o Audio** for the model fine-tuned with CMI + MusicEval, **79.20\%** on **CMI-Pref w/ Audio** for the model fine-tuned with CMI-Pref, and **82.40\%** on the fully compositional **Text + Lyrics + Audio** subset for CMI-RM fine-tuned on CMI-Pref [2603.00610].

The ablation evidence presented in the paper makes CMI-Pref particularly consequential. The reported mean CMI-Pref accuracies are **71.10\%** for **Distill only**, **75.90\%** for **Distill + CMI-Pref**, **69.00\%** for **Distill + MusicEval**, **76.05\%** for **Distill + Both**, and **72.15\%** for **Scratch + Both**. The authors summarize this result with the statement that **“CMI-Pref is the primary driver for cross-benchmark generalization.”** They further report that pseudo-pretraining is useful but most effective when regularized by label smoothing and followed by fine-tuning on human-labeled CMI-Pref [2603.00610].

Confidence-stratified analysis supports the internal consistency of the labels. For **musicality**, CMI-RM fine-tuned on CMI-Pref achieves **66.67\%** when confidence is \(<3\), **76.56\%** when confidence \(=3\), and **80.72\%** when confidence \(>3\). For **audio-music alignment**, the CMI-Pref-finetuned model achieves **66.67\%**, **72.34\%**, and **84.47\%** in the same confidence bins. This monotonic increase with annotator confidence suggests that the task noise captured by the confidence field is informative rather than incidental [2603.00610].

Several caveats are also explicit. Music preference remains subjective, with only moderate agreement. There is a distribution shift between pseudo-labeled and human-labeled data; without smoothing, pseudo-pretraining can yield overconfident boundaries. The appendix notes an anomalous drop when using prompt conditioning for musicality in the **Text + Lyrics w/o Audio** case, which the authors hypothesize may reflect a bottleneck in the text encoder’s handling of raw lyric structure without acoustic grounding. The release policy is **TOS-aware**, with only redistributable components public and restricted parts potentially requiring application-based access; the dataset and benchmark are released under **CC-BY-NC-SA** and accompanied by a datasheet or data card documenting source and compatibility information [2603.00610].

## 6. Terminological ambiguity and related usages

The label **CMI-Pref** is highly specific to the music-generation context just described, where **CMI means Compositional Multimodal Instruction**. This usage should not be conflated with the much older and broader use of **CMI** for **conditional mutual information** in machine learning theory and privacy-preserving feature valuation. Papers on Shapley-CMI for vertical federated learning and on unified or sharpened CMI-based generalization bounds use “CMI” in the information-theoretic sense, not in the multimodal music sense [2512.14767] [2605.21056]. The same distinction applies to evaluated-CMI frameworks in supervised and meta-learning, where CMI denotes a train/test selector information quantity rather than a compositional instruction format [2210.06422] [2210.06511].

A second source of ambiguity is astronomical nomenclature. In astrophysical literature, **\(\beta\) CMi** refers to **Beta Canis Minoris**, a Be star studied in work on viscous decretion discs and spectroscopic binarity, and **BG CMi** names an intermediate polar cataclysmic variable [1202.4561] [1701.05201] [2601.14156]. Those usages are orthographically close but scientifically unrelated to CMI-Pref.

Within music AI itself, the closest adjacent term is **CMI-Bench**, which shares the same expansion of CMI and evaluates music instruction following across MIR-style tasks, but it is not a preference corpus. CMI-Bench is concerned with task-specific instruction-following metrics such as accuracy, \(R^2\), WER, CER, and MIR event metrics, whereas CMI-Pref centers on human pairwise preference supervision for reward modeling [2506.12285]. This suggests that, in the contemporary music-AI literature, CMI-Bench and CMI-Pref occupy complementary rather than interchangeable roles: one organizes instruction-following evaluation, and the other organizes preference-based alignment.

Source: https://www.emergentmind.com/topics/cmi-pref