Music-Driven Preference Elicitation

• Music-driven preference elicitation is the process of transforming diverse musical signals—such as pairwise comparisons, ratings, and physiological responses—into structured representations.
• It employs techniques like RLHF, decision trees, and Bayesian models to align music generation with human judgments and enhance recommendation systems.
• Key challenges include subjectivity, low inter-rater agreement, and temporal dynamics, prompting the need for innovative, multi-stage elicitation methods.

Searching arXiv for the cited works to ground the article in current papers. Music-driven preference elicitation is the family of methods that infer, operationalize, and exploit listener preferences from music-centered signals such as pairwise clip choices, ratings, conversational critiques, descriptor-bearing queries, and physiological responses. In current research, it appears in at least four closely related settings: alignment of generative music models to human judgments, cold-start recommendation, conversational playlist curation, and cross-domain retrieval in which music preferences are used to retrieve non-musical items such as paintings. A recent systematic analysis characterizes audio preference learning as a field in transition, with early ranking methods giving way to RLHF-style pipelines, while music-specific studies emphasize that subjectivity, temporal structure, and weak agreement among raters remain central technical constraints (Cideron et al., 2024, Broukhim et al., 17 Nov 2025).

1. Problem formulations and conceptual scope

The formal object of elicitation depends on the downstream task. In text-to-music alignment, the target is often a latent utility over generated clips conditioned on a prompt. MusicRL fits a reward model $R_\psi(c,a)$ so that, under a Bradley–Terry pairwise loss, the clip a user chose has higher reward than the alternative for the same caption $c$ (Cideron et al., 2024). In cold-start recommendation, the target is a user preference vector over items or a latent-factor representation that supports ranking unseen songs; Magron and Févotte formulate this within weighted matrix factorization and inject content side information through psychologically grounded descriptors (Magron et al., 2020).

In conversational playlist curation, the preference object is not a single item but an evolving set. The Conversational Playlist Curation Dataset defines a dialogue state in which $Z_t \subseteq C$ is the current working playlist, $\Delta Z_t$ are the newly added items at turn $t$, and the system aims to drive $Z_t \to Z^*$, where $Z^*$ is the user’s hidden ideal playlist (Chaganty et al., 2023). In music query understanding, the preference object is a structured interpretation of the request itself: MusicRecoIntent annotates user requests with descriptor spans across seven categories and assigns each span a positive, negative, or referential role, denoted $+$, $−$, and $\sim$ (Baranes et al., 11 Feb 2026).

A further formulation arises when pairwise judgments are sparse, noisy, and non-transitive. The Bayesian Mallows framework assumes each assessor has a latent full ranking $c$0 drawn from a Mallows model around a consensus permutation $c$1, then adds a mistake model to account for flipped pairwise reports (Crispino et al., 2017). This reframes elicitation as posterior inference over rankings rather than direct estimation of scalar ratings.

Across these formulations, a recurrent distinction is between explicit preference signals and latent preference structure. Explicit signals include direct ratings, pairwise wins, and natural-language descriptors. Latent structure includes hidden full rankings, collaborative embeddings, or shared semantic spaces. This suggests that music-driven preference elicitation is not a single protocol but a modeling layer that converts heterogeneous music-related observations into a tractable representation for optimization or retrieval.

2. Elicitation protocols and signal types

Existing systems collect preferences through markedly different interfaces, but most reduce them to structured comparisons or weighted descriptors.

Paradigm	Signal	Representative protocol
Pairwise generative evaluation	Preferred clip among two outputs	AI Test Kitchen users hear two 20 s clips for the same prompt and choose by “awarding a trophy” (Cideron et al., 2024)
Conversational playlist curation	Item ratings plus set-level language feedback	Wizard-of-Oz dialogues with like/dislike judgments and free-form critique over playlists (Chaganty et al., 2023)
Cold-start recommender elicitation	Item, pairwise, and attribute ratings	Dynamically rebuilt decision trees ask about artists, genres, or item pairs (Gharahighehi et al., 31 Oct 2025)
Query-intent elicitation	Descriptor spans with $c$2, $c$3, $c$4 roles	Manual annotation of 2,291 Reddit music requests across seven categories (Baranes et al., 11 Feb 2026)
Behavioral and physiological elicitation	Familiarity, response rate, EEG	Self-assessment after listening to Melody or Song stimuli, with covert timing and EEG features (Sangnark et al., 2021)
Cross-domain affect elicitation	Ratings on music excerpts used to retrieve paintings	Participants rate 11 excerpts on a 5-point Likert scale; ratings are normalized into weights (Yilma et al., 18 Jul 2025)

MusicRL provides the largest pairwise protocol in the supplied literature. After filtering, it collected 300,000 pairwise comparisons, with 285,000 used for training and 15,000 for validation of the reward model (Cideron et al., 2024). The design choice to provide no special instructions beyond selecting a preferred clip is notable because it reduces the elicitation task to holistic musical judgment rather than decomposed annotation.

Conversational elicitation captures a different granularity. CPCD contains 917 dialogues, 4,800+ total utterances, and 22,000+ item ratings, with users providing both song-level reactions and playlist-level critiques such as cohesion, activity fit, or instrumental emphasis (Chaganty et al., 2023). This is important because item-level labels alone do not expose global set properties.

Cold-start elicitation via decision trees further broadens the space of queries. The decision tree framework can ask a user to rate an item, compare an item pair, or rate an attribute such as genre, and it rebuilds the tree before each iteration using currently known ratings (Gharahighehi et al., 31 Oct 2025). The paper’s pairwise variants are motivated by the observation that humans often make more consistent judgments in comparative form than in absolute ratings.

Music-driven elicitation can also be indirect. In the EEG-based study on popular Thai music, preference labels are operationalized as “favored” versus “non-favored,” while familiarity $c$5, response rate $c$6, and regional EEG bandpower features are used for classification (Sangnark et al., 2021). In affect-aware art therapy, the elicited object is not a ranking of songs at all, but normalized music-preference weights later used to retrieve affectively related paintings (Yilma et al., 18 Jul 2025).

3. Preference representations and learned reward functions

A central research question is how to represent musical preference once raw feedback has been collected. MusicRL uses three reward components. First, text adherence is computed with a pretrained MuLan model by embedding the prompt and three non-overlapping 10 s segments $c$7 from a 30 s generation, then averaging cosine similarities: $c$8 Second, acoustic quality is predicted by a reference-free neural MOS estimator trained on 20 s clips and aggregated over two overlapping windows. Third, a learned user-preference reward $c$9 is trained on pairwise human choices; with 285,000 training pairs, 15,000 eval pairs, batch size 32, and 10,000 steps, the final validation accuracy is approximately $Z_t \subseteq C$0 (Cideron et al., 2024).

The same paper’s ablations are important because they delimit what these representations capture. Dropping text entirely from the reward-model input leaves its 60% accuracy almost unchanged; using only MuLan or only MOS to predict user choice yields approximately $Z_t \subseteq C$1 and $Z_t \subseteq C$2 accuracy; and cropping input audio from 10 s to 5 s to 3 s degrades reward-model accuracy from 60 to 58 to 56 (Cideron et al., 2024). These findings support the narrower claim that text adherence and artifact avoidance explain only part of human preference, and they suggest that listeners respond to broader musicality cues not isolated by current automatic metrics.

In recommendation settings, the representation is often lower-dimensional and explicitly interpretable. The AVD model embeds each song into $Z_t \subseteq C$3 using arousal, valence, and depth, derived from 16 high-level Essentia features through PCA followed by oblimin rotation (Magron et al., 2020). Because the factors are linked to high-level perceptual descriptors, AVD functions as a compact coordinate system for preference elicitation.

The Bayesian Mallows model takes a different route by representing each assessor with a latent permutation $Z_t \subseteq C$4, distributed according to

$Z_t \subseteq C$5

where $Z_t \subseteq C$6 is the consensus ranking and $Z_t \subseteq C$7 is a concentration parameter (Crispino et al., 2017). A Bernoulli mistake model or logistic variant then explains observed pairwise inconsistencies. This is particularly relevant for music and sound judgments because non-transitive answers are common in hard perceptual tasks.

Textual preference-bearing representations extend elicitation upstream to the query itself. MusicRecoIntent marks descriptors for Decade, Genre, Instrument, Listening Context, Mood, Named Entities, and Country, and labels whether each span is positively requested, negatively excluded, or referentially invoked (Baranes et al., 11 Feb 2026). This representation is finer-grained than standard query tagging because it distinguishes “want sad indie,” “no EDM,” and “more like Björk.”

4. Integration into generation, recommendation, and retrieval systems

Once preferences are represented, they are coupled to a policy or retrieval model. MusicRL fine-tunes a pretrained autoregressive MusicLM over interleaved semantic and coarse-acoustic tokens with a KL-regularized REINFORCE objective and a learned baseline $Z_t \subseteq C$8. The variants are structurally distinct: MusicRL-R uses a linear combination $Z_t \subseteq C$9 for 20,000 steps; MusicRL-U uses only $\Delta Z_t$0 on user data for 5,000 steps; and MusicRL-RU sequentially fine-tunes MusicRL-R on $\Delta Z_t$1 for 1,000 steps with a KL term relative to the recipe-tuned policy (Cideron et al., 2024).

In content-aware recommendation, the integration point is the item prior inside weighted matrix factorization. Magron and Févotte replace the standard isotropic prior on each item factor $\Delta Z_t$2 with $\Delta Z_t$3, where $\Delta Z_t$4 is the AVD descriptor and $\Delta Z_t$5 is a learned linear map. This yields a cold-start predictor $\Delta Z_t$6 and is trained by alternating least squares over user factors, item factors, and the mapping $\Delta Z_t$7 (Magron et al., 2020).

Decision tree elicitation uses preference signals to choose subsequent questions rather than to train a generator directly. Candidate items or attributes are evaluated by an error-based splitting criterion $\Delta Z_t$8, users are partitioned into lovers, haters, and unknowns, and the tree is rebuilt from scratch before each elicitation iteration. The pairwise variants either take the two items with the lowest $\Delta Z_t$9 or combine the best item with the most similar item in the top-$t$0 list (Gharahighehi et al., 31 Oct 2025).

Conversational playlist curation uses elicitation to support retrieval over a large song corpus. CPCD benchmarks sparse and dense retrieval baselines conditioned on dialogue history and recently added songs, with the retrieval target being the next additions $t$1 rather than an isolated item (Chaganty et al., 2023).

Cross-domain retrieval extends the same principle beyond music recommendation. In the art-therapy setting, each participant rates 11 music excerpts, the ratings are normalized into weights $t$2, and paintings are ranked by a weighted average Euclidean distance

$t$3

Mozart learns a 128-dimensional shared space with an affect-aware contrastive objective, Haydn retrieves directly in valence-arousal space, and Salieri combines deep acoustic and language-vision features (Yilma et al., 18 Jul 2025). A plausible implication is that music-driven elicitation can serve as a general affective interface, not only as a means to recommend music.

5. Evaluation protocols and empirical findings

Empirical evaluation in this area is dominated by side-by-side human comparisons, ranking metrics for recommendation, and agreement analyses. MusicRL reports that, against MusicLM, MusicRL-R has an 83% win rate, MusicRL-U 74%, and MusicRL-RU 87%; MusicRL-RU also beats MusicRL-R at 66% and MusicRL-U at 62%, with all differences passing Wilcoxon tests. Average MOS on a 1–5 scale rises from 3.07 for MusicLM to 3.54 for MusicRL-R and MusicRL-U, and 3.82 for MusicRL-RU (Cideron et al., 2024).

The recommendation literature shows a different pattern. On the Million Song Dataset subset, in-matrix NDCG is already strong for content-free WMF at 0.35, with marginal changes from WMF + Essentia(16) at 0.36 and WMF + AVD(3) at 0.35. In cold-start, pure content with AVD mean yields 0.19, WMF + Essentia(16) yields 0.22, and WMF + AVD(3) yields 0.21 (Magron et al., 2020). The relevant point is not that AVD is universally best, but that a three-dimensional, psychologically grounded representation remains competitive for cold-start recommendation.

Conversational retrieval on CPCD further shows that richer elicitation signals do not automatically simplify retrieval. Macro-averaged Hits@100 reaches 37.9% for Popularity, 45.5% for BM25, and 49.6% for Contriever fine-tuned on CPCD, while query rewriting produces mixed or negative gains (Chaganty et al., 2023). This underscores that the value of conversational feedback depends on retrieval models that can exploit both lexical anchors and higher-order set constraints.

Query-intent extraction itself is measurable. On MusicRecoIntent, Gemma 3:27B reaches overall exact-F1 0.69 and partial-F1 0.76 for descriptor extraction, while preference-bearing intent prediction on 3,030 common spans is 89% correct overall (Baranes et al., 11 Feb 2026). The strongest exact-F1 scores occur for Country, Decade, Named Entities, Genre, and Mood, whereas Listening Context is substantially harder.

Behavioral and physiological elicitation produces lower-level signals with moderate predictive performance. In the EEG study, the best mean $t$4 SE F1-scores with behavioral features are 71.1% $t$5 2.7 for Melody using SVM on the right frontal region, and 74.9% $t$6 2.8 for Song using Random Forest on the right hemisphere (Sangnark et al., 2021). These results support the narrower claim that familiarity, response timing, and regional EEG features carry preference-related information.

Benchmarking studies have also started to evaluate the evaluators. A large human-preference study over 6,000 generated songs and 15,600 pairwise judgments fits Bradley–Terry strength parameters to model-level win counts, then correlates those parameters with automatic metrics. FAD-CLAP-MA has the strongest negative correlation with music preference, and LAION-MA has the strongest correlation with text-audio alignment; inter-annotator agreement on pairwise tasks exceeds 80% (Grötschla et al., 23 Jun 2025). This provides a quantitative basis for metric selection but does not eliminate the need for human comparison.

In the art-therapy study, no engine shows a statistically significant advantage across user-centric metrics except Visual $t$7 Haydn in Diversity with $t$8, while mood improves in all groups and no between-engine difference is detected on aggregated Pick-A-Mood categories or PANAS changes (Yilma et al., 18 Jul 2025). This is a reminder that successful preference elicitation does not always manifest as large between-model differences on downstream outcomes.

6. Subjectivity, disagreement, and open research issues

The defining difficulty of music-driven preference elicitation is that musical preference is subjective, temporally extended, and often weakly transitive. MusicRL reports low inter-annotator alignment of approximately 60% in both the reward model and a small audit set, and its ablations show that neither prompt adherence nor audio quality alone captures the relevant decision criteria (Cideron et al., 2024). The Bayesian Mallows work reaches a related conclusion from a statistical angle: 80% of listeners in the electroacoustic experiment exhibit intransitivities, motivating latent uncertainty and mistake models rather than assuming coherent deterministic rankings (Crispino et al., 2017).

A second recurring issue is that item-level feedback is incomplete. CPCD shows that users naturally express set-level preferences such as cohesion, diversity, and activity fit, while the decision-tree literature finds that including genre ratings and pairwise item comparisons improves cold-start RMSE, especially with fewer questions (Chaganty et al., 2023, Gharahighehi et al., 31 Oct 2025). A common misconception is therefore that eliciting enough single-item likes and dislikes is sufficient. The evidence suggests otherwise.

A third issue is measurement mismatch. The systematic review of preference-based learning in audio finds three critical patterns: multi-dimensional evaluation strategies combining synthetic, automated, and human preferences; inconsistent alignment between traditional metrics and human judgments across contexts; and convergence on multi-stage training pipelines that combine reward signals (Broukhim et al., 17 Nov 2025). The metric benchmark refines that point within music generation by showing that some embedding-based metrics correlate strongly with human preference while others do not (Grötschla et al., 23 Jun 2025).

Finally, query interpretation remains fragile. MusicRecoIntent shows strong role agreement after adjudication, with global Cohen’s $t$9 on common spans, yet LLM extraction still struggles on context-dependent categories such as Listening Context and on the distinction between referential and positive intent (Baranes et al., 11 Feb 2026). This suggests that preference elicitation for music search is limited not only by recommendation models but also by semantic parsing of what listeners ask for.

Taken together, the supplied literature portrays music-driven preference elicitation as a technically heterogeneous but increasingly unified research area. Its common pattern is the transformation of sparse, subjective, and sometimes contradictory musical judgments into structured signals that can drive ranking, retrieval, or policy optimization. Its unresolved questions concern standardization, cultural scope, long-form temporal structure, and the extent to which learned rewards can stand in for direct human listening.