MuChator: Conversational Music LLM

• MuChator is a domain-specialized large language model designed for active, conversational music discovery on Douyin Music.
• It employs a three-stage curriculum that injects objective music data, subjective reviews, and personalized preferences using advanced instruction tuning and reinforcement learning.
• Empirical evaluations show significant improvements in personalization, relevance, and engagement, with metrics like a 46% increase in active days and 77% longer session durations.

MuChator is a domain-specialized LLM framework that enables active, conversational music discovery on Douyin Music, supporting expressive, situational, and colloquial user intents that surpass conventional keyword search or passive feed paradigms. Developed by ByteDance (Liang et al., CIKM '26), MuChator systematically injects multidimensional music knowledge, aligns with dynamic user preferences, and is optimized via hybrid reinforcement learning to deliver context-aware, diverse, and highly personalized music recommendations in real time (Liang et al., 26 May 2026).

1. Architectural Overview

MuChator is an end-to-end conversational MusicLLM solution constructed on Qwen3-8B, encompassing three interconnected stages (see Figure 1, (Liang et al., 26 May 2026)):

• Music Knowledge Pre-training: A curriculum that incrementally injects (i) objective (factual) music data, (ii) subjective music semantics, and (iii) personalized preference signals through large-scale user logs and curated corpora.
• Context-aware Instruction Tuning: Generation of high-quality (user context, query, music list) triplets via automated clustering and retrieval for supervised fine-tuning, enabling accurate handling of nuanced, colloquial, and multi-intent queries.
• Preference Alignment with Hybrid Reward Modeling: Reinforcement-learning post-training using a hybrid reward that integrates intent relevance, personalization fidelity, and rule constraints, optimized by Group-wise PPO (GRPO).

This multi-stage architecture ensures the LLM possesses deep music domain grounding, flexible generative reasoning, and robust alignment with both global and individual user objectives.

2. Music Knowledge Pre-training

A three-stage curriculum learning schema embeds progressively more complex information:

Stage 1: Objective Music Knowledge

• Data drawn from Douyin Music’s metadata (tracks, artists, lyrics, genres), platform articles, and LLM-synthesized lyric descriptions.
• Training loss: standard autoregressive language modeling,

$$\mathcal{L}_{\text{LM}} = -\mathbb{E}_{\mathbf{w}\sim\mathcal{D}_1} \sum_{t}\log p_{\theta}(w_t\mid w_{<t})$$

• Purpose: ground the model in canonical musical entities and terminology.

Stage 2: Subjective Music Knowledge

• Incorporates user comments, professional reviews, curated playlists, collaborative filtering (CF) item-item relations, and music knowledge graphs.
• Continues autoregressive LM training on enriched corpus, optionally with contrastive objectives for embedding alignment.
• Endows model with interpretive, affective, and community-driven perspectives on music.

Stage 3: Personalized Music Preferences

• Integrates user profiles ($p$) and event sequences $\tau_t = (s_1,i_1,f_1,\dots,s_t,i_t,f_t)$, where $s$ denotes situation (e.g., time/mood), $i$ track ID, and $f$ engagement (like/skip).
• Next item and feedback prediction optimized as:

$$\max_{\theta} \sum_{t}\Big[ \log p_{\theta}(i_{t+1}\mid \mathcal{C}_t,s_{t+1}) +\log p_{\theta}(f_{t+1}\mid \mathcal{C}_t,s_{t+1},i_{t+1}) \Big],$$

where $\mathcal{C}_t=(p,\tau_t)$.

• This stage densely supervises the model toward both user musical trajectories and explicit feedback.

3. Context-aware Instruction Tuning

MuChator’s instruction tuning leverages a synthetic dataset $\mathcal{D}_{\mathrm{UQ2I}}$ of $(\mathcal{C},q,\mathbf{y})$ triplets:

• Semantic Clustering: Both authentic and LLM-generated music queries grouped into $p$040,000 intent clusters.
• Candidate Retrieval: User-specific sampling and retrieval of in-cluster candidates based on session history and favorites.
• Personalization Filtering: Top-K selection using an online ranking model as an oracle.

Supervised fine-tuning employs the objective:

$p$1

This process calibrates the LLM to align ranked-list responses with contextually rich, ambiguous, or under-specified user inputs, reflecting both situational and historical cues.

4. Preference Alignment via Hybrid Reward Model and RL

MuChator frames playlist generation as sequence policy learning, optimized by Group-wise PPO (GRPO) with a composite reward structure:

• Intent Relevance $p$2: Assessed by a fine-tuned judge LLM, acting as binary gating (0/1).
• Personalized Preference $p$3: Continuous score from an online ranking model quantifying match with user-specific tastes.
• Rule-based Constraints $p$4: Aggregate of format validity, catalog factuality, intra-list diversity, and prevention of recently played duplicates.

The hybrid reward is formulated:

$p$5

Policy optimization applies the GRPO clipped surrogate loss:

$p$6

where $p$7 is the probability ratio and $p$8 is the advantage computed as reward deviation from group mean. A KL penalty stabilizes large model updates.

5. Empirical Results and Evaluation

Offline experiments use the DouyinMusic-MuChator dataset (~20K user-query-music triples, 4 weeks), benchmarked against GPT-5.2 (zero/few-shot), Gemini-3-Pro, Qwen-3-8B (SFT), and Qwen-3-8B (RAG). Evaluation encompasses:

Metric	MuChator	Best Baseline	Absolute Gain
Personalization	8.4	5.7–7.1	+18%–50%
Relevance	89.1%	79.1%	+10pp
Diversity	51.1%	42.2%	+8.9pp
Factuality	99.3%	97.4%	+1.9pp

Statistically significant improvements ($p$9) are reported. Online A/B testing (vs internal search) on 100M+ DAU over one month yields:

• Active Day +46.49%
• Session Duration +77.36%
• CTR +11.26%

These metrics indicate enhanced sustained engagement and music exploration.

6. Deployment and System Characteristics

MuChator operates in Douyin Music App’s production environment, relying on Qwen3-8B with GPU inference. Runtimes are:

• First token latency: 0.09 seconds
• Full response (140 tokens): 1.87 seconds

The scope of real-world deployment (100M+ users, 1 month A/B) illustrates scalability and user impact. MuChator successfully parses and responds to natural-language, situational prompts (e.g., “songs for a seaside walk”), outperforming traditional passive-feed recommender paradigms by enabling users to proactively specify music intent and receive diverse, context-adaptive recommendations.

7. Significance and Future Directions

MuChator represents a shift from passive, feed-based discovery toward active, conversational music exploration, leveraging LLMs for deep intent modeling and user engagement. The three-stage knowledge injection, large-scale instruction tuning, and hybrid RL alignment—together with empirical superiority over proprietary LLMs—demonstrate its practical viability (Liang et al., 26 May 2026). Potential research extensions include continual learning from implicit user feedback, orchestration across multimodal/musicology sources, and low-latency deployment optimizations for broader interactive music information retrieval contexts.