MultiVerse: A Creator-Centered Approach to Steering Context-Adaptive Lyrics

Abstract: Generative AI may enable new forms of context-aware creative expression by dynamically tailoring media content to its consumption context. For instance, AI systems could adapt song lyrics to the listener and their current activity. However, existing media adaptation systems primarily optimize for audience experience, often neglecting artists' intent, style, and preference. We address this challenge by introducing a novel creator-centered approach to adaptive media authoring and present MultiVerse, a system that instantiates this approach for steering adaptive lyrics. Our approach allows creators to explicitly author controls based on their intent, lyric structure, and audience context, and uses rule-based validations to ensure controls are followed. We conducted a study with 10 songwriters, comparing MultiVerse with a prompting-based workflow for composing adaptive lyrics. The comparison revealed that creators preferred to author how lyrics adapt by explicitly specifying relevant context and adaptation constraints, while recognizing trade-offs in flexibility and iteration speed. Interviews further revealed that creators viewed adaptive media as enabling new forms of audience connection, introducing a distinct creative process, favoring new compositional strategies, and reshaping notions of authorship.

• The paper introduces the C³ framework and MultiVerse system, combining creator-defined intent, structural lyric controls, contextual queries, and rule-based verification to steer LLM-generated adaptations.
• The user study found that 8 of 10 songwriters preferred MultiVerse’s explicit constraints and rated adaptation predictability 4.8 versus 3.8 for prompting, while strict controls sometimes reduced flexibility and lyric quality.
• The findings show that creators favor theme-aligned contextual queries over generic demographic signals, but unresolved issues include trust in unseen outputs, ownership, privacy, over-personalization, and conflicts between hard constraints and artistic intent.

Problem statement

Generative models now make it feasible to adapt media content at consumption time to individual audience members. For song lyrics specifically, prior systems have demonstrated real-time content modification—replacing lyric lines with notification text or restructuring songs for activities—but these efforts optimize almost exclusively for the listener. This leaves a gap on the creator side: because adaptive lyrics are generated after publication, the artist cannot review each variation, and automated adaptation risks violating artistic intent, structural integrity of the composition, and the artist's sense of authorship. The paper addresses this by asking how creators—not audiences—can retain control over media that is never finalized in a single fixed artifact.

The C³ framework

The authors propose $C^3$, a creator-centered authoring paradigm that externalizes three dimensions of adaptive media as explicit, inspectable controls rather than leaving them implicit inside a generation pipeline:

• Creator: mechanisms to articulate artistic intent (theme, emotion, narrative voice), specify where and how intent applies globally versus locally, define prioritization among competing intents, and bound permissible deviation.
• Content: mechanisms to declare which structural properties must be preserved, which may relax during adaptation, and what can be added or removed—addressable at global or local granularity.
• Context: mechanisms to select which consumption-side signals drive adaptation, assign their influence and priority, and constrain how contextual information shapes outputs.

The paper's central conceptual claim is that adaptive media differs categorically from both static media (fixed output) and interactive media (variation driven by explicit user input along pre-authored branches). Adaptive media varies implicitly along contextual parameters that the creator cannot enumerate exhaustively, which means the unit of authorship shifts from artifacts to behavior over an unbounded output space. Three design axioms formalize this: support creator intent (DA1), content structure (DA2), and context-driven adaptation (DA3).

MultiVerse: system instantiation

MultiVerse operationalizes $C^3$ for lyrics under an assumed fixed melody with finalized source lyrics. Its interface exposes:

Creator controls at three granularities: global anchors (theme as natural language; emotion and style as keyword tags, auto-generated from the source lyrics then editable), optional per-line intent statements, and word locks that pin specific tokens against modification.

Content controls targeting prosodic structure: syllable count enforced as a hard constraint against vocal notes; melody clustering so multisyllabic words do not span annotated rhythmic breaks (preventing artifacts like "miss Lee dean list"); lexical stress marking aligned to melodic emphasis (disambiguating PRE-sent from pre-SENT); and rhyme bindings between note positions.

Context controls built around simulated listener personas described across nine profile dimensions (demographics, occupation, formative experiences, personality, routines, etc.). Because no persona biography can be exhaustive, MultiVerse adds contextual queries: creator-authored questions posed to personas ("what happened recently that makes you feel lost?") whose responses bind to lyric variables. Creators also see simulated listener reactions to candidate adaptations.

The implementation pipeline uses Gemini 3 Flash for generation with RiTa.js for deterministic verification. Generation is per-line with prior-line context; each attempt requests 15 candidates across rotating prompt formulations (structure-referencing, structure-ignoring, and template completion when locked words or variables are present). Candidates pass through rule-based checks on syllables, clusters, locks, accents, and rhymes; surviving candidates are then ranked by a second LLM call simulating listener preference. This hybrid generate-then-verify architecture is the paper's concrete answer to the guarantee problem: constraints verifiable by rules are enforced deterministically, not merely requested in a prompt.

User study

Ten songwriters (7 male, 3 female; mean age 23.7) with substantial experience—mean 9.2 years of songwriting and 8.2 years of formal training, including professionals who release albums and compose commercially—adapted one of their own truncated songs (8–16 lines) under two counterbalanced conditions. MultiVerse offered the full structured interface; Prompt was a hypothetical baseline using free-form instruction text applied whole-song through Gemini, with no per-line iteration and no validation guarantees. Both used identical persona sets, with held-out personas for evaluation to approximate unseen deployment contexts. Measures included CSI-derived Likert items, modified MICSI items, interaction logs, and semi-structured interviews analyzed via affinity diagramming.

Comparative findings

No subjective rating differed significantly between conditions (all $p > .05$), and overall enjoyment was identical ($M = 5.0$ for both). The value lies in the qualitative trade-off structure rather than aggregate preference:

Dimension	Finding	Evidence
Perceived control	MultiVerse preferred	8/10 cited explicit constraints; predictable adaptation rated 4.8 vs. 3.8
Meaningful context	Contextual queries widely used	9/10 queried context, 6/10 aligning queries with song themes
Flexibility cost	Strict constraints narrowed adaptation	7/10 noted fragmented or constrained output quality
Iteration speed	Prompting enabled faster exploration	7/10 emphasized rapid full-song regeneration

Two results deserve emphasis. First, participants did not want generic demographic-driven adaptation: they actively rejected default signals like profession and instead authored theme-aligned contextual queries, describing resulting lyrics as less "surface-level." This validates the paper's claim that creator-defined relevance of context is a distinct authoring act, not something a generation model supplies by default. Second, the predictability ratings cut directly against prompting: participants worried that regenerating with unchanged instructions could yield disliked variations—an unacceptable property when adaptations deploy to thousands of unreviewed listeners.

Creator perspectives on adaptive media

Interviews surfaced findings that generalize beyond either interface. Participants valued adaptive lyrics as enabling new artist–audience relationships—superfans hearing their own stories sung back, adaptive music as fan co-creation analogous to K-pop personalized messaging—but flagged over-personalization risks: reduced interpretive openness, "hyper-tailored" lyrics losing broad relatability, and perceived inauthenticity. Authoring emerged as an iterative process of discovering invariants rather than specifying intent upfront, with several participants reporting that steering forced productive self-reflection on what their songs were actually about. Notably, participants judged some songs poorly suited to adaptation—deeply personal narratives ("it's not your story to tell") and already-universal material—and indicated they would compose differently from the outset if designing for variation. Views on ownership ranged from full retention (AI as tool) through distributed models spanning artist, listener, AI developer, and platform, to framing outputs as derivative style-inspired works deserving credit and compensation. Ownership ambiguity is compounded by audience context itself making adaptations feel "audience-authored."

Limitations and open questions

Several limitations bear directly on the results. The evaluation used simulated personas rather than deployed listeners; audience reception, privacy expectations, and acceptable contextual cues remain unstudied, and the authors explicitly defer these. Trust in unreviewed generations remained low in both conditions (roughly 3.0/7), indicating that deterministic constraint checking does not yet resolve the fundamental problem of controlling outputs the creator will never observe. The study also exposed that hard constraints can conflict with softer intent—one participant found rhyme enforcement forcing lexically compatible but emotionally wrong word choices—motivating future work on prioritized, negotiable adaptation policies rather than uniform hard constraints. Scope assumptions include finalized melodies, lyrics-only adaptation (no melodic variation), audience-as-context only (excluding environment or activity triggers), and a 20-minute lab session that participants themselves said was too short to fully refine controls. Finally, the Prompt baseline is hypothetical and conversation-free by design; comparisons against richer interactive prompting workflows would strengthen the trade-off claims.

Conclusion

This paper contributes $C^3$ as a domain-agnostic authoring framework for adaptive media and demonstrates its instantiation in MultiVerse, combining structured creator controls with rule-based validation over LLM generation. The empirical finding that songwriters favor explicit specification of relevant context and adaptation constraints—while accepting costs in flexibility and iteration speed—positions steering-with-verification as a viable alternative to prompt-based control for media whose outputs are generated at scale beyond the creator's review. The open problem the work most sharply articulates is how creators should express graded commitment and conflict resolution among adaptation policies, since rigid constraints demonstrably trade off against artistic quality.