Mood-Guided Music Embedding Transformation

• The paper presents a novel framework that selectively transforms music embeddings along the mood axis while maintaining genre and instrumental characteristics.
• It employs a lightweight multi-MLP translation model with cosine similarity, triplet, and cosine BCE losses to achieve precise mood adjustments.
• Empirical evaluations show high mood precision and effective preservation of non-mood attributes, enabling applications in personalized music retrieval and adaptive playlist generation.

Mood-guided music embedding transformation refers to the process of modifying or generating music representations in such a way that a specific musical attribute—namely mood—is selectively controlled or adjusted, while other salient characteristics (such as genre, instrumentation, or timbral qualities) are preserved. This transformation usually takes place directly in a shared music embedding space, enabling controlled retrieval and recommendation tasks wherein users can query for tracks that are similar except for a desired shift in mood. Unlike earlier architectures that focused exclusively on classification or clustering, the latest approaches learn a mapping function over continuous or categorical mood labels, introducing targeted shifts within embedding manifolds without compromising non-mood musical information (Wilkins et al., 23 Oct 2025).

1. Foundational Principles and Motivation

Mood is a key dimension in music information retrieval, often orthogonal or only weakly coupled to genres or stylistic categories. However, existing embedding spaces such as those produced by large music LLMs or self-supervised architectures typically blend all musical attributes, making it difficult to alter mood in isolation. The central challenge addressed in mood-guided music embedding transformation is to disentangle mood from confounding variables (e.g., genre or instrumentation) within the high-dimensional embedding space, allowing a targeted transformation along the mood axis only.

This paradigm is motivated by several application demands: enabling "similar but happier" (or "more energetic") music search, supporting playlist generation that adapts mood without repeating styles, and giving users precise control over the affective qualities of their listening experiences—all with the efficiency and scalability of working directly in learned embeddings rather than synthesizing new audio or requiring extensive manual curation (Wilkins et al., 23 Oct 2025).

2. Controlled Transformation Frameworks

The general framework for mood-guided embedding transformation comprises three principal components: (1) an embedding space with meaningful latent dimensions (often derived from pretrained representation models), (2) a translation model operating on these embeddings that can accept explicit mood targets, and (3) a training procedure that both provides supervision of mood manipulation and actively preserves non-mood musical information.

Model Structure

• Embedding: The framework typically assumes fixed audio embeddings (e.g., of size 1728 in the MULE system) representing tracks.
• Guidance Input: Mood labels for both the seed and target states are encoded as one-hot vectors.
• Translation Model: A lightweight, multi-MLP architecture processes the concatenation of a projected seed embedding and a projected mood shift vector. The output is a transformed embedding meant to represent the original track in the target mood.

Let $x_s$ be the seed embedding, $y_s$ and $y_t$ the one-hot seed and target mood vectors. The transformation proceeds as:

• Project $x_s$ via $p_s$, yielding a 512-dim vector.
• Compute $p_y(y_t - y_s)$, the mood shift, as a 128-dim vector.
• Concatenate and feed into $p_f$ to get the transformed embedding $\hat{x}_t \in \mathbb{R}^{1728}$.

The architectural design ensures that transformation is parameter-efficient and scales to large embedding spaces (Wilkins et al., 23 Oct 2025).

3. Proxy Target Sampling and Mood Disentanglement

To solve the ill-posed problem that mood cannot be directly manipulated in the raw audio or its embedding, the framework employs a proxy target sampling strategy:

• For each seed embedding $x_s$, pre-compute a similarity map by retrieving the top‑100 most similar tracks for every mood category.
• During training, select a desired target mood $y_t$ and retrieve from the pre-selected list a track $x_t$ that is maximally similar to $x_s$ but has mood $y_t$.
• If $y_s = y_t$, use $x_s$ as the target to operationalize identity mapping.

This proxy approach ensures the model is exposed to mood changes within highly related musical neighborhoods, balancing between sufficient diversity in mood outcomes and preservation of non-target musical context. Importantly, this sampling enforces alignment between mood shifts and embedding-space locality, which is essential for disentangling mood from genre, instrumentation, and other latent factors (Wilkins et al., 23 Oct 2025).

4. Objective Functions for Transformation and Preservation

The training objective is a composite of three losses, each regularizing a distinct transformation property:

• Cosine Similarity Loss: $$L_\text{cosine} = \frac{1}{B} \sum_{i=1}^B [1 - \cos(\hat{x}_t^{(i)}, x_t^{(i)})]$$ directly aligns the transformed embedding with its proxy.
• Triplet Loss: $$L_\text{triplet} = \frac{1}{B} \sum_{i=1}^B \max(0, \alpha + \cos(\hat{x}_t^{(i)}, x_s^{(i)}) - \cos(\hat{x}_t^{(i)}, x_t^{(i)}))$$ pushes the transformed embedding closer to the proxy target than to the seed, ensuring a minimal degree of change for successful mood transfer.
• Cosine BCE Loss: $$L_\text{cosBCE} = \frac{1}{B} \sum_{i=1}^B \mathrm{BCE}(\sigma(\gamma \cdot \cos(\hat{x}_t^{(i)}, x_t^{(i)})), t^{(i)})$$ enforces identity mapping when no mood change is required ($t^{(i)}=1$), and sets a moderate cosine similarity ($t^{(i)}=0.5$) otherwise for controlled partial change.

The final objective combines these with tuned coefficients: $$L_\text{total} = \lambda_\text{cosine} L_\text{cosine} + \lambda_\text{triplet} L_\text{triplet} + \lambda_\text{cosBCE} L_\text{cosBCE}$$. This formulation enables smooth interpolation between preserving track identity and effecting a targeted attribute shift.

5. Empirical Performance and Attribute Preservation

Empirical results on both a large-scale proprietary dataset and the public MTG-Jamendo dataset demonstrate that the transformation method achieves high precision in mood change (Mood P@1 of 0.96 vs. random 0.25; on Jamendo, 0.83), while preserving non-mood attributes such as genre (Genre P@1 = 0.32) and instrument tags. Notably, performance remains strong under noisy real-world label conditions.

Baseline comparisons further establish the necessity of the proxy-guided, supervised transformation: simple methods such as average mood vector addition, oracle-based selection, or training-free approaches all underperform the proposed framework on both mood accuracy and attribute retention. This indicates that learning a targeted mapping with explicit losses and sampling is essential for effective mood-guided embedding transformation (Wilkins et al., 23 Oct 2025).

6. Application Scope, Limitations, and Research Directions

Applications of controllable mood-guided embedding transformation include:

• Personalized Music Retrieval and Recommendation: Enabling queries like "find me tracks similar to this but more energetic/happier."
• Interactive Playlist Generation: Supporting mood-adaptive playlists that do not monotonically filter by genre or artist.
• Efficient Large-Scale Retrieval: The embedding-only operation makes it suitable for streaming and on-device scenarios where audio resynthesis or re-encoding is computationally prohibitive.

Potential limitations include the reliance on embedding quality and label granularity, as well as the assumption that non-mood attributes are adequately represented in the embedding space. Extending the paradigm to multi-dimensional mood or text-based attribute shifts, integrating with audio resynthesis for full controllability, or adapting to polyphonic/classical domains are highlighted research avenues.

7. Significance in the Broader MIR Landscape

The controllable embedding transformation framework introduces a new research direction for music information retrieval, bridging the gap between simple label-based filtering and full generative audio manipulation. By operating in the embedding space, the approach offers both precision (in mood control) and efficiency (for candidate retrieval), which can be systematically evaluated using precision-at-k and category preservation metrics. This methodology provides a foundation for richer, user-centric music exploration tools and establishes mood-guided transformation as a core capability in next-generation recommendation systems (Wilkins et al., 23 Oct 2025).