Domain Adaptation in Symbolic Music

• The paper presents domain adaptation techniques that transfer models from text and synthetic domains to symbolic music using adversarial, moment matching, and supervised fine-tuning strategies.
• It highlights the use of diverse music encodings, such as time-slice (piano-roll) and event-based representations, to capture the multi-dimensional and polyphonic structure of music.
• Recent studies demonstrate that adapting LLMs and specialized generative frameworks improves performance metrics like perplexity, cosine similarity, and Fréchet Music Distance.

Domain adaptation for symbolic music refers to the suite of methodologies enabling models, data, or learning signals originating in one domain (often language or synthetic music) to transfer effectively to the target domain of symbolic music representations. Symbolic music—encoded as MIDI, ABC notation, or other event-based or pianoroll sequences—presents unique challenges for domain adaptation, including modality gaps, high-dimensional event structure, long-range dependencies, and a lack of large aligned parallel corpora. The field has progressed rapidly through the adaptation of NLP models and the development of specialized generative frameworks, with research spanning adversarial, supervised, and preference-based strategies tailored for the music context (Le et al., 2024, Cífka et al., 2019, Kumar et al., 30 Jan 2026, Brunner et al., 2018).

1. Representations and Domain Gap

Symbolic music differs from text in both semantics and data structure. Sequential representations must map music's multi-dimensional, polyphonic, and temporally quantized structure onto linear token streams for model compatibility. The two dominant schemes are:

• Time-slice or piano-roll serializations: Quantized time steps, each expressed as a multi-hot vector over pitches, optionally encoding velocities. This is analogous to fixed-grid character representations in text.
• Event-based encodings: Decompose musical events into atomic (“Pitch”, “Duration”, “Velocity”, “TimeShift”) or composite tokens, streaming them as sequences akin to text (Le et al., 2024, Cífka et al., 2019).

Formally, let $D_s = (X_s,Y_s,p_s)$ be the source (e.g., text) and $D_t = (X_t,Y_t,p_t)$ the target domain (symbolic music), both expressed as token sequences over their respective vocabularies $V_s$, $V_t$. Domain adaptation seeks a feature mapping $\phi_\theta : V_s \cup V_t \to \mathcal{H}$ such that both domains are close in a shared latent space $\mathcal{H}$. However, modality gaps—due to token-type heterogeneity, structure, and different underlying semantics—make adaptation nontrivial (Le et al., 2024).

2. Domain Adaptation Methodologies

Three broad families of domain adaptation algorithms are prominent in symbolic music:

1. Adversarial Domain Alignment

• Introduces a domain discriminator trained to distinguish between encoded text and music features.
• Objective:

$$\min_{\theta, \psi} \max_\phi [L_{\text{task}}(\theta, \psi) + \lambda L_{\text{dom}}(\theta, \phi)]$$

with domain confusion driving the encoder to produce domain-invariant features (Le et al., 2024).

2. Moment Matching (MMD)

• Aligns higher-order statistics (means and covariances) between domains using kernel-based divergence metrics.

$$\operatorname{MMD}^2(\phi(X_s), \phi(X_t)) = \|\mathbb{E}_{x\sim p_s}[\phi_\theta(x)] - \mathbb{E}_{x\sim p_t}[\phi_\theta(x)]\|^2_{\mathcal{H}_k}$$

• Used as a regularizer alongside the primary task loss (Le et al., 2024).

3. Supervised and Multi-task Fine-tuning

• Leverages supervised labels from the source and (when available) the target, either sequentially (pretrain-then-finetune) or in weighted multi-task objectives:

$$L_{\text{total}}(\theta, \psi) = L_s(\theta, \psi) + \alpha L_t(\theta, \psi)$$

• Recent studies show that LLMs, after pretraining and instruction tuning, can be further adapted to symbolic music using supervised cross-entropy learning, or preference-based updates that exploit musicality-based weak negatives (Kumar et al., 30 Jan 2026).

Unsupervised approaches (e.g., CycleGANs) have been applied to music, treating genre or style as domain variables and inferring mappings through cycle consistency and adversarial learning (Brunner et al., 2018).

3. Model Architectures and Conditioning Mechanisms

Models adapted for symbolic music span RNNs, CNNs, and—most successfully—Transformer-based architectures, many leveraging modifications for music-specific tasks:

• Conditional encoder-decoder models: Accept both content and explicit style/domain embeddings, facilitating conditioning and domain transfer (Cífka et al., 2019).
• CycleGAN frameworks: Employ dual generators and discriminators to map between musical style or genre domains, with cycle consistency enforcing content preservation (Brunner et al., 2018).
• LLMs and multi-modal Transformers: Adapted through fine-tuning, domain-specific vocabulary expansion, or contrastive pretraining to bridge language and music representations (Le et al., 2024, Kumar et al., 30 Jan 2026).

Conditioning on style or genre is typically implemented via embedding injection (e.g., learned style vector $W^s z$), promoting disentanglement between content and domain-specific attributes (Cífka et al., 2019).

4. Data Strategies and Synthetic Parallel Corpora

The scarcity of aligned symbolic music data motivates synthetic data generation:

• Synthetic parallel corpus generation: Tools like Band-in-a-Box generate multiple renderings of the same chord chart in different styles, creating aligned style pairs for supervised learning—enabling large-scale, fully supervised symbolic style transfer (Cífka et al., 2019).
• Musically motivated corruption: For preference-based adaptation, negatives are constructed via operations such as key swaps, note substitutions, and bar deletions, facilitating weak preference supervision in the absence of true labels (Kumar et al., 30 Jan 2026).

This enables controlled experiments in “70-style” translation, exceeding the limitations of single-pair or small-domain adaptation, and is key to generalization from synthetic to real data.

5. Loss Functions and Adaptation Objectives

Adaptation is enforced through a variety of tailored objectives:

• Supervised sequence loss: Conditional negative log-likelihood or cross-entropy, both at event and token levels, dominate in supervised fine-tuning and translation setups (Cífka et al., 2019, Kumar et al., 30 Jan 2026).
• Adversarial/cycle consistency loss: GAN-based approaches supplement adversarial objectives with cycle-consistency $L_1$-loss to guarantee invertibility and preservation of structure (Brunner et al., 2018).
• Preference/ranking loss: Direct Preference Optimization (DPO) and reinforcement-learning objectives introduce reward structures favoring musicality, using log-probability difference as a proxy for musical preference (Kumar et al., 30 Jan 2026).

Regularization terms (e.g., moment matching, adversarial domain loss) target distributional alignment, while embedding-based conditioning promotes explicit control over style or genre.

6. Evaluation Protocols and Metrics

Assessment of domain adaptation quality in symbolic music operates across multiple axes:

• Token-level and Language Modeling Metrics: Perplexity and cross-entropy on held-out sequences (Kumar et al., 30 Jan 2026).
• Distributional/Global Metrics: Fréchet Music Distance (FMD) over learned embeddings (e.g. CLAMP2), pitch-class histogram distance, and n-gram-based distances (Le et al., 2024, Kumar et al., 30 Jan 2026).
• Content Preservation: Cosine similarity of chroma features per frame, quantifying harmonic and melodic retention after transfer (Cífka et al., 2019).
• Style or Genre Fit: Cosine between style profiles—histograms of onset/pitch intervals—comparing generated and reference outputs (Cífka et al., 2019).
• Cycle Consistency Loss: Ensuring invertibility in CycleGANs and invariance under round-trips (Brunner et al., 2018).
• Transfer Strength via Classifiers: Domain-specific discriminators or external CNN-based genre classifiers evaluate transformation efficacy (Brunner et al., 2018).
• General Capability: Out-of-domain benchmarks (e.g., MMLU) measure preservation or decay of broader language skills in adapted architectures (Kumar et al., 30 Jan 2026).

It is standard practice to report both token-wise metrics and global musicality scores, and to monitor for catastrophic forgetting when adapting general LLMs.

7. Challenges, Limitations, and Research Directions

Key open challenges in symbolic music domain adaptation include:

• Modality gap and token heterogeneity: Music tokens are more diverse than textual ones, limiting direct embedding sharing. Customized embedding modules or radical decompositions may mitigate this (Le et al., 2024).
• Sequence length and polyphony: Musical sequences are markedly longer than typical text, requiring long-context or hierarchical attention architectures (Le et al., 2024).
• Insufficient unlabeled corpora: Symbolic-music datasets are orders of magnitude smaller than text corpora, impeding large-scale pre-training (Le et al., 2024).
• Catastrophic forgetting in LLMs: Supervised adaptation can rapidly decrease general language and instruction-following ability, especially with excessive fine-tuning epochs or aggressive low-level adaptation (Kumar et al., 30 Jan 2026).
• Evaluative bottlenecks: Many metrics (e.g., classifier-based genre fit) fail to capture perceptual musicality or stylistic authenticity; listening tests and perceptual evaluation protocols are underdeveloped (Brunner et al., 2018).

Promising avenues include cross-modal contrastive pre-training, low-rank and adapter-based efficient adaptation, augmentation via synthetic pseudo-music, meta-learning for rapid style transfer, and hybrid supervised–preference learning strategies (Le et al., 2024, Kumar et al., 30 Jan 2026). The field continues to seek principled integration of musicological knowledge (structure, phrasing, harmony) into domain adaptation frameworks.