Harmony-Generation Agent

• Harmony-Generation Agent is an autonomous system that creates musically coherent harmonic content using symbolic, neural, rule-based, or hybrid methods.
• It employs multi-agent architectures and formal constraint schemas to coordinate tasks like intent extraction, chord generation, and audio synthesis.
• Advanced models such as transformers, LSTMs, genetic algorithms, and diffusion techniques ensure precise, reproducible harmonic outputs.

A Harmony-Generation Agent is an autonomous or modular computational system designed to generate musically coherent, contextually appropriate harmonic content through symbolic, neural, rule-based, or hybrid approaches. This class of agents plays a central role in music information retrieval, composition, orchestration, and cross-modal content generation, unifying constraint satisfaction, data-driven inference, and practical deployment within controllable, extensible frameworks.

1. System Architectures and Agent-Based Paradigms

Harmony-Generation Agents typically employ structured, multi-agent system designs that allocate specialized subtasks—user intent interpretation, constraint extraction, symbolic decoding, validation, and rendering—to individual modules or agents, coordinated by a manager/core agent. In open, modular systems such as WeaveMuse, specialist agents may include: a symbolic composition agent (for constrained chord-sequence inference), adapters for music-theoretic extraction and formatting, and audio synthesis modules. Inter-agent communication is standardized through schema-validated JSON payloads encapsulating task, constraints, input/output format, and validation fields (Karystinaios, 14 Sep 2025, Ganapathy et al., 29 Sep 2025).

The agent hierarchy provides robust workflow separation:

• Manager/Coordinator Agent: Maintains dialogue, mediates agent pipelines, enforces resource and format constraints, and validates sub-outputs.
• Specialist Agents: Include modules such as harmony-extraction adapters, constrained symbolic generators (e.g., transformer-based or LSTM-based), notation converters, and audio renderers.

The agentic paradigm enables dynamic tool orchestration, user controllability (constraint schemas, structured decoding), and reproducible, extensible deployments across varying hardware and model budgets (Karystinaios, 14 Sep 2025, Ganapathy et al., 29 Sep 2025).

2. Formal Constraint Schemas and Decoding Strategies

A hallmark of modern harmony-generation agents is the explicit use of formalized constraint schemas to encode key harmonic parameters. These are machine-actionable representations specifying:

• Key: $k \in \{0,\dots,11\}$, mode: $m \in \{\mathrm{maj},\mathrm{min}\}$
• Length: $n$ chords/bars, progression constraints (allowable bigrams, style templates)
• Voice-leading: For each SATB part, $$|\mathrm{pitch}_v(i+1) - \mathrm{pitch}_v(i)| \leq 12$$
• Diatonic membership: $\forall i,\,c_i \in \mathrm{Scale}(k, m)$

These schemas serve as inputs to structured decoding policies:

• Constrained Beam Search: Candidate sequences are pruned if constraint-violating.
• Policy-based Sampling: Each decision step samples chords or voices with adjusted log-probabilities penalizing constraint breaches:

$$p(c_i | c_{<i}, \mathrm{constraints}) \propto \exp(\log p_\theta(c_i|...) - \lambda \cdot \mathrm{viol}(c_i))$$

Violation counts, pseudo-rewards (e.g., diatonic ratio, forbidden interval counts), and rule-based repairs are integrated in post-generation validation (Karystinaios, 14 Sep 2025).

3. Model Classes and Learning Objectives

Harmony-Generation Agents implement a diversity of generation backends:

• Transformer-based Decoders: Multi-head self-attention models (e.g., Harmony-GPT, Hierarchical Transformers) with context embeddings for melody, chords, and prior harmony, masked loss over target voices, and cross-part conditioning (Ganapathy et al., 29 Sep 2025, Zhang et al., 2021).
• Recurrent Models: LSTM or Bi-LSTM layers, typically for harmony conditioned on melody or chord contexts, with standard cross-entropy or MSE losses for note prediction and duration (Goren et al., 2021, Ganapathy et al., 29 Sep 2025).
• Genetic Algorithms: Multi-objective genetic frameworks with fitness functions integrating grammar-based music-theory metrics, predicted listener satisfaction (via trained Bi-LSTM regressors), and iterative population evolution via mutation and crossover (Majidi et al., 2021).
• Policy Gradient/RL: Group Relative Policy Optimization (GRPO) and audio-perceptual cross-modal rewards align symbolic harmony output with modern acoustic standards, supporting both supervised and reinforcement learning refinements (He et al., 28 Apr 2026).

Losses are constructed as weighted combinations of token classification, constraint violation penalties, style or perceptual rewards, and, where applicable, cross-modal or user satisfaction components (Karystinaios, 14 Sep 2025, Majidi et al., 2021, He et al., 28 Apr 2026).

4. Modalities, Input/Output Representations, and Symbolic-to-Audio Pipelines

Harmony-Generation Agents are architected for multimodal operation, enabling seamless transitions from:

• User Query (natural language or symbolic prompt) $\rightarrow$ Constraint Schema Extraction (via LLM or rule-based adapter)
• Symbolic Composition: Sequence of chord symbols, multi-part notes, or bar-level events, output as MusicXML, MIDI, or proprietary event fields (tokenized multi-level events) (Karystinaios, 14 Sep 2025, Zhang et al., 2021)
• Validation & Correction: Rule-based or statistically driven repair steps enforce key, voice-leading, and progression constraints.
• Rendering: Intermediate symbolic output is further formatted and sonified via synthesis engines (e.g., Stable Audio Open, GAN-based synthesis, voice synthesis via RVC) (Karystinaios, 14 Sep 2025, Ganapathy et al., 29 Sep 2025, Blanchard et al., 22 Jun 2025).

Input event representations are typically tuples assembling pitch, duration, onset, and chord label; outputs include detailed symbolic scores and optionally time-aligned, performance-grade audio.

5. Evaluation Metrics and Automated Validation

Performance of harmony-generation agents is quantitatively and qualitatively assessed through:

• Diatonic Ratio: Fraction of chords or notes inside the key/scale.
• Forbidden Interval Score (FIS): 1 minus the ratio of parallel fifth/octave pairs to possible voice-pairings.
• Harmony Precision/Recall: Agreement of generated skeleton with reference skeleton (SymphonyGen).
• Perceptual Audio Metrics: Cross-modal audio similarity (CLaMP), subjective listening panels.
• Musicality/Structure: Accompaniment Groove Stability (AGS), Chord Progression Realism (CPR), pairwise-interval entropy, just-intonation proximity, track density, melodic movement, and ornamentation (He et al., 28 Apr 2026, Zhang et al., 2021, Majidi et al., 2021, Takahashi, 26 Mar 2026).

Empirical results from the literature indicate that explicit constraint schemas, multi-objective fitness, and policy-gradient refinement enhance both theoretical correctness and listener satisfaction. Automated repair and in-pipeline validation provide feedback for iterative or real-time harmony adjustment.

6. Paradigm Extensions: Cross-Modal and Bio-Acoustic Harmony Agents

Harmony-Generation Agents are not confined to traditional symbolic systems. Recent research extends the paradigm to:

• Cross-modal Joint Diffusion: Multi-branch architectures enforcing synchronous audio-video (Harmony framework), solving alignment via Cross-Task Synergy training and Synchronization-Enhanced Classifier-Free Guidance (Hu et al., 26 Nov 2025).
• Bio-acoustic Agent Collectives: Systems such as Conchordal utilize artificial life dynamics within a psychoacoustic fitness landscape, eschewing symbolic rules for direct cognitive coupling to a continuous consonance field. Agents adapt pitch, metabolism, and phase—a model supporting emergent harmonic structure, evolutionary selection, and synchronization via ecological principles (Takahashi, 26 Mar 2026).

The generalization to non-symbolic and cross-domain harmony expands the conceptual boundaries of agent-based musical generation and opens avenues for further exploration in generative music intelligence, adaptive performance systems, and psychoacoustically grounded collective sound design.

7. Implementation and Reproducibility Practices

Robust deployment of Harmony-Generation Agents relies on:

• Formalization of Agent Communication: All interactions between manager and specialist agents are standardized, with schema-validated fields for task assignment, constraints, and validation (Karystinaios, 14 Sep 2025).
• Systematic Constraint Enforcement: Structured search or sampling, stepwise pruning, or logit masking are employed to ensure compliance with music-theoretic, user-defined, or psychoacoustic constraints at each generative step (Ganapathy et al., 29 Sep 2025, Majidi et al., 2021).
• Automated Validation and Repair Loops: Rule-based repair or stochastic minimal substitutions maintain harmonic integrity—critical for reproducible, user-facing applications.
• Benchmarking and Open API/Model Access: Agentic frameworks support interchangeable models, memory-efficient local inference, and reproducible research through open toolkits and deployment recipes (Karystinaios, 14 Sep 2025, Mavrin, 1 Apr 2026).

These practices have collectively standardized the Harmony-Generation Agent as a central instrument in modern MIR, algorithmic composition, and creative AI research.