Automatic Music Engraving

• Automatic music engraving is a process that algorithmically converts audio, MIDI, or score data into visually formatted, performance-ready sheet music by adhering to musical notation conventions.
• The approach integrates interdependent subtasks such as transcription, voice/staff assignment, pitch spelling, and score rendering using advanced methods like graph neural networks, dynamic programming, and adversarial models.
• Systems in this field achieve high performance metrics in onset detection, pitch accuracy, and voice separation, and are applied to digital score generation, ancient manuscript reconstruction, and multi-instrument arrangements.

Automatic music engraving is the algorithmic process of converting musical content—audio recordings, symbolic representations (e.g., MIDI), or scores—into a visually formatted, performer-ready score that adheres to music notation conventions. Recent research formalizes engraving as a collection of interdependent subtasks that require integrated modeling: symbolic transcription, voice and staff assignment, pitch spelling, key estimation, stem direction, measure grouping, and more. Contemporary systems apply advanced signal processing, machine learning, graph neural networks, and dynamic programming to jointly or sequentially solve these subtasks, targeting multi-instrument scenarios, ancient manuscripts, and complex piano music across diverse genres.

1. Core Subtasks in Automatic Music Engraving

Automatic music engraving is best understood as a pipeline involving several tightly interrelated processes:

• Transcription (AMT): Converts audio data into symbolic note events with precise pitch, onset, and duration (Ewert et al., 2016, Derby et al., 9 Dec 2024, Sofronievski et al., 2021).
• Voice and Staff Assignment: Separates notes into structural parts (e.g., voices, staves), supports homophonic voices and cross-staff arrangements, and ensures that chords and melodic lines are correctly labeled for engraving (Foscarin et al., 15 Jul 2024, Karystinaios et al., 23 Sep 2025).
• Pitch Spelling and Key Estimation: Determines the optimal spelling of notes (e.g., F♯ vs. G♭) and both global and local key signatures, minimizing printed accidentals for readability (Bouquillard et al., 15 Feb 2024).
• Additional Notational Attributes: Computes stem direction, octave shifts, clef signs, note durations, and tuplets for each event (Karystinaios et al., 23 Sep 2025).
• Score Rendering and Output: Produces a print-ready output (MusicXML, MEI, or PDF) via integration with notation engines such as LilyPond, MuseScore, or similar (Derby et al., 9 Dec 2024, Mccloskey et al., 2023).

Each subtask may be approached separately or via integrated, multi-task models using shared feature representations.

2. Modeling Frameworks and Algorithmic Advances

Several modeling frameworks have emerged for automatic music engraving, each tailored to different input modalities and complexity:

• Signal Models and Factorizations: Variable-length spectro-temporal models and tensor factorizations enable note event extraction from audio by leveraging patterns matched to single-note isolated recordings (Ewert et al., 2016, Marmoret et al., 2021).
• Graph Neural Networks (GNNs): Music is represented as a directed, heterogeneous graph—a structure where notes are nodes equipped with high-dimensional features (pitch class, timing, duration) and edges encode musical relations (onset, follow, chord grouping, etc.) (Foscarin et al., 15 Jul 2024, Karystinaios et al., 23 Sep 2025).
  • End-to-End Systems: Multi-task GNNs allow for simultaneous prediction of voice grouping, staff assignment, pitch spelling, key signature, and more, using shared encoders and task-specific decoders.
  • Hybrid Architectures: Integration of graph convolutions (to exploit relational context) with recurrent units (such as GRUs) enables capturing both local and long-range dependencies (Karystinaios et al., 23 Sep 2025).
• Dynamic Programming: Used for joint pitch spelling and key estimation, constructing acyclic directed graphs over possible spelling states and solving for minimal accidental printing (Bouquillard et al., 15 Feb 2024).
• Adversarial Generative Models and Conditioning: GANs and conditional generative models synthesize lead sheets and multi-track arrangements by conditioning on symbolic harmonic features (e.g., chord-roll, chroma) (Liu et al., 2018, Manzelli et al., 2018).

Table: Key Modeling Approaches

Framework	Input Type	Main Subtasks Addressed
Signal Models & Factorization	Audio recordings	Note detection, onset, duration
GNNs/Hybrid GNNs	Symbolic scores/MIDI	Voice/staff separation, pitch spelling, key signature, stem direction
Dynamic Programming	MIDI	Pitch spelling, global/local key
GANs/Conditional Generators	Symbolic audio/piano-roll	Arrangement, harmonic context

3. Performance Metrics and Evaluation Protocols

Systems are evaluated using technical and musical criteria:

• F-measure for Onset Detection: High onset F-measures (93–95%) are achieved on controlled piano recordings (Ewert et al., 2016). Performance degrades slightly in reverberant settings but remains robust with tailored modeling and regularization.
• Accuracy in Voice and Staff Assignment: Homophonic voice separation F1-scores reach 96.8 on modern pop music, while more complex Romantic repertoire sees scores above 90 (Karystinaios et al., 23 Sep 2025, Foscarin et al., 15 Jul 2024).
• Pitch Spelling and Key Estimation: Joint algorithms report average pitch spelling accuracy of 99.5% on a Bach corpus and 98.2% on aggregate datasets, with global key estimation reaching 95.58% on piano works (Bouquillard et al., 15 Feb 2024).
• Additional Notational Attributes: Stem direction, octave shifts, clef reflect high classification rates (stem: 85%, octave: ~99%, clef: 96% on J-Pop; slightly lower in more complex contexts) (Karystinaios et al., 23 Sep 2025).
• Aggregate Staff Line Reconstruction: Ancient music staff reconstruction systems reach 97.55% pixel accuracy in reconstructing staff lines compared to ground truth, validated by vertical displacement and artifact rates (Tardon et al., 19 Nov 2024).

4. Technical Components and Postprocessing Pipelines

Modern systems incorporate both trainable and rule-based mechanisms to ensure musical and notational validity:

• Regularizers and Constraints: Sparsity (ℓ₁ norm), temporal diagonal variation, and Markov-state behavioral constraints enforce realistic progression, minimize spurious activation, and model expressive timing (Ewert et al., 2016).
• Chord Pooling and Linear Assignment: Chord pooling merges synchronous notes for voice assignment; final voice grouping is solved as a linear assignment problem (Hungarian algorithm), with beaming and rest infilling performed in postprocessing to yield complete measures and proper bar filling (Karystinaios et al., 23 Sep 2025, Foscarin et al., 15 Jul 2024).
• Symbolic Postprocessing: After initial predictions, rules enforce proper grouping, accidental assignment, clef placement, and notational displays per musical conventions. Deterministic variants enable efficient real-time correction (Bouquillard et al., 15 Feb 2024).
• Ancient Staff Reconstruction: Detection (Otsu binarization, area opening, morphological closing), local maxima search over stripe-wise histograms, spline fitting for smoothing, and bidirectional tracking provide robust staff preservation in optical music recognition (Tardon et al., 19 Nov 2024).

5. Applications Across Domains

Engraving systems have demonstrated utility in diverse real-world scenarios:

• Digital Audio-to-Score Pipelines: Deep learning source separation and transcription enable the end-to-end generation of separate sheet music for each instrument stem using standardized notation software such as MuseScore CLI (Derby et al., 9 Dec 2024).
• Lead Sheet Arrangement and Multi-Instrument Adaptation: Automated arrangement frameworks transpose and permute score voices to fit specified instrument ranges and output ready-to-play MusicXML, supporting ensembles with diverse instrumentations (Mccloskey et al., 2023).
• Time Signature Recovery from Lyrics: Machine learning models (e.g., XGBoost, random forest) can predict score time signatures from lyrical pattern features with an F1 score of 97.6% and AUC of 0.996, suggesting future integration in lyric-driven music composition workflows (Liao et al., 2023).
• Ancient Score Analysis and Preservation: Reconstruction algorithms facilitate analysis and performance of Renaissance and medieval scores, restoring staff visibility for scholarly and archival purposes (Tardon et al., 19 Nov 2024).

6. Current Challenges and Ongoing Directions

Despite significant advances, several open challenges persist:

• Polyphonic and Multi-Instrument Generalization: Most advanced models focus on piano or monophonic arrangements; adaptation to ensembles with polyphonic texture and diverse timbres remains a priority (Marmoret et al., 2021, Mccloskey et al., 2023).
• Handling Notational Ambiguity and Expressive Timing: Transcription models face difficulty with ambiguous note attacks, expressive rubato, and octave confusions. Postprocessing routines and robust detection logic are under active development (Sofronievski et al., 2021, Derby et al., 9 Dec 2024).
• Staff Line Detection in Noisy/Ancient Scores: Manual parameter selection and artifact rejection requires further automation for scalable OMR solutions (Tardon et al., 19 Nov 2024).
• Musical Validity and Genre Adaptation: Systems built on tonal conventions require extension for jazz modes, contemporary genres, or non-Western notation (Bouquillard et al., 15 Feb 2024).
• Data Scarcity and Model Training Constraints: Copyright limitations impact available training datasets (e.g., MUSDB18, MAESTRO), motivating augmented mixing and domain adaptation approaches (Derby et al., 9 Dec 2024).

7. Future Prospects and Research Trajectories

Ongoing research is focused on:

• Unified Multi-task Learning: Hybrid GNN architectures that jointly train on all engraving subtasks streamline the pipeline and improve cross-task performance via shared representations (Karystinaios et al., 23 Sep 2025).
• Integration with Notation Engines: Seamless pipelines converting deep learning outputs into engrave-ready formats (MusicXML, MEI) and automated rendering systems (MuseScore, LilyPond) facilitate practical deployment (Derby et al., 9 Dec 2024, Mccloskey et al., 2023).
• Robustness to Diverse Music Contexts: Expanding tensor algebra models and semi-supervised approaches to handle multi-channel (spatial) recordings and complex acoustic settings (Marmoret et al., 2021).
• Automation in Lyric-to-Score Generation: Algorithmic time signature and rhythmic framework prediction from text alone may streamline the composition and engraving of new song material (Liao et al., 2023).
• OMR and Manuscript Preservation: Extending detection, interpolation, and smoothing techniques to broader classes of historic documents and developing automatic parameter selection for rapid adaptation (Tardon et al., 19 Nov 2024).

In summary, automatic music engraving stands at the intersection of symbolic processing, machine learning, and computational musicology, with contemporary systems leveraging integrated modeling, graph neural networks, and robust optimization pipelines. Ongoing innovation in dataset, modeling strategy, and musical knowledge integration continues to advance the field toward fully automated, musically valid score production for diverse applications in performance, arrangement, analysis, and archival preservation.