Expressive Performance Rendering (EPR)

• Expressive Performance Rendering (EPR) is a method that converts symbolic musical scores into nuanced performances by adding timing, dynamics, and articulation.
• It employs diverse modeling paradigms including neural sequence models, transformer architectures, and latent audio generation to capture performance subtleties.
• Recent advances emphasize controllability through score descriptors, latent factorization, and multimodal conditioning to bridge automated synthesis with human-like interpretation.

Searching arXiv for recent and foundational papers on expressive performance rendering to ground the article. arxiv_search: query="expressive performance rendering piano score performance arXiv", max_results=10 Expressive performance rendering (EPR) is the task of transforming a symbolic musical representation into a more human-like performance by adding expressive timing, dynamics, articulation, and related controls. In the literature, the task appears as score-to-performance rendering, symbolic-to-symbolic rendering, direct score-to-audio generation, and deadpan-audio-to-expressive-audio translation; the dominant application domain is solo piano, although restricted multi-instrument and multi-modal variants also exist (He et al., 11 Oct 2025, Tang et al., 17 Jan 2025, Gavrilev, 10 Jun 2026).

1. Task definition and problem scope

In piano-centered work, EPR is usually framed as generating expressive deviations that transform a notated or score-derived symbolic input into a performed realization. Early basis-function work formulates expressive parameters as manipulations of dynamics, tempo, timing, and articulation informed by structural score information such as pitch, meter, tempo markings, and dynamics markings (Shi et al., 2019). Later symbolic systems make the task more explicit at note level: low-informed models operate on score-derived MIDI with onset, offset, pitch, and velocity-like note information, whereas highly-informed systems use richer notation-derived structure such as tempo indications, time signature, key signature, and note values (He et al., 11 Oct 2025).

The scope of EPR has broadened in two directions. First, some systems stop at expressive symbolic output, predicting note-level performance parameters that can later be rendered by an external synthesizer (Rhyu et al., 2022, You et al., 2 Dec 2025). Second, more recent work treats EPR as a score-to-audio problem, either by combining symbolic rendering with neural piano synthesis or by conditioning latent audio generation on a deadpan score rendering (Tang et al., 17 Jan 2025, Gavrilev, 10 Jun 2026). A further extension introduces text-and-score controlled audio performance generation across multiple instruments, so that the score specifies composition-level content while natural language specifies coarse interpretive intent such as speed, mistakes, and style diversity (Zhang et al., 11 Feb 2025).

A recurring distinction is between rendering the fixed content of a score and modeling interpretive variation. Some systems explicitly define EPR as generating expressive timing, articulation, dynamics, and pedal behavior while preserving score identity; others formulate it as composition of score content and performance style, or as a transferable expressive sketch plus a score-conditioned realization process (Zeng et al., 28 Sep 2025, Rhyu et al., 2022).

2. Representations and expressive parameters

A central issue in EPR is how score structure and expressive output are represented. In basis-function modeling, each score element—either a note or a score onset—is represented by a vector of numerical score descriptors called basis functions. These encode structural aspects of the score, including low-level notated features and higher-level music-theoretic or cognitively motivated descriptors such as pitch, metrical information, tempo markings, dynamics markings, downbeat salience, and beat-phase timing cues (Shi et al., 2019). The rendering problem is then expressed as

$y_i = f(\mathbf{\varphi}_i),$

where $y_i$ is an expressive parameter and $\mathbf{\varphi}_i$ is the basis-function vector at score element $x_i$ (Shi et al., 2019).

Several later symbolic systems adopt note-level representations. One conditional VAE model represents the input score with eight categorical features per note: pitch, relative duration, relative inter-onset interval, whether the note is in the top voice, position within the chord, number of notes in the chord, staff, and whether the note is on a downbeat; it predicts exactly three note-level expressive parameters per note: MIDI velocity, IOIRatio, and Articulation (Rhyu et al., 2022). A low-informed transformer system instead represents a performance as a note list $n_j=(i_j,d_j,p_j,v_j)$, where $i_j$ is inter-onset interval in seconds, $d_j$ is duration in seconds, $p_j$ is MIDI pitch, and $v_j$ is MIDI velocity; pitch is copied from the input score-derived MIDI, while IOI, duration, and velocity are predicted (He et al., 11 Oct 2025).

Piano-oriented systems differ on whether pedaling is modeled. The conditional VAE rendering model and the low-informed transformer explicitly do not model pedaling (Rhyu et al., 2022, He et al., 11 Oct 2025). An integrated score-to-audio piano system likewise identifies the inability of its MIDI-to-MIDI stage to generate pedalling as a limitation (Tang et al., 17 Jan 2025). By contrast, a self-supervised piano rendering model defines a fixed eight-token note representation

$$[\text{Pitch}, \text{IOI}, \text{Velocity}, \text{Duration}, \text{Pedal1}, \text{Pedal2}, \text{Pedal3}, \text{Pedal4}],$$

thereby making pedaling part of the symbolic rendering target alongside timing, dynamics, and articulation (You et al., 2 Dec 2025).

Beyond piano, EPR can be defined more narrowly as the assignment of synthesis controls conditioned on a score. NES-MDB separates composition from performance by modeling $y_i$0, where performance consists of dynamics and timbre rather than expressive timing; the expressive score augments a separated score with per-frame velocity and timbre controls and can be rendered exactly through NES APU emulation (Donahue et al., 2018).

3. Modeling paradigms

Early neural EPR systems use supervised sequence models over engineered score features. In basis-function EPR, bidirectional LSTMs are trained separately for onset-wise and note-wise prediction, with expressive targets standardized per piece to zero mean and unit variance and learned by minimizing reconstruction error on aligned score-performance data (Shi et al., 2019). This framework already encodes an important architectural insight: some expressive dimensions are onset-wise and shared across notes at a score onset, while others are note-specific.

A major subsequent line of work factorizes expression into global and local components. One conditional VAE for expressive piano rendering disentangles “explicit planning” and “structural attribute,” using chord-level latent sequences, a two-step note-to-chord and chord-to-note hierarchy, self-supervised regularizers, and attribute-aligned latent dimensions to support independent control over dynamics, articulation, and tempo-related behavior (Rhyu et al., 2022). Another unified framework for joint piano rendering and transcription separates note-level score content from global performance style: content is encoded as a note-level sequence, style as a global latent vector, and rendering is performed as

$y_i$1

with an additional diffusion-based Performance Style Recommendation module that generates style embeddings directly from score content (Zeng et al., 28 Sep 2025).

Transformer-based low-informed EPR treats score-derived MIDI as the only input and predicts notewise expressive parameters by regression. Peransformer uses a transformer encoder as the performance model and introduces a score-aware discriminator that receives both the score and either a generated or human performance, so that adversarial supervision is conditioned on the underlying composition rather than on generic realism alone (He et al., 11 Oct 2025). A distinct transformer line uses self-supervised pre-training rather than adversarial learning: Pianist Transformer adopts a unified MIDI tokenization for score and performance, an asymmetric encoder-decoder architecture with encoder sequence compression, and a 10B-token pre-training pipeline for downstream piano rendering (You et al., 2 Dec 2025).

Audio-domain EPR replaces explicit note-parameter prediction with latent audio generation. PianoKontext synthesizes a MIDI score into deadpan audio, encodes deadpan and expressive audio into a pretrained Music2Latent space, aligns them with latent-space DTW, and trains a DiT-based conditional flow-matching model to generate variable-length expressive latent sequences from deadpan context (Gavrilev, 10 Jun 2026). RenderBox, in turn, uses latent diffusion with joint text and score conditioning to generate direct expressive audio performances, with a curriculum that progresses from strict synthesis to expressive, mistake-corrupted, and style-directed performance (Zhang et al., 11 Feb 2025). A different score-to-audio design keeps the symbolic EPR stage explicit, combining a Transformer-based MIDI-to-MIDI renderer with a fine-tuned neural MIDI synthesizer for polyphonic classical piano (Tang et al., 17 Jan 2025).

4. Interpretability, controllability, and style

A persistent issue in EPR is the gap between automatic prediction and musically meaningful control. Basis-function work addresses this by exposing the influence of score descriptors through a locally linear approximation,

$y_i$2

and, for onset-wise models, a temporal Jacobian whose columns can be scaled to reweight the contribution of individual score features across the piece (Shi et al., 2019). This produces an interactive interface in which users manipulate feature-level contribution weighting, overall tempo preference, and articulation preference while the system updates expressive curves and rendered piano playback.

The conditional VAE literature pursues a different form of control: latent factorization. In “explicit planning” models, a higher-level expressive sketch is separated from score-conditioned structural behavior, and self-supervised regularizers align parts of the planning latent with dynamics, articulation, and tempo so that moving a latent coordinate yields “sliding-fader” control; coherent long-range plans can also be injected manually as “smooth sketches” (Rhyu et al., 2022). The content-style disentanglement framework likewise supports several inference modes: default automated rendering using recommended style, style-conditioned rendering from a reference performance, and cross-piece style transfer through a target style vector or its interpolation (Zeng et al., 28 Sep 2025).

Text conditioning expands controllability beyond symbolic parameters. RenderBox treats text as coarse-level expressive control and score as granular content control, enabling prompts such as “a bit slower,” “style of Vladimir Ashkenazy,” “expressive performance,” “calm,” or “less-experienced player with mistakes” (Zhang et al., 11 Feb 2025). This suggests a shift from latent-only steering toward user-facing semantic control. A plausible implication is that EPR control interfaces are increasingly being organized at three levels: score-faithful local structure, interpretable symbolic performance parameters, and high-level semantic guidance.

At the same time, controllability is not uniform across paradigms. Audio-first systems such as PianoKontext currently expose mainly the score and the desired output duration or length, rather than explicit controls for articulation, dynamics profile, or pedaling (Gavrilev, 10 Jun 2026). Self-supervised piano rendering models improve expressive quality at scale, but their stated future direction is toward controllable generation from intuitive inputs such as natural language rather than toward direct user manipulation of explicit score features (You et al., 2 Dec 2025).

5. Data resources, benchmarks, and evaluation

EPR research relies heavily on aligned score-performance corpora. Basis-function work uses the Magaloff/Chopin and Zeilinger/Beethoven datasets, consisting of recordings of piano music performed on computer-controlled Bösendorfer grand pianos and aligned to scores (Shi et al., 2019). Flexible rendering with disentangled planning and structure trains on 356 performances of 34 Chopin pieces from the Yamaha e-Competition Dataset and Vienna 4x22 Piano Corpus, and tests generalization on ASAP and a small non-Classical set (Rhyu et al., 2022). Peransformer constructs ASAP-MIDI, a note-to-note aligned paired dataset derived from ASAP, with approximately 8:1:1 train/validation/test splits and explicit alignment-based filtering (He et al., 11 Oct 2025). The integrated score-to-audio piano system uses aligned subsets of ATEPP-1.2 for both its MIDI-to-MIDI and MIDI-to-audio stages (Tang et al., 17 Jan 2025).

The field also includes dataset work that separates score from expressive realization more explicitly. NES-MDB contains 397 games, 296 composers, 5,278 songs, 2,325,636 notes, and 46.1 hours, with expressive score representations and deterministic symbolic-to-audio rendering via NES APU emulation (Donahue et al., 2018). Large-scale symbolic pre-training moves further away from pairwise alignment: Pianist Transformer pre-trains on 10B MIDI tokens aggregated from Aria-MIDI, GiantMIDI-Piano, PDMX, POP909, and Pianist8 before supervised adaptation to ASAP (You et al., 2 Dec 2025).

Evaluation remains heterogeneous. Prototype-oriented work sometimes offers only conceptual and qualitative evidence, with no formal user study or numerical benchmark (Shi et al., 2019). Symbolic framewise rendering on NES-MDB uses negative log-likelihood and accuracy, especially at Points of Interest, to avoid being dominated by trivial frame repetition (Donahue et al., 2018). Disentangled piano rendering reports Pearson correlation, disentanglement errors $y_i$3 and $y_i$4, and controllability metrics called consistency, restrictiveness, and linearity (Rhyu et al., 2022). Peransformer introduces Generalized EPR Metrics (GEM), which align each rendition to each human performance, convert MIDI files into a common note representation, support multiple human references, and report MSE and Pearson correlation for IOI, duration, and velocity (He et al., 11 Oct 2025). Pianist Transformer instead evaluates global token-distribution realism using Jensen–Shannon divergence and Intersection Area over Velocity, Duration, IOI, and Pedal, alongside a listening study (You et al., 2 Dec 2025).

Public comparative evaluation has also re-emerged. RenCon 2025 revived the expressive performance rendering competition, using MusicXML input, a preliminary online round, and a live final on an unseen piano piece with a human Disklavier baseline; the final attracted 48 evaluators, and 36 correctly identified the human performance (Zhang et al., 3 May 2026). This indicates that benchmark protocol, playback conditions, and evaluation granularity remain consequential.

6. Current limitations and emerging directions

Several limitations recur across the literature. Local linear control over nonlinear score-to-expression models may only be accurate near the chosen reference point, and some short-paper systems provide no formal validation that such Jacobian-based manipulation preserves musical plausibility under larger edits (Shi et al., 2019). In disentangled rendering, the planning prior may be temporally independent, so coherent long-range expressive trajectories must be inserted manually as smooth sketches rather than generated automatically (Rhyu et al., 2022). Low-informed rendering remains underdetermined relative to notation-rich settings, and even strong low-informed systems still lag highly-informed systems and human performance on several metrics (He et al., 11 Oct 2025).

Pedaling remains unevenly treated. Some symbolic piano systems explicitly omit it (Rhyu et al., 2022, He et al., 11 Oct 2025), and integrated score-to-audio work identifies missing pedalling as a factor that degrades downstream synthesis quality (Tang et al., 17 Jan 2025). Audio-domain EPR can in principle capture richer acoustic cues, but current results remain segment-limited, piano-only, and weak on some articulatory details; PianoKontext itself notes that a Debussy example lacks the desired non-legato articulation (Gavrilev, 10 Jun 2026). Even where a symbolic representation includes Pedal1–Pedal4 tokens, the available corpus may mostly expose binary on/off sustain rather than nuanced half-pedaling (You et al., 2 Dec 2025).

Claims about human parity are also benchmark-dependent. One self-supervised piano rendering system reports outputs that are statistically indistinguishable from a human pianist in its listening-study setup (You et al., 2 Dec 2025), whereas RenCon 2025 reports that the human baseline received the highest score and was correctly identified in 75% of final-round responses (Zhang et al., 3 May 2026). This suggests that “human-level” in EPR is not a settled designation but depends strongly on repertoire, playback conditions, comparison set, and evaluation design.

The most visible research directions are therefore converging rather than singular. One direction emphasizes scalable self-supervised pre-training on massive symbolic corpora (You et al., 2 Dec 2025). Another emphasizes disentangled or recommended style variables for controllable rendering and transfer (Zeng et al., 28 Sep 2025, Rhyu et al., 2022). A third moves toward multimodal conditioning, combining text, score, and direct audio generation (Zhang et al., 11 Feb 2025, Gavrilev, 10 Jun 2026). A fourth seeks standardized comparison procedures through MIDI-space evaluation workflows and revived public competitions (He et al., 11 Oct 2025, Zhang et al., 3 May 2026). Taken together, these developments suggest that EPR is evolving from narrowly supervised score-to-MIDI regression into a broader family of controllable, multi-representation rendering systems that operate across symbolic, semantic, and audio domains.