---
title: Expressive Performance Rendering (EPR)
url: https://www.emergentmind.com/topics/expressive-performance-rendering-epr
type: topic
---

# Expressive Performance Rendering (EPR)

Searching arXiv for recent and foundational papers on expressive performance rendering to ground the article.
arxiv_search: query="expressive performance rendering piano score performance arXiv", max_results=10
Expressive performance rendering (EPR) is the task of transforming a symbolic musical representation into a more human-like performance by adding expressive timing, dynamics, articulation, and related controls. In the literature, the task appears as score-to-performance rendering, symbolic-to-symbolic rendering, direct score-to-audio generation, and deadpan-audio-to-expressive-audio translation; the dominant application domain is solo piano, although restricted multi-instrument and multi-modal variants also exist [2510.10175] [2501.10222] [2606.12282].

## 1. Task definition and problem scope

In piano-centered work, EPR is usually framed as generating expressive deviations that transform a notated or score-derived symbolic input into a performed realization. Early basis-function work formulates expressive parameters as manipulations of dynamics, tempo, timing, and articulation informed by structural score information such as pitch, meter, tempo markings, and dynamics markings [1906.06428]. Later symbolic systems make the task more explicit at note level: low-informed models operate on score-derived MIDI with onset, offset, pitch, and velocity-like note information, whereas highly-informed systems use richer notation-derived structure such as tempo indications, time signature, key signature, and note values [2510.10175].

The scope of EPR has broadened in two directions. First, some systems stop at expressive symbolic output, predicting note-level performance parameters that can later be rendered by an external synthesizer [2208.14867] [2512.02652]. Second, more recent work treats EPR as a score-to-audio problem, either by combining symbolic rendering with neural piano synthesis or by conditioning latent audio generation on a deadpan score rendering [2501.10222] [2606.12282]. A further extension introduces text-and-score controlled audio performance generation across multiple instruments, so that the score specifies composition-level content while natural language specifies coarse interpretive intent such as speed, mistakes, and style diversity [2502.07711].

A recurring distinction is between rendering the fixed content of a score and modeling interpretive variation. Some systems explicitly define EPR as generating expressive timing, articulation, dynamics, and pedal behavior while preserving score identity; others formulate it as composition of score content and performance style, or as a transferable expressive sketch plus a score-conditioned realization process [2509.23878] [2208.14867].

## 2. Representations and expressive parameters

A central issue in EPR is how score structure and expressive output are represented. In basis-function modeling, each score element—either a note or a score onset—is represented by a vector of numerical score descriptors called basis functions. These encode structural aspects of the score, including low-level notated features and higher-level music-theoretic or cognitively motivated descriptors such as pitch, metrical information, tempo markings, dynamics markings, downbeat salience, and beat-phase timing cues [1906.06428]. The rendering problem is then expressed as
$$
y_i = f(\mathbf{\varphi}_i),
$$
where \(y_i\) is an expressive parameter and \(\mathbf{\varphi}_i\) is the basis-function vector at score element \(x_i\) [1906.06428].

Several later symbolic systems adopt note-level representations. One conditional VAE model represents the input score with eight categorical features per note: pitch, relative duration, relative inter-onset interval, whether the note is in the top voice, position within the chord, number of notes in the chord, staff, and whether the note is on a downbeat; it predicts exactly three note-level expressive parameters per note: MIDI velocity, IOIRatio, and Articulation [2208.14867]. A low-informed transformer system instead represents a performance as a note list \(n_j=(i_j,d_j,p_j,v_j)\), where \(i_j\) is inter-onset interval in seconds, \(d_j\) is duration in seconds, \(p_j\) is MIDI pitch, and \(v_j\) is MIDI velocity; pitch is copied from the input score-derived MIDI, while IOI, duration, and velocity are predicted [2510.10175].

Piano-oriented systems differ on whether pedaling is modeled. The conditional VAE rendering model and the low-informed transformer explicitly do not model pedaling [2208.14867] [2510.10175]. An integrated score-to-audio piano system likewise identifies the inability of its MIDI-to-MIDI stage to generate pedalling as a limitation [2501.10222]. By contrast, a self-supervised piano rendering model defines a fixed eight-token note representation
$$
[\text{Pitch}, \text{IOI}, \text{Velocity}, \text{Duration}, \text{Pedal1}, \text{Pedal2}, \text{Pedal3}, \text{Pedal4}],
$$
thereby making pedaling part of the symbolic rendering target alongside timing, dynamics, and articulation [2512.02652].

Beyond piano, EPR can be defined more narrowly as the assignment of synthesis controls conditioned on a score. NES-MDB separates composition from performance by modeling \(P(\mathbf{e}\mid \mathbf{c})\), where performance consists of dynamics and timbre rather than expressive timing; the expressive score augments a separated score with per-frame velocity and timbre controls and can be rendered exactly through NES APU emulation [1806.04278].

## 3. Modeling paradigms

Early neural EPR systems use supervised sequence models over engineered score features. In basis-function EPR, bidirectional LSTMs are trained separately for onset-wise and note-wise prediction, with expressive targets standardized per piece to zero mean and unit variance and learned by minimizing reconstruction error on aligned score-performance data [1906.06428]. This framework already encodes an important architectural insight: some expressive dimensions are onset-wise and shared across notes at a score onset, while others are note-specific.

A major subsequent line of work factorizes expression into global and local components. One conditional VAE for expressive piano rendering disentangles “explicit planning” and “structural attribute,” using chord-level latent sequences, a two-step note-to-chord and chord-to-note hierarchy, self-supervised regularizers, and attribute-aligned latent dimensions to support independent control over dynamics, articulation, and tempo-related behavior [2208.14867]. Another unified framework for joint piano rendering and transcription separates note-level score content from global performance style: content is encoded as a note-level sequence, style as a global latent vector, and rendering is performed as
$$
\hat{\mathbf{y}} = g_{\mathcal{Y}}(\mathbf{z}_x \oplus \mathbf{z}_s),
$$
with an additional diffusion-based Performance Style Recommendation module that generates style embeddings directly from score content [2509.23878].

Transformer-based low-informed EPR treats score-derived MIDI as the only input and predicts notewise expressive parameters by regression. Peransformer uses a transformer encoder as the performance model and introduces a score-aware discriminator that receives both the score and either a generated or human performance, so that adversarial supervision is conditioned on the underlying composition rather than on generic realism alone [2510.10175]. A distinct transformer line uses self-supervised pre-training rather than adversarial learning: Pianist Transformer adopts a unified MIDI tokenization for score and performance, an asymmetric encoder-decoder architecture with encoder sequence compression, and a 10B-token pre-training pipeline for downstream piano rendering [2512.02652].

Audio-domain EPR replaces explicit note-parameter prediction with latent audio generation. PianoKontext synthesizes a MIDI score into deadpan audio, encodes deadpan and expressive audio into a pretrained Music2Latent space, aligns them with latent-space DTW, and trains a DiT-based conditional flow-matching model to generate variable-length expressive latent sequences from deadpan context [2606.12282]. RenderBox, in turn, uses latent diffusion with joint text and score conditioning to generate direct expressive audio performances, with a curriculum that progresses from strict synthesis to expressive, mistake-corrupted, and style-directed performance [2502.07711]. A different score-to-audio design keeps the symbolic EPR stage explicit, combining a Transformer-based MIDI-to-MIDI renderer with a fine-tuned neural MIDI synthesizer for polyphonic classical piano [2501.10222].

## 4. Interpretability, controllability, and style

A persistent issue in EPR is the gap between automatic prediction and musically meaningful control. Basis-function work addresses this by exposing the influence of score descriptors through a locally linear approximation,
$$
\tilde{y}_i = c + \left(\nabla f(\mathbf{\varphi}_{*})\right)^{\top} (\mathbf{\varphi}_i - \mathbf{\varphi}_{*}),
$$
and, for onset-wise models, a temporal Jacobian whose columns can be scaled to reweight the contribution of individual score features across the piece [1906.06428]. This produces an interactive interface in which users manipulate feature-level contribution weighting, overall tempo preference, and articulation preference while the system updates expressive curves and rendered piano playback.

The conditional VAE literature pursues a different form of control: latent factorization. In “explicit planning” models, a higher-level expressive sketch is separated from score-conditioned structural behavior, and self-supervised regularizers align parts of the planning latent with dynamics, articulation, and tempo so that moving a latent coordinate yields “sliding-fader” control; coherent long-range plans can also be injected manually as “smooth sketches” [2208.14867]. The content-style disentanglement framework likewise supports several inference modes: default automated rendering using recommended style, style-conditioned rendering from a reference performance, and cross-piece style transfer through a target style vector or its interpolation [2509.23878].

Text conditioning expands controllability beyond symbolic parameters. RenderBox treats text as coarse-level expressive control and score as granular content control, enabling prompts such as “a bit slower,” “style of Vladimir Ashkenazy,” “expressive performance,” “calm,” or “less-experienced player with mistakes” [2502.07711]. This suggests a shift from latent-only steering toward user-facing semantic control. A plausible implication is that EPR control interfaces are increasingly being organized at three levels: score-faithful local structure, interpretable symbolic performance parameters, and high-level semantic guidance.

At the same time, controllability is not uniform across paradigms. Audio-first systems such as PianoKontext currently expose mainly the score and the desired output duration or length, rather than explicit controls for articulation, dynamics profile, or pedaling [2606.12282]. Self-supervised piano rendering models improve expressive quality at scale, but their stated future direction is toward controllable generation from intuitive inputs such as natural language rather than toward direct user manipulation of explicit score features [2512.02652].

## 5. Data resources, benchmarks, and evaluation

EPR research relies heavily on aligned score-performance corpora. Basis-function work uses the Magaloff/Chopin and Zeilinger/Beethoven datasets, consisting of recordings of piano music performed on computer-controlled Bösendorfer grand pianos and aligned to scores [1906.06428]. Flexible rendering with disentangled planning and structure trains on 356 performances of 34 Chopin pieces from the Yamaha e-Competition Dataset and Vienna 4x22 Piano Corpus, and tests generalization on ASAP and a small non-Classical set [2208.14867]. Peransformer constructs ASAP-MIDI, a note-to-note aligned paired dataset derived from ASAP, with approximately 8:1:1 train/validation/test splits and explicit alignment-based filtering [2510.10175]. The integrated score-to-audio piano system uses aligned subsets of ATEPP-1.2 for both its MIDI-to-MIDI and MIDI-to-audio stages [2501.10222].

The field also includes dataset work that separates score from expressive realization more explicitly. NES-MDB contains 397 games, 296 composers, 5,278 songs, 2,325,636 notes, and 46.1 hours, with expressive score representations and deterministic symbolic-to-audio rendering via NES APU emulation [1806.04278]. Large-scale symbolic pre-training moves further away from pairwise alignment: Pianist Transformer pre-trains on 10B MIDI tokens aggregated from Aria-MIDI, GiantMIDI-Piano, PDMX, POP909, and Pianist8 before supervised adaptation to ASAP [2512.02652].

Evaluation remains heterogeneous. Prototype-oriented work sometimes offers only conceptual and qualitative evidence, with no formal user study or numerical benchmark [1906.06428]. Symbolic framewise rendering on NES-MDB uses negative log-likelihood and accuracy, especially at Points of Interest, to avoid being dominated by trivial frame repetition [1806.04278]. Disentangled piano rendering reports Pearson correlation, disentanglement errors \(\text{MSE}_p\) and \(\text{MSE}_s\), and controllability metrics called consistency, restrictiveness, and linearity [2208.14867]. Peransformer introduces Generalized EPR Metrics (GEM), which align each rendition to each human performance, convert MIDI files into a common note representation, support multiple human references, and report MSE and Pearson correlation for IOI, duration, and velocity [2510.10175]. Pianist Transformer instead evaluates global token-distribution realism using Jensen–Shannon divergence and Intersection Area over Velocity, Duration, IOI, and Pedal, alongside a listening study [2512.02652].

Public comparative evaluation has also re-emerged. RenCon 2025 revived the expressive performance rendering competition, using MusicXML input, a preliminary online round, and a live final on an unseen piano piece with a human Disklavier baseline; the final attracted 48 evaluators, and 36 correctly identified the human performance [2605.02059]. This indicates that benchmark protocol, playback conditions, and evaluation granularity remain consequential.

## 6. Current limitations and emerging directions

Several limitations recur across the literature. Local linear control over nonlinear score-to-expression models may only be accurate near the chosen reference point, and some short-paper systems provide no formal validation that such Jacobian-based manipulation preserves musical plausibility under larger edits [1906.06428]. In disentangled rendering, the planning prior may be temporally independent, so coherent long-range expressive trajectories must be inserted manually as smooth sketches rather than generated automatically [2208.14867]. Low-informed rendering remains underdetermined relative to notation-rich settings, and even strong low-informed systems still lag highly-informed systems and human performance on several metrics [2510.10175].

Pedaling remains unevenly treated. Some symbolic piano systems explicitly omit it [2208.14867] [2510.10175], and integrated score-to-audio work identifies missing pedalling as a factor that degrades downstream synthesis quality [2501.10222]. Audio-domain EPR can in principle capture richer acoustic cues, but current results remain segment-limited, piano-only, and weak on some articulatory details; PianoKontext itself notes that a Debussy example lacks the desired non-legato articulation [2606.12282]. Even where a symbolic representation includes Pedal1–Pedal4 tokens, the available corpus may mostly expose binary on/off sustain rather than nuanced half-pedaling [2512.02652].

Claims about human parity are also benchmark-dependent. One self-supervised piano rendering system reports outputs that are statistically indistinguishable from a human pianist in its listening-study setup [2512.02652], whereas RenCon 2025 reports that the human baseline received the highest score and was correctly identified in 75% of final-round responses [2605.02059]. This suggests that “human-level” in EPR is not a settled designation but depends strongly on repertoire, playback conditions, comparison set, and evaluation design.

The most visible research directions are therefore converging rather than singular. One direction emphasizes scalable self-supervised pre-training on massive symbolic corpora [2512.02652]. Another emphasizes disentangled or recommended style variables for controllable rendering and transfer [2509.23878] [2208.14867]. A third moves toward multimodal conditioning, combining text, score, and direct audio generation [2502.07711] [2606.12282]. A fourth seeks standardized comparison procedures through MIDI-space evaluation workflows and revived public competitions [2510.10175] [2605.02059]. Taken together, these developments suggest that EPR is evolving from narrowly supervised score-to-MIDI regression into a broader family of controllable, multi-representation rendering systems that operate across symbolic, semantic, and audio domains.

Source: https://www.emergentmind.com/topics/expressive-performance-rendering-epr