--- title: Automatic Piano Transcription url: https://www.emergentmind.com/topics/automatic-piano-transcription-apt type: topic --- # Automatic Piano Transcription Automatic Piano Transcription (APT) converts a piano performance recorded in audio into a symbolic representation, typically note events or MIDI capturing pitch, onset, offset, and often velocity and pedal-related information. Across the literature, the task appears in several closely related formulations: framewise prediction of 88-key piano rolls, event-sequence generation with MIDI-like tokens, score-level quantization from performance representations, and score-informed refinement of expressive attributes [1411.1623] [2107.09142] [2508.07757] [2509.23878]. The central difficulty is that piano audio combines dense polyphony, long decays, overlapping resonances, expressive timing, and wide dynamic range, so successful systems typically combine acoustic modeling, temporal or symbolic priors, and increasingly explicit strategies for robustness and efficiency [2402.01424]. ## 1. Task definition and scope APT is usually treated as the piano-specialized case of Automatic Music Transcription, with subtasks that include pitch or note activity, onsets, offsets, pedaling, and dynamics or velocity [2508.07757]. Common symbolic targets include event lists, piano-rolls, and framewise activation matrices followed by event-level refinements [2402.01424]. In framewise formulations, the acoustic input is a time series of spectrogram frames and the symbolic output is a time series of 88-dimensional binary vectors, one bit per piano key per frame [1411.1623]. The literature also contains narrower and broader task variants. In monophonic settings, Scorpiano treats APT as a DSP pipeline of onset detection, tempo estimation, beat-duration computation, pitch detection, and score generation for simple piano melodies [2108.10689]. At the other extreme, some recent work defines APT as symbolic performance-to-score transcription: the input is expressive piano performance MIDI and the output is notation-oriented score tokens, without raw audio in the APT module itself [2509.23878]. A major distinction is therefore between framewise and sequence formulations. Framewise systems predict pitch activity at each analysis frame and then assemble note events through decoding or post-processing [1411.1623]. Sequence-to-sequence systems instead model the conditional distribution over a variable-length event sequence, so timing, pitch, offset, and velocity dependencies are learned directly in the decoder [2107.09142]. This split structures much of the modern APT literature. ## 2. Signal representations and symbolic vocabularies APT front ends vary substantially, but most modern systems use spectrogram-like time-frequency representations. A hybrid recurrent model from 2014 uses normalized magnitude spectrograms from audio downsampled to 16 kHz, with a 64 ms window and a 32 ms hop for training and validation, and a 10 ms hop for test [1411.1623]. A generic encoder-decoder Transformer uses 16,000 Hz audio, STFT with FFT length 2048 and hop size 128, followed by 512 mel bands and log scaling; each input segment contains 511 spectrogram frames plus an EOS embedding, corresponding to 4.088 s [2107.09142]. The hFT-Transformer also operates at 16 kHz, but with a 256-bin log-mel front end, 16 ms hop size, and chunked tensors of shape $(B, N, F, 65)$ built from local temporal context around each frame [2307.04305]. Constant-Q representations are used when harmonic alignment is treated as a primary design objective. HRplus and HRplus-hybrid use CQT with 48 bins per octave, 352 total bins, hop length 320 samples at 16 kHz, and high-resolution onset and offset targets defined by a peaked function $g(\Delta_i)$ over a window $\pm J$ frames [2409.19614]. A simpler CQT+CNN pipeline for synthetic classical piano uses 44.1 kHz audio, contiguous 2756-sample frames, and 192-by-11 CQT patches for 72-way framewise pitch activity over C2–C8 [2505.04451]. Audio-to-score work has also used HCFP features with $H=6$ harmonics, $F=352$ frequency bins, and 20 ms frame shift as input to a DeepLabv3+-based multipitch front end [2008.12710]. Symbolic outputs are equally heterogeneous. In framewise systems, the dominant representation is an 88-key piano roll or a set of per-frame onset, offset, frame, and velocity targets [1411.1623] [2204.03898]. In event-based systems, the output vocabulary usually includes time, note on, note off, and velocity tokens. The 2021 seq2seq Transformer uses 128 Note tokens, 128 Velocity tokens, 6,000 absolute Time tokens quantized in 10 ms bins, and EOS; velocity persists until updated, and note-offs are represented by Note tokens emitted after a Velocity=0 token [2107.09142]. A hierarchical language-model approach uses a token dictionary of size 1265 with special tokens, query tokens $q_p$, $q_v$, $q_f$, 1001 time tokens per 10-second segment, 128 pitch tokens, 128 velocity tokens, and a “note-sustain” token for boundary continuity [2501.03038]. An efficient sparse-attention seq2seq system uses absolute Time, Note On, Note Off, Velocity, and BOS/EOS/PAD tokens with 20 ms time resolution [2509.09318]. These design choices reflect piano-specific acoustics. Long decays, expressive timing, and polyphonic chord structures are repeatedly identified as core complications [1411.1623]. CQT-based systems argue that a logarithmic frequency axis better matches musical intervals and harmonic spacing, reducing false positives in which overtones are mistaken for separate fundamentals [2409.19614]. ## 3. Modeling paradigms One long-standing line of work is instrument-dependent generative modeling with explicit spectro-temporal templates. A studio-oriented ADMM framework builds variable-length patterns from single-note recordings of the target piano and optimizes a combination of generalized KL divergence, sparsity, total diagonal variation, and temporal-semantic regularizers resembling factorial scaled HMM constraints [1606.00785]. On Yamaha Disklavier recordings from MAPS, it reports an onset F-measure of 93–95%. A later extension adds equal-loudness thresholding based on the Glasberg–Moore loudness model and LSTM or BLSTM re-weighting of note constellations, improving F-measure from 95% to 97% on MAPS ENSTDkCl [1707.00160]. Semi-supervised CNMF is a related template-based alternative: it learns one isolated-note template per key, fixes the dictionary at test time, infers activations with KL-based CNMF, and uses adaptive onset picking; this improves over prior low-rank factorization baselines on MAPS but shows strong train-test mismatch effects on MAESTRO [2202.04989]. A second line combines acoustic classifiers with symbolic priors. The hybrid recurrent neural network formalizes the joint model as $$ P(z,x) \propto P(z_1;\Theta_l)\frac{P(z_1|x_1)}{P(z_1)} \prod_{t=2}^{T} P(z_t|\mathcal{A}_t)\frac{P(z_t|x_t)}{P(z_t)}, $$ where $P(z_t|x_t)$ is a frame-level acoustic term, $P(z_t|\mathcal A_t)$ is a music language model, and $P(z_t)$ is the empirical marginal over target vectors [1411.1623]. Because piano frames are 88-dimensional binary vectors, the language model uses RNN-NADE rather than independent sigmoids, so it can represent joint pitch co-occurrence and temporal dependencies exactly enough for beam search [1411.1623]. A later high-resolution system retains a multitask CNN-GRU architecture for onset, offset, and frame activity but replaces the velocity branch with a Transformer encoder, finding that self-attention helps velocity estimation more than frame or offset modeling [2204.03898]. Recent neural systems move further toward end-to-end event modeling. The 2021 seq2seq Transformer treats APT as spectrogram-to-event translation optimized by teacher-forced cross-entropy, $$ L = -\sum_{t=1}^{T} \log p(y_t \mid y_{