Aria-MIDI: Large-Scale Expressive Piano Dataset

• Aria-MIDI is a dataset of solo piano MIDI transcriptions derived from real YouTube performances, capturing expressive timing, articulation, and rubato.
• The corpus is created through a multi-stage pipeline using language-model scoring, CNN-based audio classification, and robust Whisper-based transcription.
• With over 1.18M files and 100,629 hours of audio, Aria-MIDI provides scalable infrastructure for generative modeling and music information retrieval.

Aria-MIDI is a dataset of piano MIDI files for symbolic music modeling, created by transcribing audio recordings of piano performances into their constituent notes. It is presented as a large scored corpus of expressive solo-piano performance transcriptions, metadata tags, and piano-audio classifier scores, intended to bridge the gap between small, high-quality aligned performance datasets and large but noisy symbolic corpora. The released collection contains 1,186,253 distinct MIDI files comprising 100,629 hours of transcribed audio; after metadata-based compositional deduplication it reduces to 800,973 files and 66,650 hours (Bradshaw et al., 21 Apr 2025).

1. Definition, scope, and scale

Aria-MIDI is a single-track piano dataset derived from publicly accessible internet audio, primarily YouTube recordings. Unlike raw internet MIDI files, which are often sequenced or score-converted and may miss expressive timing and dynamics of human playing, Aria-MIDI starts from real audio performances. In the paper’s framing, the dataset addresses three motivations: scale, expressive realism, and diversity. Manual alignment and transcription bottlenecks limit dataset growth; audio performances preserve timing, articulation, rubato, and dynamics; and a purely manually curated pipeline does not scale well across internet music and styles.

The corpus is described as one of the largest publicly available piano MIDI collections to date. The raw MIDI collection is about 20 GB, and the dataset is accompanied by metadata tags and piano-audio classifier scores for each file. The paper does not define conventional train/validation/test splits for Aria-MIDI itself; instead, it provides the full dataset and discusses ways to create cleaner or deduplicated subsets using metadata and audio scores.

Dataset	Files	Hours
MAESTRO	1,276	199
GiantMIDI	10,855	1,237
ATEPP	11,742	1,009
PiJAMA	2,777	223
Lakh	176,581	9,567
Aria-MIDI	1,186,253	100,629

Relative to earlier symbolic datasets reported in the paper, Aria-MIDI is substantially larger while remaining focused on expressive solo piano rather than multitrack symbolic mixtures. The paper therefore positions it as infrastructure for generative modeling, symbolic understanding, and curated downstream subsets rather than as a narrowly scoped benchmark (Bradshaw et al., 21 Apr 2025).

2. Corpus construction pipeline

The construction pipeline has three main stages: crawling and metadata-based candidate discovery, audio classification and segmentation, and transcription plus metadata extraction. Rather than beginning from a large manually collected table of pieces, composers, and performers, the pipeline starts from a manually curated set of 50 solo-piano seed videos spanning multiple genres and styles. From those seeds, the system alternates between language-model scoring of each unscored video and crawling related videos from YouTube, prioritized by score.

For every discovered but unscored video, the system sends the YouTube title and description to Llama 3.1 70B with a fixed system prompt and requests a score from 0 to 5 indicating how likely the video is to be a solo piano performance. The prompt explicitly down-scores other instruments or vocalists, tutorials or lessons, organ or harpsichord, narration, orchestral accompaniment such as piano concertos, non-performance content, and repetitive loops. The crawler then queries the YouTube related videos endpoint, taking higher-scoring videos first. Across the whole pipeline, the authors collected YouTube data for 3,290,453 videos and processed 1,713,650 of those using the audio classifier.

The paper’s rationale for the language-model stage is that a LLM can recognize repertoire, performers, and naming conventions in titles and descriptions much better than crude keyword filters. Even so, text-only filtering was found insufficient, because titles and descriptions may imply solo piano while the audio contains wrong instruments, orchestra, speech, applause, poor recording quality, long silences, additional accompaniment, or misleading content. This motivates the second-stage audio classifier.

The classifier is trained using a source-separation-based pseudo-labeling procedure. Given an audio file, the pipeline applies the MVSep Piano source-separation model to obtain piano and other components. For each 5-second clip, both stems are resampled to 22,050 Hz, spectrograms are computed with 2048 frequency bins, frame length 2048, and hop length 512, RMS energy per frame is computed, and energies are converted to dBFS. A clip is labeled non-piano if either the other component contains a contiguous region longer than $l_\text{min}$ whose energy exceeds $\text{dB}_\text{min}$, or the piano component contains a contiguous silent region below -20 dB lasting more than 4 seconds. Otherwise the clip is labeled piano (Bradshaw et al., 21 Apr 2025).

3. Segmentation, transcription, and metadata extraction

The audio classifier is a CNN-based model with 5 convolutional layers, followed by 2 dense layers and 1 output neuron. Its input is a mel-spectrogram from a 5-second audio clip at 22,050 Hz, with 2048 spectrogram frequency bins, 256 mel bins, and hop length 220 samples, corresponding to 10 ms hops. Training uses AdamW with $\beta_1, \beta_2 = 0.9, 0.95$, $\epsilon = \text{1e-6}$, weight decay 0.01, a linear learning-rate schedule, 500 optimizer warmup steps, and decay to 10% of the initial learning rate. To improve robustness, the pipeline adds corresponding audio augmentation in about 10% of batches, randomly applies pitch shifting and bandpass filters, and includes labeled examples of noisy-but-acceptable piano recordings.

After training, the classifier is used both to reject bad files and to segment good ones into contiguous piano-only regions. The paper gives the segmentation rule explicitly: “We classify a region $(n, m+5)$, $m \geq n+d$, as non-piano if and only if all segments starting between $n$ and $m$ are scored below $\lambda$.” The parameters are $d = 3$ and $\text{dB}_\text{min}$0. Inference scores every 5-second interval with a 1-second stride; non-piano spans are removed; remaining contiguous spans are kept as piano only if they are longer than 45 seconds; and any piano segment whose average score is below 0.7 is discarded. Because mostly silent intervals are scored non-piano, piano performances separated by at least $\text{dB}_\text{min}$1 seconds of silence are split into separate segments.

After segmentation, audio segments are transcribed using Aria-AMT, a Whisper-based piano transcription model. The paper states that Aria-AMT was chosen for robustness to diverse timbres, different recording environments, and internet-quality audio. The authors report that they transcribed 100,629 hours of audio in 765 hours on an NVIDIA H100 with batch size 128, corresponding to about 131× real time and approximately 2327 notes per second.

Metadata extraction is then performed with Llama 3.1 70B from YouTube titles and descriptions. The requested fields are composer, opus, piece_number, genre, form, performer, key_signature, difficulty, and music_period. The prompt instructs the model to omit uncertain fields entirely, use composer and performer last names only, restrict opus and piece number to integers, include opus and piece_number only for complete performances of a single traditional opus or catalogued work, avoid confusing sonata numbers with piece numbers, and infer well-known works such as “Moonlight Sonata” if certain. These metadata are subsequently used both for corpus analysis and for compositional deduplication (Bradshaw et al., 21 Apr 2025).

4. Reported quality control and evaluation

The paper’s empirical emphasis is on validating the dataset-construction pipeline rather than training downstream symbolic models. Language-model text-only classification was evaluated on 250 randomly sampled crawled videos with human audio-ground-truth labels from two musically trained pianists. For Llama 3.1 70B, the reported precision/recall/F1 values were 64.76 / 97.14 / 77.71 at threshold score $\text{dB}_\text{min}$2, 70.83 / 97.14 / 81.93 at threshold score $\text{dB}_\text{min}$3, and 84.51 / 85.71 / 85.11 at threshold score $\text{dB}_\text{min}$4. The authors conclude that threshold 4 gives a strong balance of cost and accuracy, while also emphasizing a clear discrepancy between text-only predictions and audio truth.

Audio segmentation was evaluated on a 250-recording random sample with language-model score at least 3, excluding training recordings. Two musically trained annotators segmented each file and labeled it as not solo-piano, solo-piano with significant audio artifacts, or solo-piano with good to pristine quality. A predicted segment counted as correct if its start matched reference within ±2 seconds and its end matched within ±5 seconds, with each reference segment matched to at most one prediction. For the full corpus, the proposed setting $\text{dB}_\text{min}$5 achieved segmentation precision 71.97%, recall 75.84%, F1 73.86%, piano overlap 94.22%, and non-piano overlap 98.83%. For quality solo-piano recordings, the same setting achieved precision 75.36%, recall 80.62%, F1 77.90%, piano overlap 96.38%, and non-piano overlap 84.80%.

The file-level filtering results are particularly central to the paper’s cleanliness claim. Using segments generated with $\text{dB}_\text{min}$6 and requiring a minimum average segment score $\text{dB}_\text{min}$7, the paper reports that for all solo-piano, at $\text{dB}_\text{min}$8, precision is 100.00%, recall 89.86%, F1 94.66%, and false positives 0. For quality solo-piano, at $\text{dB}_\text{min}$9, precision is 90.98%, recall 95.28%, F1 93.08%, with 12 false positives; at $\beta_1, \beta_2 = 0.9, 0.95$0, quality precision rises to 95.54% and recall drops to 84.25%, with 5 false positives.

Transcription quality is evaluated in Appendix A.3 against the piano transcription model from Kong et al. For the chosen model, MAESTRO note precision/recall/F1 are 98.86 / 96.45 / 97.63; MAESTRO note+offset 91.63 / 89.42 / 90.50; MAESTRO note+offset+velocity 90.56 / 88.39 / 89.45; and MAPS note 91.78 / 89.47 / 90.58. The paper interprets these results as especially strong on note offsets, note offsets with velocity, and MAPS generalization, which is important because Aria-MIDI depends on out-of-domain transcription robustness (Bradshaw et al., 21 Apr 2025).

5. Metadata structure, deduplication, and content profile

Aria-MIDI provides metadata tags and piano-audio classifier scores for each file, but metadata coverage is uneven across fields. On a random sample of 200 files manually checked against titles and descriptions, reported presence and accuracy are 71.0% and 99.3% for composer, 62.0% and 99.2% for performer, 32.0% and 100.0% for opus number, 22.0% and 93.2% for piece number, 23.0% and 97.8% for key signature, 86.5% and 94.2% for genre, and 63.0% and 92.9% for music period. Fine-grained work identification is therefore sparser than broad descriptive labeling.

The compositional deduplication procedure uses composer, opus number, and piece number. A file is removed as a compositional duplicate if it matches another file on all three tags, or matches on composer plus opus when piece_number is absent. In addition, for composers appearing more than 250 times, the pipeline prunes all extra files that lack both opus number and piece number. This yields 23,283 unique metadata triples and the deduplicated corpus of 800,973 files. The paper presents this as a useful but imperfect deduplication scheme: it controls repeated works to the extent allowed by text metadata, but not at the level of exact performance duplication or all possible aliases.

The repertoire is described as mixed genre but overwhelmingly classical in total volume. The supported genre labels are classical, jazz, pop, blues, ragtime, atonal, rock, soundtrack, ambient, and folk. The extracted metadata distributions show that classical piano dominates, but many other genres are present. The paper also notes strong concentration around canonical composers and long-tail imbalance. An illustrative example is that “moonlight” appears in 6,819 titles, likely referring to Beethoven’s Moonlight Sonata.

The paper does not provide a rich table of note-count distributions, duration histograms, pitch histograms, pedal statistics, or velocity distributions for the MIDI files themselves. The principal reported symbolic-scale statistics are the numbers of files and hours, the number of unique metadata triples, and the size of the deduplicated collection. This suggests that Aria-MIDI is intended primarily as a large scored resource from which researchers can derive task-specific subsets, rather than as a fully characterized benchmark with exhaustive symbolic corpus statistics (Bradshaw et al., 21 Apr 2025).

6. Role in symbolic music modeling and downstream systems

Aria-MIDI is explicitly framed as infrastructure for symbolic music models that need large corpora for pretraining, generation, MIR tasks, and metadata-filtered or deduplicated subsets. The paper itself does not present downstream symbolic music generation experiments trained on Aria-MIDI. Its proposed use cases are pretraining or scaling studies in symbolic music modeling, generative symbolic music models, MIR tasks needing metadata labels, creation of cleaner subsets using classifier scores, and compositional deduplication using metadata.

A concrete downstream linkage appears in the real-time duet system Aria-Duet. That paper describes Aria as “an autoregressive transformer model designed to model expressive symbolic piano performances (i.e., on the note-level)” and states that Aria was “pretrained on a refined subset of Aria-MIDI, a large-scale (100k+ hours) dataset of solo piano music spanning a wide range of genres and styles.” It further states that the Aria-MIDI data was “curated using a transcription model that was itself trained on paired audio and MIDI recordings from a Disklavier,” which “creates a direct correspondence between the model's training data and the Disklavier-based I/O of our system.” In that deployment, Aria uses a note-centric tokenizer, explicit pedal-on and pedal-off tokens, and next-token prediction to generate continuations for a Yamaha Disklavier in a turn-taking human–AI duet (Bradshaw et al., 3 Nov 2025).

The relation to interactive symbolic systems can also be understood by contrast with piano-only editing models such as PIA. PIA is an inpainting system trained on GiantMIDI-Piano rather than Aria-MIDI; it uses Structured MIDI Encoding, an encoder-decoder Linear Transformer, and a DAW-facing workflow centered on contiguous-region regeneration. This suggests a division between two complementary paradigms in expressive symbolic piano research: large-scale corpus construction for broad pretraining and continuation, as exemplified by Aria-MIDI and its use in Aria, and editing-first localized regeneration, as exemplified by PIA’s inpainting formulation (Hadjeres et al., 2021).

7. Limitations, bias, and research implications

The paper acknowledges several limitations, explicit or implicit. Aria-MIDI is piano-only and single-track. Classical piano overwhelmingly dominates the dataset. Discovery is mediated by YouTube’s related-video recommendation system, while metadata depends on uploader-provided titles and descriptions plus language-model inference. The deduplication procedure is metadata-based and therefore imperfect; repeated uploads or duplicate recordings with different metadata may remain. Quality depends on both Aria-AMT and the audio classifier, and non-solo-piano contamination is reduced but not literally guaranteed absent.

The release itself is constrained. Because using copyrighted works and derivatives such as MIDI transcriptions is legally complex, the authors do not release the original audio or raw copyrighted metadata; they release derived MIDI plus extracted metadata, under CC-BY-NC-SA. The absence of canonical splits and the incompleteness of some metadata fields also mean that Aria-MIDI is not a fixed benchmark in the usual sense.

Within those constraints, the paper presents Aria-MIDI as a large, automatically curated and transcribed corpus of solo-piano performance MIDI whose main methodological contributions are language-model-guided discovery, source-separation-distilled pruning and segmentation, robust transcription, and structured metadata extraction at scale. A plausible implication is that the dataset’s principal value lies less in any single benchmark number than in enabling researchers to construct cleaner, deduplicated, style-filtered, or score-thresholded subsets suitable for different symbolic modeling objectives (Bradshaw et al., 21 Apr 2025).