RP1M++: Robotic Piano Dataset

• RP1M++ is a large-scale dataset of over one million fixed-length robotic piano-playing trajectories aggregated from numerous specialized RL agents.
• The dataset leverages DAgger-based replay relabeling and optimal transport methods to optimize precision in fingering and key actuation.
• Its structured organization, standardized evaluation protocols, and diverse classical repertoire form a robust benchmark for robotic imitation learning.

RP1M++ is a large-scale, high-fidelity dataset of dexterous robotic piano-playing trajectories, produced by aggregating policy rollouts from numerous specialist reinforcement learning agents, each trained to master distinct classical music pieces. Designed to enable scalable, human-demonstration-free learning in bimanual robotic piano performance, RP1M++ supplies over one million fixed-length episodes, providing a critical resource for the development, training, and benchmarking of both specialist and generalist robotic agents in contact-rich, high-dimensional music-related manipulation tasks (Chen et al., 4 Nov 2025).

1. Dataset Composition and Structure

RP1M++ consists of more than 1,000,000 trajectories, each an unbranched episode of 550 timesteps (27.5 seconds at a 0.05 s control interval, with negligible length variance). These trajectories are sourced from 2,091 music clips representing approximately 1,053 unique songs. Primary data sources are the PIG dataset (∼300 expert-annotated MIDI songs) and a curated subset (1,788 pieces) of the GiantMIDI-Piano corpus. Clip segmentation strictly adheres to 550-timestep boundaries to enforce homogeneous episode duration.

The dataset's key-press distribution is centered, with middle piano keys significantly more frequently actuated than peripheral keys. White keys constitute 65.7% of all key presses. Across all timesteps, the number of concurrent key presses per timestep forms an approximately Gaussian profile, with 90.7% of steps exhibiting between 1,000 and 4,000 total active key presses throughout the dataset. Quality of agent demonstration is tracked per trajectory: 79% of 2,089 RL specialists achieve an F1 score ≥ 0.75, and 99.9% exceed 0.5, enabling robust filtering for downstream usage.

2. Data Collection Methodology

Piano playing in RP1M++ is formalized as a finite-horizon Markov Decision Process (MDP) $(\mathcal{S}, \mathcal{A}, \mathcal{T}, R, \gamma)$. The observation space $\mathcal{S}$ comprises 1,144 real-valued dimensions, integrating robot proprioception, goal vectors (both current and lookahead), key/pedal states, and hand kinematics. The action space $\mathcal{A}$ is 39-dimensional: joint positions, forearm poses, and sustain pedal control.

Trajectory rewards combine optimal-transport-based fingering, correct/incorrect key presses, precise sustain pedal usage, avoidance of self/piano collisions, and actuation energy efficiency: $$r_t = r_t^{OT} + r_t^{Press} + r_t^{Sustain} + \alpha_1 r_t^{Collision} + \alpha_2 r_t^{Energy}$$ where $\alpha_1 = 0.5$, $\alpha_2 = 5 \times 10^{-3}$. Notably, fingering is autonomously optimized via a discrete Optimal Transport (OT) assignment between active keys and robot fingers, solved using a Jonker-Volgenant variant. The resulting OT cost $d_t^{OT}$ yields a reward term: $$r_t^{OT} = \begin{cases} \exp(c \cdot (d_t^{OT} - \delta)^2), & \text{if } d_t^{OT} \geq \delta \ 1.0, & \text{otherwise} \end{cases}$$ with $\delta = 0.01$ and $c<0$.

The rollout capture protocol utilizes DAgger-based replay relabeling: after DroQ specialist RL training (8M steps), a student agent collects episodes using its policy, which are then relabeled offline by the frozen specialist expert. This pipeline broadens state-action visitation well beyond what pure expert rollouts yield.

3. Trajectory Encoding and Modalities

Each RP1M++ trajectory encodes the following per-timestep entries for $\mathcal{S}$0:

• Observation vector $\mathcal{S}$1
• Expert action $\mathcal{S}$2 (robot joint targets, pedal)
• Scalar reward $\mathcal{S}$3
• Next observation $\mathcal{S}$4

Metadata—such as piece ID, random seed, and expert F1 score—is attached per trajectory. Data are stored as compressed HDF5 or NumPy npz files, with each file containing arrays (“obs”, “act”, “rew”, etc.) as well as a JSON or YAML metadata header specifying “piece_name”, “expert_f1”, and “timing” (0.05 s).

4. Dataset Organization, File Structure, and Access

RP1M++ employs a hierarchical directory structure. Music pieces are grouped by folder, named in the format “Composer_Title_Opus_ClipID”, containing trajectory files (“traj_{i:03d}.npz”). A top-level metadata.csv indexes all clips and trajectories, holding columns: piece_id, piece_name, clip_id, agent_f1, num_traj, and train/val/test split.

Level	Naming Convention	Contents
Piece Dir	Composer_Title_Opus_ClipID/	Trajectories
Trajectory	traj_{i:03d}.npz	obs, act, rew
Metadata	metadata.csv	Descriptive data

A practitioner can batch load, filter by expert F1, and align trajectories via uniform time steps.

5. Applications, Benchmarking, and Evaluation Protocols

RP1M++ is tailored for data-driven imitation learning, supporting both specialist (single-song) and generalist (multi-song) piano-playing agents. Canonical usage includes training policies via behavior cloning, Diffusion Policy, or Flow Matching, with evaluation protocols specifying:

• In-distribution generalization: Test F1 recall on the training songs.
• Out-of-distribution generalization: Zero-shot F1 recall on 100 previously unseen GiantMIDI-Piano clips.

Empirical baselines:

• RL-trained specialists (per-song): F1 $\mathcal{S}$5 0.8–0.9 using OT-based fingering.
• Multi-song Flow Matching Transformer: mean in-distribution F1 rises from $\mathcal{S}$6 0.8 (12 songs) to 0.86 (900 songs), while out-of-distribution F1 increases from 0.45 to 0.55 over the same range.
• Diffusion Policy and U-Net flow matching models underperform FMT, particularly in OOD settings.

Recommended evaluation relies on the RoboPianist-ETUDE-12 set for in-distribution and 100 GiantMIDI test pieces as the OOD benchmark. Practitioners are advised to:

1. Prefer clips with expert F1 $\mathcal{S}$7,
2. Employ time-aligned loading and batching,
3. Use per-timestep key-press F1 to gauge accuracy,
4. Benchmark policies under matched train/test splits against standardized protocols.

6. Significance and Impact

By systematically aggregating carefully relabeled rollouts from over 2,000 independently trained agents, RP1M++ provides an unprecedented corpus for the study of dexterous robotic manipulation in music. The design—singularly large scale, uniform temporal granularity, and breadth of coverage across the classical piano repertoire under copyright permissions—distinctively positions RP1M++ as a foundational benchmark for scalable dexterous learning, imitation learning algorithms, and robust state-action generalization in high-dimensional contact-rich domains (Chen et al., 4 Nov 2025). Standardized metrics, protocol definitions, and rigorous agent performance tracking facilitate reproducible research and comparative analysis.