Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization

Abstract: Reinforcement learning can train bimanual dexterous hands to play piano in physics simulation with high note accuracy, but for high-DoF dexterous hands, relying solely on task rewards or IK inversion often leads to unnatural postures and joint overextension. We propose \textit{Adversarial Posture Regularization (APR)}. It avoids expensive, song-aligned expert demonstration data and instead uses a small amount of casual human playing data. By matching the distribution of the posture of the policy with the human prior through an adversarial objective, APR encourages more human-like hand shapes. Meanwhile, we collect and release unstructured hand motion data of piano playing using a consumer-grade Meta Quest 3, and retarget the key motion information to the Shadow Hand. Finally, we achieve significantly better performance than prior methods on all three human-likeness metrics (cPSI, BSE, and FAC) as well as in visual quality. Project repository: https://github.com/APRProject/APRPianist.

• The paper presents APR, an adversarial method that aligns robotic hand postures with human demonstrations to mitigate reward hacking and biomechanical artifacts.
• It combines a shared-weight discriminator and VR-captured demonstration data to efficiently regularize high-dimensional hand kinematics.
• APR achieves notable reductions in joint strain, synergy errors, and finger arc discontinuities while maintaining high keystroke accuracy on multiple musical tracks.

Adversarial Posture Regularization for Human-Like Dexterous Piano Playing

Motivation and Problem Setting

Dexterous piano playing with robotic hands requires simultaneous bimanual coordination and accurate high-frequency keystroke execution, posing severe challenges in high-dimensional (high-DoF) control. Existing RL and IK-based methods achieve task-centric rewards, such as keystroke F1 scores, but often result in biomechanically implausible postures, including hyperextension and "zombie hand" artifacts due to reward hacking and kinematic redundancy. Prior attempts to mitigate unnatural postures via endpoint-focused 2D tracking or classical IK solvers inadequately constrain intermediate joints, leading to unnatural solutions.

Imitation-based approaches are hamstrung by the cost and complexity of collecting state-aligned, full-pose demonstration data for complete musical pieces. This paper addresses the gap in robust, scalable posture regularization for dexterous piano playing by proposing Adversarial Posture Regularization (APR), which matches the distribution of policy-generated hand postures with casual human demonstration data via an adversarial objective.

Figure 1: Qualitative comparison of hand postures—APR reduces biomechanical distortions and maintains natural finger arches, avoiding "zombie hand" artifacts.

Methodology: Adversarial Posture Regularization

APR regularizes hand kinematics by adversarially matching state transition distributions from policy rollouts with reference transitions from short, unstructured human demonstrations. Data collection utilizes consumer-grade VR hardware (Meta Quest 3) for egocentric hand motion capture, retargeted from human to Shadow Hand kinematics via bone vector and palm frame mapping to achieve morphology-invariant correspondence.

Figure 2: Egocentric photo-capture setup for hand demonstration collection.

The reference feature space, crucial for APR, is constructed per hand using normalized joint angles, angular velocities, and fingertip heights (41 dimensions), yielding posture descriptors robust against both global motion and hardware tracking noise. State transition pairs across both hands are pooled for adversarial training, leveraging a shared-weight bimanual discriminator architecture.

The discriminator is implemented as a fully connected MLP trained online with the Least-Squares GAN (LSGAN) objective, outputting $+1$ for expert transitions and $-1$ for policy transitions, regularized with a gradient penalty for smoothness near expert data. This robust distributional matching—unlike per-frame tracking—eliminates the need for precise alignment or song-specific reference data.

The hybrid reward formulation combines task accuracy (pressing correct keys) and style naturalness (APR output), allowing flexible trade-offs. The overall system employs PPO for policy optimization, using a residual learning approach atop an IK prior.

Figure 3: Data pipeline—human hand demonstrations retargeted to Shadow Hand joint angles, APR uses LSGAN reward in PPO outer loop.

Experimental Evaluation and Results

Task Performance

APR achieves F1 scores comparable to the PianoMime baseline (2D fingertip tracking) across diverse, challenging musical tracks, indicating that soft posture regularization does not hinder keystroke accuracy or task completion. Final performance gaps are consistently minimal (approx. $0.01$-$0.02$), as shown in learning curve plots.

Figure 4: F1 score learning curves—APR and baseline converge comparably with negligible final performance differences across four tracks.

Motion Naturalness

Three quantitative metrics rigorously assess motion naturalness: cPSI (joint strain), BSE (biomechanical synergy error), and FAC (finger arc continuity), all lower-is-better. APR outperforms PianoMime on all metrics, reducing posture strain (cPSI) by up to $36.6\%$, synergy violations (BSE) by up to $62.9\%$, and finger arc discontinuities (FAC) by up to $48.5\%$ (notable improvement on redundant Shadow Hand topology). These improvements are consistent across left and right hands and all test songs.

Visual inspection further confirms APR's elimination of unnatural hyperextension and its maintenance of natural finger motion during dynamic execution.

Figure 5: Qualitative results—green circles indicate APR-driven natural, human-like hand motions; red circles denote artifacts in baselines.

Data Efficiency

APR requires only $\sim$19 seconds of casual, unaligned demonstration data (one short VR session), universally reused for all songs. In contrast, DeepMimic-style or per-song tracking baselines require costly, labor-intensive, song-aligned full-pose tracking. APR’s distributional approach substantially increases scalability and generalizability to new musical pieces and manipulation tasks.

Implications and Future Directions

APR demonstrates that adversarial distributional matching with low-fidelity, unstructured reference data suffices to constrain high-DoF hand posture, mitigating reward hacking and producing biomechanically plausible kinematics—even with challenging task priors and redundant actuation. This advances the practical deployment of dexterous manipulation agents in real-world, multi-task settings where full-pose demonstration data is prohibitively expensive.

Practically, APR can be adopted in other dexterous manipulation tasks where kinematic redundancy causes reward exploitation, and physical plausibility is important. Theoretically, the success of distributional matching over frame-aligned tracking suggests that further integration of GAN-based and self-supervised regularizers could supersede traditional imitation learning for high-DoF tasks. Limitations include the sensitivity of LSGAN hyperparameters and occasional posture degradation during extreme timing demands; future work may entail adaptive style weighting or alternative stable generative priors.

Conclusion

Adversarial Posture Regularization offers a scalable, data-efficient mechanism to enforce human-like kinematics in high-DoF piano playing. By matching policy motion distributions to casual human priors via adversarial training, APR eliminates reward hacking and biomechanical artifacts, achieving strong naturalness gains without compromising task performance. This paradigm positions adversarial posture regularization as a viable solution for dexterous robotic control tasks requiring both functional and stylistic fidelity (2606.23848).