Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?

Abstract: Multi-fingered robots promise the speed and dexterity of human hands, yet challenging problems such as precise assembly have remained out of reach. These tasks are contact-rich, making data collection for imitation learning difficult, and sparse-reward, making direct exploration with reinforcement learning (RL) intractable. Consequently, prior work has made progress by structuring the problem with specialized grippers, tool attachments, and environment fixtures. In this work, we argue that before a robot can perfect precise assembly, it must first learn to play. We further ask the question: what factors in the process of learning to play matter for precise assembly? We propose Play2Perfect, an RL framework for task-agnostic pretraining through play on diverse objects and goals, which is then perfected on precise assembly. The goal of play is to acquire reusable manipulation priors, such as grasping, in-hand reorientation and pose reaching. Finetuning then adapts this general prior to assembly, focusing exploration on the final contact-rich, high-precision interactions needed for success. We systematically study key design choices in play pretraining, including object diversity, training objective, trajectory diversity, and goal precision. We show that our prior is 33x more sample-efficient than RL training from scratch, even when provided with dense, multi-stage rewards. We demonstrate zero-shot sim-to-real transfer, achieving 60% success on tight insertions with only 0.5 mm contact clearance, and over 50% success on long-horizon multi-part assembly and screwing.

• The paper demonstrates that task-agnostic dexterous play pretraining significantly enhances sample efficiency and robustness in sparse-reward, contact-rich assembly tasks.
• The paper leverages diverse object interactions, precise 6D pose control, and varied trajectory sampling to generate transferable manipulation priors for subsequent policy finetuning.
• The paper validates sim-to-real transfer on multi-fingered robotic systems, achieving robust performance without hand-engineered demonstrations or reward shaping.

Play2Perfect: Analysis of Task-Agnostic Dexterous Play Pretraining for Precise Robotic Assembly

Introduction

"Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?" (2606.26428) presents a systematic investigation of how task-agnostic dexterous play pretraining enables efficient and robust RL-based learning of contact-rich, sparse-reward assembly tasks with multi-fingered hands. The paper critically analyzes the impact of pretraining design choices—object and trajectory diversity, goal structure, and precision—on the quality of manipulation priors and subsequent sample efficiency and performance for precise assembly. By circumventing the need for task-specific demonstrations or reward shaping, the approach provides a scalable pipeline for dexterous robotic assembly that exhibits strong sim-to-real transfer.

Methodology

Framework Overview

Play2Perfect consists of two primary phases: (1) play pretraining, in which a dexterous agent learns general in-hand object manipulation skills using goal-conditioned RL over diverse, procedurally generated objects and random 6D goal trajectories with high-precision success criteria, and (2) assembly policy finetuning, where this agent is specialized via RL to novel CAD-defined sparse-reward assembly tasks requiring contact-rich precision.

Figure 1: Key elements of play pretraining—diverse objects and in-hand 6D pose-reaching with precise goal tolerances—yield robust manipulation priors.

The pretraining tasks emphasize finger-based in-hand reorientation and pose control across varied objects and physical dynamics. RL policies utilize 6D goal-conditioning and reward components for smoothness, grasping, and high-precision pose achievement.

Assembly Task Construction

Given a completed CAD model, assembly-by-disassembly is applied to produce a structured sequence of assembly stages, where each is formulated as a sparse-reward RL task derived directly from CAD-defined part and fixture relationships.

Figure 2: CAD-based disassembly yields sparse, sequential assembly objectives for RL finetuning.

Contact-rich transitions (e.g., aligned pre-insert, axial screwing intermediates) are included, and reward signals are limited to sparse goal completion—to probe the utility of play-derived priors in credit assignment across long-horizon, partially observed manipulation.

Experimental Results

Impact of Play Pretraining

Empirically, the play-pretrained agent achieves over 33$\times$ greater sample efficiency during downstream RL finetuning compared to training from scratch, even with dense hand-engineered rewards. The pretraining provides robust, transferrable manipulation skills: efficient grasping, reliable in-hand reorientation, and stable object control under perturbations.

Figure 3: Dramatic speedup in downstream RL thanks to dexterous priors from play pretraining; training from scratch with sparse or dense rewards remains unable to solve complex assembly tasks.

Figure 4: Finetuned policies develop robust strategies (e.g., stable grasps, recovery behaviors) versus brittle heuristics from scratch training, which collapse under small perturbations.

Ablation and Design Choice Analysis

The authors conduct comprehensive ablations on object diversity, training objective (translation, rotation, or full pose), trajectory diversity, and goal precision.

• Object diversity: Transfer benefits saturate after moderate diversity, but a minimal set impairs robustness.
• Training objective: Only full 6D pose-reaching (translation and rotation) yields an in-hand prior effective for assembly; translation-only policies lack reorientation capability.
• Goal/trajectory diversity: Online sampling and broad goal coverage outperform training on fixed sets.
• Goal precision: Tight tolerances ($\epsilon = 1$cm) are essential; coarse pretraining induces policies incapable of high-precision assembly.

  Figure 5: Only play policies emphasizing accurate in-hand 6D control across diverse objects/trajectories with fine tolerances transfer effectively to assembly.

Is Finetuning Necessary?

The transfer study demonstrates that, while pretrained policies can succeed at loose-tolerance insertions, RL finetuning on sparse assembly rewards is required for mastery of precise, contact-rich tasks. Play-only policies treat contact as disturbance, whereas finetuned policies develop local search and corrective strategies.

Figure 6: Only assembly-finetuned policies succeed at tight-clearance insertions, both in simulation and real-world. Play-only policies degrade rapidly as task precision increases.

Sim-to-Real Transfer and Deployment

Zero-shot sim-to-real transfer is demonstrated on a Sharpa hand/KUKA iiwa platform. CAD-based 6D pose tracking (FoundationPose) and domain randomization enable robust deployment, with the policy receiving object pose and geometry as well as proprioception and goals, and outputting full joint targets for arm and fingers.

Figure 7: At inference-time, the pipeline uses CAD models for real-time pose tracking, goal generation, and grasp region specification, closing the loop between synthetic and real domains.

Qualitative rollouts show adaptive behaviors: closed-loop local search during insertion, reorientation under screw tasks, and recovery/grasp adaptation following drops or contact failure.

Figure 8: Observations and behavior tracking during real-world deployment; the policy aligns observed part pose and sequential CAD-defined goals to direct real-time manipulation.

Figure 9: Tight-clearance insertion is accomplished via tilted approaches and incremental local search/correction before final insertion.

Figure 10: Robust recovery—after object drops, the policy regrasps and retries the assembly without any hand-scripted logic.

Implications and Future Directions

This work clarifies the mechanisms underlying successful transferable dexterous manipulation pretraining:

• Generalization to challenging contact-rich assembly depends crucially on the acquisition of precise, finger-driven 6D in-hand control during play pretraining.
• Sample efficiency and robustness in sparse-reward RL can be dramatically improved by such task-agnostic priors, enabling scalable training pipelines for complex robotic skills.
• The methodology eliminates hand-engineered fixtures, demonstrations, or reward shaping—suggesting avenues for modular skill sequencing and integration into hierarchical task and memory architectures.
• The reliance on precise pose tracking points to vision/perception and tactile integration as key next steps for unstructured, occlusion-prone environments.

Conclusion

Play2Perfect provides a rigorous, quantitative account of what makes dexterous play pretraining effective for complex, precise robotic assembly. The study exposes the limitations of baseline RL (even with shaped rewards), quantifies the necessary play curriculum ingredients, and demonstrates robust, efficient sim-to-real transfer on high-DOF hands. The architecture and findings specify a generic, scalable, demonstration-free path toward generalist manipulation policies, informing both the design of dexterous foundation models and new RL training regimes for contact-rich, high-precision tasks.