- The paper introduces ADEPT, a two-stage reinforcement learning framework that pre-trains generic object-reposing skills and adapts them to contact-rich tasks such as peg insertion using behavior-cloning distillation, critic warm-up, and low-learning-rate PPO.
- ADEPT prevents the catastrophic collapse seen in direct PPO fine-tuning, reaches the hardest training stage in all five seeds, and reduces new-task training to 3 billion environment steps after 8 billion steps of reusable pre-training.
- Real-world students complete long-horizon pick–reorient–insert behaviors in 5–10 seconds, achieving up to 8/10 success with visuo-tactile sensing compared with 3/10 using vision alone and outperforming slower parallel-jaw pipelines.
Overview
ADEPT is a reinforcement learning (RL) framework for learning sim-to-real transferable dexterity on high degree-of-freedom (DoF) arm–hand systems directly from raw visuo-tactile perception. The framework separates the problem into a pre-training stage, in which a policy learns generic object reposing skills (reaching, grasping, lifting, in-hand reorientation, transport), and a post-training stage, in which that pretrained behavior serves as an initialization for contact-rich downstream tasks such as peg insertion. The central empirical claims are threefold: the pretrained policy zero-shots the reposing segment of downstream tasks it never saw during training; naive RL fine-tuning of this pretrained policy collapses its behavior within a few PPO updates; and a structured post-training recipe—behavior-cloning (BC) distillation, critic warm-up, and conservative on-policy updates—prevents this collapse while enabling task-specific adaptation. Deployed zero-shot, distilled students solve long-horizon pick–reorient–insert tasks at 5–10 s per trial, a 2×–14× execution-time speedup over the FMB parallel-jaw pipeline.
Framework and Method
The framework operates over two Markov decision processes sharing the same action space A=[−1,1]nq (nq=23 for Kuka-Allegro): a pretraining MDP for reposing and a post-training MDP whose observation space extends the pretraining one with receptacle pose and object–receptacle contact forces. Policies are trained with PPO under an asymmetric actor-critic, with the critic consuming privileged simulator state.
Pre-training uses a generic reposing task over 16 primitive shapes (cylinders, cuboids, spheres, cones) at randomized scales, trained with Automatic Domain Randomization (ADR) as an online curriculum, Population-Based Training (PBT) over PPO hyperparameters across 16 parallel policies, and gravity annealed from zero to −9.81m/s2. Objects are represented by point clouds, which the authors note improves zero-shot generalization to downstream tasks.
Post-training proceeds in three steps: (1) BC distillation of the pretrained actor into a downstream actor with the new observation space (40k iterations); (2) critic warm-up, where the actor is frozen for ~20 PPO iterations (~1M environment steps per GPU with 4096 environments) while a fresh downstream critic calibrates against the insertion reward; and (3) conservative PPO with the actor learning rate reduced from 10−3 to 10−5 and the PPO clip tightened from 0.2 to 0.05, while the critic learning rate stays fixed at 5×10−5.
Distillation converts state-based post-trained teachers into deployable students via DAgger-style BC with a Mahalanobis-weighted action-matching loss plus an auxiliary 8-keypoint object pose prediction head weighted at waux=20. A two-stage student curriculum first distills the pretrained reposing teacher through a perception-heavy surrogate task (peg detection, tracking, upright reorientation from RGB alone), then continues distillation from the downstream insertion teacher. This isolates perception learning from policy refinement; the single-stage baseline fails entirely in real deployment (0/10 on both pegs).
Full Cspace Geometric Fabrics
Between the policy and the robot sits a geometric fabric—a second-order dynamical system on configuration space providing collision avoidance, joint-limit repulsion, damping, and speed control with stability guarantees. Unlike DextrAH-G/DextrAH-RGB, which restrict hand control to a 5D PCA grasp subspace plus a 6D palm pose target, ADEPT drives the fabric in the full joint configuration space: the policy emits per-joint relative deltas scaled by δmax=0.1 rad, mapped to a Cspace target consumed by the fabric's forcing term. This exposes all 23 (or 29) DoF of kinematic dexterity needed for finger gaiting and contact-rich manipulation, at the cost of a harder learning problem. The identical fabric instance runs in simulation and on hardware, removing the control-stack gap from the sim-to-real problem; fabric parameters themselves (damping gain, speed-control energy target, velocity attenuation) are annealed via ADR rather than held fixed.
Simulation Results
The pretrained teacher generalizes beyond its training distribution: despite training only on primitives, the Kuka-Allegro teacher achieves success rates of 0.76 on FMB pegs and 0.77 on 152 VisDex objects, matching or slightly exceeding its 0.73 in-distribution rate. Zero-shot performance on FMB peg insertion degrades monotonically with ADR level—from 98.6% at ADR 0 to above 50% through ADR 35, but to 0% at the actual insertion goal (ADR 50), where receptacle contact is out-of-distribution. Post-training therefore begins at ADR 20, where only the contact-rich insertion segment remains to be learned.
The comparison against alternatives is stark. Training from scratch is highly seed-sensitive, with most seeds plateauing below ADR 6 after roughly 9B environment steps. Direct PPO fine-tuning of the pretrained teacher drives success to zero within a few updates. The authors attribute this collapse to a miscalibrated critic: value estimates tuned to the reposing reward produce unreliable advantages under the insertion reward, so policy gradients push the actor off its pretrained distribution faster than the critic can recalibrate, creating a compounding degradation loop. ADEPT post-training reaches ADR 50 in 3B steps atop the 8B-step pretraining, and because the pretraining cost amortizes across tasks, the marginal cost per new task is 3B steps.
The ablation study over five seeds per variant identifies which components matter:
| Variant |
LR |
Clip |
ADR 50 reached |
Train SR (%) |
Time |
| ADEPT (full) |
10−5 |
0.05 |
5/5 |
46.0 ± 1.1 |
19.9 h |
| No BC |
10−5 |
0.05 |
4/5 |
38.5 ± 1.2 |
35.2 h |
| No warm-up |
nq=230 |
0.05 |
4/5 |
28.4 ± 3.7 |
20.2 h |
| Standard PPO |
nq=231 |
0.20 |
0/5 |
0.0 |
Collapse |
| High LR only |
nq=232 |
0.05 |
0/5 |
0.0 |
Collapse |
| Loose clip |
nq=233 |
0.20 |
5/5 |
46.8 ± 2.0 |
17.6 h |
| Direct FT + KL penalty |
nq=234 |
0.20 |
0/5 |
0.0 |
Collapse |
Three findings stand out. First, the reduced actor learning rate is strictly necessary: every nq=235 variant collapses at ADR 20, including one augmented with a KL regularization penalty toward the pretrained policy, which still yields 0% across all five seeds. Second, the tightened PPO clip contributes little—the loose-clip variant matches or exceeds the full recipe—contradicting the intuition that conservative clipping is the key stabilizer. Third, without both BC and warm-up, every seed stalls even at the low learning rate, regardless of whether new observation dimensions are injected by input-layer remapping or append-only extension, indicating the failure is not an artifact of network surgery.
On behavior quality, reposing pretraining produces emergent natural grasps (multi-contact finger wraps) that from-scratch policies do not develop; from-scratch policies converge to functionally adequate but unnatural grasps (e.g., two-finger or single-finger-wrapped configurations). On FMB, post-training primarily selects and refines pretrained grasp modes, filtering out task-misaligned behaviors such as bottom-up grasps. The dish-rack task demonstrates that post-training can also learn genuinely novel behaviors: no pretrained grasp succeeds on the plate, yet starting from zero-shot reaching and plausible failed grasp attempts, downstream RL discovers successful plate grasping and a flip-and-regrasp strategy for upside-down initializations. The authors' stated assumption here is that the prior need not solve the downstream behavior—it suffices to place the policy in a productive region of parameter space—while conceding that tasks requiring qualitatively different skills may require broader pretraining coverage.
Real-World Results
Deployment uses two platforms: a 23 DoF Kuka iiwa7 + Allegro with two RGB cameras, and a 29 DoF Flexiv Rizon + Sharpa with two RGB cameras and five vision-based fingertip tactile sensors rendered as TacMap penetration-depth maps shared between simulation and hardware, spatially anchored via SaTA-style FiLM conditioning on forward-kinematics fingertip positions. Students run on an external GPU workstation issuing Cspace targets at 60 Hz over ZMQ, with a 1 kHz admittance controller underneath.
Zero-shot real-world results over ten trials per configuration show cumulative per-stage success:
| Modality |
Robot |
Task |
Reach |
Grasp |
Lift |
Reorient |
Align |
Insert |
| Vision |
Kuka-Allegro |
FMB Star |
10/10 |
9/10 |
8/10 |
8/10 |
7/10 |
5/10 |
| Vision |
Kuka-Allegro |
FMB Sq/Rd |
10/10 |
8/10 |
6/10 |
4/10 |
3/10 |
3/10 |
| Vision |
Flexiv-Sharpa |
FMB Sq/Rd |
10/10 |
7/10 |
5/10 |
3/10 |
3/10 |
3/10 |
| Visuo-tactile |
Flexiv-Sharpa |
FMB Sq/Rd |
10/10 |
10/10 |
10/10 |
9/10 |
8/10 |
8/10 |
| Vision |
Kuka-Allegro |
Dish |
10/10 |
10/10 |
8/10 |
7/10 |
6/10 |
6/10 |
Tactile sensing is decisive on Flexiv-Sharpa (8/10 vs. 3/10 overall). The vision-only failure mode is not grasp execution but grasp confidence: unable to verify contact, the policy reopens its hand, drops the object, and loops between grasping and regrasping, cascading through later stages. With tactile input, grasping and lifting succeed in every trial. The asymmetric square-and-round peg—the most difficult geometry in FMB—produces the largest per-stage drops at lifting and reorientation due to unstable point contacts on its round leg. All trials complete in 5–10 s as a single continuous behavior, versus 20–70 s for the FMB parallel-jaw pipeline, which requires external fixtures and multi-stage regrasping. To the authors' knowledge, this constitutes the first demonstration of zero-shot sim-to-real RL pick–reorient–insert on high-DoF arm–hand systems from raw visual or visuo-tactile perception without demonstrations or pose estimators.
Limitations
Perception remains the primary bottleneck. Real-world failures frequently coincide with incorrect estimates of the asymmetric peg's orientation under occlusion, and grasp instability arises from the small contact patch between rounded Allegro fingertips and the rounded peg side during fast arm motion. Tactile sensing mitigates contact ambiguity but does not resolve object-centric perception under occlusion, which the authors leave open, suggesting wrist-mounted cameras and tactile sensing on additional platforms as partial remedies. Two structural assumptions also bound the framework's applicability: the method presumes the pretrained policy solves some portion of the downstream task zero-shot, so post-training has a useful starting state distribution; and the Stage-1 vision distillation surrogate task is hand-designed, with automation left unaddressed.
Conclusion
ADEPT establishes that a generic reposing pretraining task, combined with a structured post-training recipe of actor distillation, critic warm-up, and low-learning-rate PPO, makes full-Cspace dexterous RL tractable and transferable across embodiments. The ablations sharpen the prescription: learning-rate reduction—not KL penalties or tight clipping—is what prevents catastrophic collapse during transfer, while BC and critic warm-up improve success rate and adaptation speed. The open question the paper poses for itself is how far priors built on richer pretraining distributions—in-hand manipulation, tool use, clutter, bimanual coordination—can transfer to downstream tasks differing more substantially from pretraining.
