- The paper introduces a modular framework that combines autonomous offline data collection and deep reinforcement learning for force-sensitive segments with pose estimation and motion planning for free-space motion.
- The method achieves 94–98% success across four real-world manipulation tasks, averaging 96% with 2–2.5 hours of data collection versus 55% for the strongest baseline.
- The approach maintains 85–95% success under out-of-distribution conditions and reduces peak contact forces by 49%, but requires precise hardware, reliable pose estimation, and tasks with separable critical segments.
Overview
This paper addresses a structural weakness of end-to-end robot learning for contact-rich manipulation: data collection and policy learning treat all phases of a task uniformly, even though failures concentrate in a short, force-sensitive critical segment while the remainder of the task is comparatively easy free-space motion. The authors propose a compositional framework that allocates dense robot data collection and offline deep reinforcement learning (DRL) exclusively to the critical segment, while delegating free-space motion to off-the-shelf pose estimation and motion planning. On four real-world tasks, the method achieves an average success rate of 96% using only 2–2.5 hours of autonomous data collection, versus 55% for the strongest baseline, and retains high success rates under out-of-distribution (OOD) conditions where end-to-end policies collapse.
Method
The framework consists of three components: autonomous offline data collection at the critical segment, offline policy learning, and sequential composition with planning at deployment.
Autonomous data collection. A single demonstration is recorded via kinesthetic teaching with the scene layout fixed (small 3D-printed fixtures hold parts in place where needed). The demonstration is replayed up to the critical segment, whose extent is defined by the measured uncertainty of the pose estimator. Data is then gathered with a mixed greedy-random policy that selects uniform random actions with probability p and a greedy action minimizing distance to the (randomized) goal state with probability $1-p$. A safety filter constrains the end-effector to a sphere around the critical segment. Episodes terminate by timeout or task completion, with a sparse reward based on distance to the final demonstration state. Hybrid force-position control is applied along the insertion axis during force-sensitive interaction, both during collection and deployment.
Policy learning. The authors build on SAC with RLPD-style sample-efficiency modifications: layer normalization in the critic and Randomized Ensembled Double Q-learning with NQ=10 Q-functions and random subsets of size 2 for TD-targets. Because the setting is purely offline, the entropy coefficient is fixed, and the asynchronous online rollout infrastructure of SERL/RLPD is removed entirely — a deliberate architectural simplification. Training takes roughly one hour on a single RTX 5090. Notably, the sparse-reward Q-function doubles as a vision-based success classifier at deployment, eliminating the need for a separately trained reward classifier.
Full-task execution. Free-space motion is planned via waypoints defined relative to 6D poses obtained from SAM3 segmentation and FoundationPose estimation. The learned policy is triggered on contact (an F/T threshold, which the authors find more reliable than distance- or pose-uncertainty-based switching), and terminates via a Q-function threshold with hysteresis-style detection (λhigh=0.8Rsuccess, λlow=0.6Rsuccess) to avoid false negatives.
Experimental results
The evaluation covers shelf stocking (deformable cardboard items), Lego stacking, and an industrial fan-cover assembly task from a real production line, plus a harder fan-cover variant trained with scene distractors. Policies receive a cropped wrist-camera image (DINOv2 embeddings), F/T measurements, controller error, Cartesian velocity, and the previous action, at 15 Hz over 1 kHz CRISP torque controllers.
The headline results are strong. With matched wall-clock data-collection budgets (2–2.5 hours), the method reaches 94–98% success across tasks (96% average over 50 trials each), while DiTFlow averages 37%, Diffusion Policy 36%, finetuned π0.5 20%, and pure motion planning plus pose estimation 55%. HIL-SERL performed poorly (6–10%) on these long-horizon tasks, with high training variance attributed to online policy optimization. The failure analysis is diagnostic: end-to-end baselines exhibit high partial success rates (e.g., DP at 95% partial success on average) but low full success, confirming that failures concentrate at the critical segment rather than at grasping or transport.
Two further results deserve emphasis. First, on OOD evaluations (distractors, repositioned objects, changed backgrounds), the method retains 85–95% success, while all end-to-end baselines fall to 0–25% — the locally cropped visual input substantially narrows the OOD scenario space. Second, the modular structure permits segment-specific control parameters: the planned push-down uses forces calibrated to the demonstration, reducing maximum applied forces by 49% (−4.7 N on average) relative to baselines, which matters for the deformable cardboard and PBT/PC parts. Ablations in MuJoCo show that policies require dense critical-segment coverage: success rates are low for p≤0.3 (near-imitation) and for pure exploration (p=1), with the practical sweet spot at 0.6≤p≤0.85. Dataset requirements scale with pose-estimation uncertainty (400 rollouts at 6 mm, 700 at 12 mm), and combining wrist F/T sensing with the wrist camera yields 100% simulated success.
The novice-teleoperator comparison provides direct evidence for the motivation: DiTFlow trained on novice data drops to 22–34% success, with novice demonstrations exhibiting multi-modality, higher speed variance (CV=0.285 vs. 0.185), and longer trajectories. This substantiates the claim that teleoperation-dependent pipelines inherit operator skill as a bottleneck, which the autonomous scheme removes.
Limitations and open questions
The paper is explicit about several constraints. The approach presupposes tasks with a sequentially separable free-space/critical-segment structure; for tasks such as garment folding, segment boundaries are hard to define, and the authors point to LLM-driven switching as an open direction. Success depends on hardware that reaches waypoint poses precisely, whereas end-to-end methods can absorb hardware inaccuracy through data scale — so the comparison partially trades data robustness for mechanical precision. Off-the-shelf pose estimation struggles with reflective surfaces, exotic objects, and occlusion, though the authors note it can be fine-tuned from images and CAD alone. On the harder fan-cover variant, training-time augmentation raised partial success to 50% but full insertions remained at 5%, and the authors state that substantially more diverse data would likely be required for full generalization there. Remaining failures stem from distractors placed close to the insertion point, which push the locally cropped image out of distribution, and from hardware artifacts such as parts sticking to the gripper. The Q-threshold $1-p$0 also requires per-deployment tuning.
Conclusion
The paper argues that the gap between learned manipulation policies and high-precision requirements is partly a data-allocation problem rather than purely a scale problem. By concentrating autonomous data collection and offline DRL on the contact-critical segment and relying on data-free perception and planning elsewhere, the method achieves 96% average success with 2–2.5 hours of data, robust OOD performance, and 49% lower contact forces — at the cost of requiring fixed scenes, precise hardware, and tasks with definable segment boundaries. The central open question it leaves is how to automate segment definition and policy switching for tasks without clean sequential structure.
