Contact-Anchored Policies: Contact Conditioning Creates Strong Robot Utility Models

Abstract: The prevalent paradigm in robot learning attempts to generalize across environments, embodiments, and tasks with language prompts at runtime. A fundamental tension limits this approach: language is often too abstract to guide the concrete physical understanding required for robust manipulation. In this work, we introduce Contact-Anchored Policies (CAP), which replace language conditioning with points of physical contact in space. Simultaneously, we structure CAP as a library of modular utility models rather than a monolithic generalist policy. This factorization allows us to implement a real-to-sim iteration cycle: we build EgoGym, a lightweight simulation benchmark, to rapidly identify failure modes and refine our models and datasets prior to real-world deployment. We show that by conditioning on contact and iterating via simulation, CAP generalizes to novel environments and embodiments out of the box on three fundamental manipulation skills while using only 23 hours of demonstration data, and outperforms large, state-of-the-art VLAs in zero-shot evaluations by 56%. All model checkpoints, codebase, hardware, simulation, and datasets will be open-sourced. Project page: https://cap-policy.github.io/

• The paper presents a novel framework that conditions robot actions on precise 3D contact anchors rather than symbolic language, leading to enhanced task accuracy.
• It employs synchronized RGB-D data, automated hindsight contact annotation, and a vector-quantized behavior transformer to outperform state-of-the-art baselines.
• It demonstrates robust cross-embodiment transfer and rapid policy improvement through simulation-in-the-loop, paving the way for efficient multi-step robotic tasks.

Contact-Anchored Policies: Conditioning Manipulation through Physical Contact

Introduction

"Contact-Anchored Policies: Contact Conditioning Creates Strong Robot Utility Models" (2602.09017) presents Contact-Anchored Policies (CAP), a novel framework for robot control that replaces traditional language-based task conditioning with physical contact information. The work addresses critical limitations in current vision-language-action (VLA) approaches, notably the abstraction, ambiguity, and inefficiency inherent in language conditioning for robotic tasks that demand precise spatial understanding. CAP introduces an explicit contact anchor—3D physical coordinates at points of robot-object interaction—that conditions policy execution, yielding substantial gains in sample efficiency, transferability across robot embodiments, and zero-shot generalization.

This essay offers a technical exposition of the methodology, empirical evaluations, comparative baselines, and theoretical implications of CAP, examining its impact on utility modeling in robotic manipulation and future research directions.

Methodology: Data, Annotation, and Policy Learning

The core idea of CAP is to reframe policy conditioning around contact anchors rather than symbolic language. Data collection involves a 3D-printed, ergonomic gripper with a mounted iPhone, capturing synchronized RGB-D streams and precise camera trajectories. Three atomic manipulation tasks are targeted: Pick, Open, and Close, totaling 23.1 hours of diverse demonstrations.

Contact annotation is automated via hindsight relabeling: for each demonstration, the contact anchor is identified at the moment of physical interaction, derived as the 3D midpoint between gripper fingers. Preceding frames receive anchors by back-projection through recorded odometry, ensuring the data pipeline directly links contact events to trajectories.

During training, a Vector-Quantized Behavior Transformer (VQ-BeT) is conditioned on both visual observations (ResNet-50/MoCo-pretrained RGB features) and contact anchor embeddings. Actions consist of end-effector pose deltas and gripper states. At inference, the initial contact anchor is acquired either through human input (pixel click) or a vision-LLM (VLM) conditioned on a natural language prompt. Execution relies on continuous tracking of the contact anchor as the robot manipulates objects, maintaining precise spatial conditioning throughout the trajectory.

Figure 1: The CAP pipeline, from contact-anchored trajectory annotation in training to anchor-conditioned action prediction at inference, leveraging user input or a VLM for contact selection.

Rapid Policy Iteration: Simulation-in-the-Loop with EgoGym

To facilitate efficient policy improvement, EgoGym is introduced—a lightweight simulation environment prioritizing scene diversity and fast iteration over visual realism. It supports three tasks (Pick, Open, Close) with procedurally generated objects, articulated elements, and distractors. EgoGym is embedded directly into the training loop to provide more informative signals than standard loss functions, enabling targeted refinement via failure analysis prior to deployment.

Figure 3: EgoGym simulation accelerates failure mode discovery through diversity-focused procedurally generated environments for pick, open, and close tasks.

Zero-Shot Generalization and Embodiment Transfer

Comprehensive evaluations quantify CAP's zero-shot performance across environments, objects, and robot embodiments. On the Stretch 3 robot and five unseen scenes, CAP achieves 83% success in Pick, 81% in Open, and 96% in Close on single trials with oracle contact anchors. With VLM-generated anchors, performance remains statistically indistinguishable, and auto-retry mechanisms mediated by a verifier (GPT-4o) further escalate success rates to 90–98%. CAP checkpoints are deployed on Franka FR3, XArm 6, Universal Robotics UR3e, and even real-time on an iPhone app, demonstrating robust cross-embodiment transfer without fine-tuning; comparative performance is maintained with minimal embodiment-specific adaptation.

Figure 2: The same CAP checkpoint deployed across diverse robot arms and platforms, demonstrating strong cross-embodiment zero-shot generalization.

Figure 6: Empirical robustness of CAP is corroborated by third-party external evaluations across unique physical setups.

Comparative Baseline Performance

CAP is benchmarked against contemporary state-of-the-art approaches:

• AnyGrasp: Classical RGB-D grasp pose prediction.
• $\pi_{0.5\,\text{DROID}}$: A large-scale, task-generalist VLA model fine-tuned on industrial data.
• stretch-open: Modular pipeline for articulated object manipulation.

CAP outperforms all baselines substantially, achieving 23%-56% higher success rates depending on task and embodiment. Notably, CAP eclipses $\pi_{0.5\,\text{DROID}}$ for Pick on Franka FR3 (81% vs. 25%) and AnyGrasp on Stretch (83% vs. 47%).

Long-Horizon Manipulation and Policy Composition

An outstanding claim in the paper is that, contrary to popular assumptions about end-to-end monolithic models, atomic utility models like CAP can be reliably composed via tool-calling orchestrated by a higher-level controller (e.g., a VLM). This is demonstrated through multi-stage, long-horizon tasks (e.g., coffee retrieval, table cleaning) where CAP variants are chained for sequential skills. CAP's compositional policy succeeds in all stages of table cleanup and completes the more complex coffee task in the majority of trials.

Figure 8: High-level controller composes Pick, Open, Close CAPs for long-horizon tasks—affording practical, interpretable multi-step behavior.

Simulation Correlation and Ablation Studies

A key empirical result is the strong alignment between EgoGym simulation and real-world evaluations. Single-blind studies with multiple CAP checkpoints confirm that improving simulated performance yields real gains, validating the value of fast, low-fidelity sim-in-the-loop approaches.

Ablation experiments solidify the centrality of contact anchors: removal of the anchor (RGB-only policy) results in a drastic 38% decrease in Close task success (96% to 58%). CAP is also shown to be robust to distractor objects, sustaining performance where both VLM-based contact selection and VLA policies like $\pi_{0.5}$ show degradation.

Figure 4: CAP maintains high success rates under increasing visual distractor count, unlike VLM-conditioned and baseline VLA models.

Theoretical and Practical Implications

CAP challenges the prevailing paradigm of language-conditioned robot policies by showing that direct physical contact information suffices for broad generalization, significantly reducing data, compute, and parameter requirements. This has substantial implications:

• Utility Modeling: Decoupling atomic skills as modular utility models enables compositional, verifiable, and interpretable control, exposing a new axis for scaling robot intelligence.
• Sample and Compute Efficiency: Demonstrated capacity to train high-performing, general policies from orders of magnitude less human data marks a feasible route for resource-constrained research.
• Cross-Embodiment Robustness: CAP's success in transferring policies across substantially different robot arms with only low-level mechanical adaptation highlights the power of direct contact-based conditioning.
• Simulation Methodology: The finding that low-fidelity, diversity-prioritized simulation provides realistic signals for real-world deployment could recalibrate resource allocation in robot learning pipelines.

Limitations and Future Directions

CAP's atomic skill focus leaves unanswered questions on multi-contact and bimanual tasks, necessitating extensions for representing and predicting rich contact distributions. The interplay between visual and contact modalities in decision making remains opaque, and further investigation could reveal principles underlying efficient multi-modal policy learning. Integrating verifier-guided retry directly into policy optimization (possibly using reinforcement learning) may streamline autonomy in high-risk scenarios.

Conclusion

Contact-Anchored Policies constitute a decisive step toward practical, interpretable, and efficient robot utility frameworks. By anchoring policy conditioning in physical interaction rather than symbolic language, CAP sets a new standard for generalization, sample efficiency, and transferability in robot manipulation. Its modular design, real-to-sim iterative development, and policy composition paradigms promise to expand the tractability of scalable, robust robotic systems under resource limitations. Further research on multi-anchor extension, modality weighting, and integrated verification is anticipated to consolidate CAP as a cornerstone of next-generation robotics.