ContactMimic: Humanoid Object Interaction via Contact Control

Abstract: Keypoint tracking alone is insufficient for object interaction tasks such as sitting on a chair, wiping a board, or pushing furniture, where the robot can reach the correct pose without making meaningful physical contact with the object. We present CONTACTMIMIC, a learning framework that tracks explicit partlevel binary contact commands alongside keypoint trajectories. CONTACTMIMIC is made possible through the use of contact-following rewards and a trajectory augmentation scheme aimed at breaking the correlations between keypoint trajectories and contact labels. The resulting policy successfully decouples contact behavior from keypoint geometry, and achieves precise physical contact as well as contact-controllability (produce or suppress contact during deployment as desired). Simulation experiments across 10 diverse human-object interaction motions confirm that CONTACTMIMIC exhibits contact controllability that enables it to complete manipulation tasks without task-specific rewards, while also outperforming keypoint-only trackers on contact-relevant tasks. Ablations confirm the necessity of the proposed trajectory augmentation scheme and sim2real deployment validates contact controllability in the real world across 5 different motions. Video results are available on https://lixinyao11.github.io/contactmimic-page/.

• The paper presents a novel contact-conditioned policy that integrates explicit contact commands with keypoint tracking to achieve reliable humanoid object interaction.
• It employs a unique reward structure and data augmentation pipeline that decouples geometric cues from contact, ensuring robust and controllable physical interactions.
• Simulation and real-world experiments demonstrate that explicit contact tuning significantly improves performance in tasks requiring precise contact modulation.

ContactMimic: Explicit Contact Control for Humanoid Object Interaction

Overview and Motivation

Physical contact with objects is fundamental to effective humanoid loco-manipulation, yet existing whole-body control frameworks overwhelmingly condition only on geometric keypoint trajectories. This neglect of explicit contact specification leads to failures in interaction-intensive tasks—such as sitting, wiping, or object manipulation—where correct body-object contact, not just pose, determines task success. "ContactMimic: Humanoid Object Interaction via Contact Control" (2607.08742) addresses this critical gap by proposing a contact-conditioned policy architecture trained to track both keypoint trajectories and explicit, per-body-part binary contact labels. The central thesis is that contact commands can be toggled at test-time, affording fine-grained, interpretable control over physical contact execution, irrespective of geometric trajectory.

Figure 1: Overview of the ContactMimic pipeline: Human-Object Interaction clips are retargeted, augmented, and used to train a contact-conditioned policy; contact commands can be toggled at inference.

Methodology

ContactMimic introduces two principal technical innovations: (1) a reward structure that robustly enforces both tracking and high-fidelity contact matching, and (2) a dataset augmentation pipeline specifically designed to break the entrenched correlations between human keypoint trajectories and contact patterns present in motion capture datasets.

The policy $$\pi_\theta(\mathbf{a}_t | \mathbf{p}_t, \bar{\mathbf{k}}_t, \bar{\mathbf{c}}_t)$$ receives proprioceptive features, a reference keypoint trajectory, and a contact-conditioning label vector. This label, per time-step and body link, enables test-time actuation or suppression of contacts. The reward blends standard tracking terms with two novel contact-aware signals:

• Contact label matching: Balanced accuracy (or true positive minus false positive, for sparse-contact tasks) between observed and desired contacts.
• Contact distance: Penalizes undesired proximity and rewards intentional contact, computed over all relevant part-pairs.

To counteract strong dataset-level correlations between geometry and contact, the augmentation regime creates “motion pairs” by flipping contact labels, removing the interaction object, and inflating object geometry, generating keypoint/contact decouplings the policy must genuinely learn to follow.

Simulation Results: Contact Controllability

ContactMimic is evaluated on 10 diverse human-object interactions using the Unitree G1 in Isaac Lab physics simulation. For each motion, policies are commanded along identical keypoint trajectories with either a “make contact” or “suppress contact” label. The results robustly show that contact metrics—contact count, impulse, and relevant joint torque—systematically track the supplied contact command, even when the geometric trajectory itself remains unchanged. When tasked with “no contact” under a sitting motion, for example, the same keypoints result in the robot hovering over a chair rather than sitting.

Object manipulation tasks are particularly revealing: toggling the contact label for “pick up box” transitions the policy from grasping and lifting the box to leaving it untouched, without any task-specific reward shaping, thereby directly linking the command interface to physical outcomes.

Real-World Evaluation

ContactMimic's sim2real transfer is validated on the actual G1 robot across five motion types. Qualitative and quantitative results confirm that the trained policy preserves contact controllability: the robot executes contact-affordant trajectories (e.g., applying sufficient force to wipe a whiteboard) only when instructed, while the same pose with the opposite label exhibits contact avoidance.

Necessity of Contact Conditioning

Ablations against state-of-the-art keypoint-only trackers (e.g., BeyondMimic) demonstrate that tracking geometric trajectories alone is insufficient for producing or suppressing meaningful contacts. The baseline can imitate motion shape but fails to effectuate task-relevant contact, especially in contact-critical tasks, despite similar MPJPE scores. Hence, explicit contact conditioning is not just beneficial but necessary to achieve robust task-level control over physical interaction.

Augmentation Ablations and Representation Analysis

The data augmentation pipeline—contact label flipping, object removal, and geometry inflation—proves indispensable. Ablation studies unequivocally indicate that omitting these leads to policies that ignore the contact command and revert to default correlations. With augmentation, strong separation in contact metrics is restored, showing that careful training data construction is critical for policy disentanglement.

Figure 2: Removing contact-aware data augmentation causes marked loss of contact controllability; solid arrows (with augmentation) indicate controllable contact, whereas dotted arrows (no augmentation) show policy insensitivity to contact commands.

Further analysis with linear probing reveals the policy explicitly encodes actual contact state in its intermediate representations, even though no contact sensing is provided at inference time—proprioceptive input suffices for the policy to infer runtime contacts and adhere to commanded states.

Implications and Future Directions

ContactMimic substantiates the feasibility and utility of explicit, test-time-controllable contact conditioning in learned humanoid control. This paradigm enables flexible contact-rich task specification, offering a more general and interpretable interface compared to implicit contact via geometric tracking alone.

The framework, currently limited to per-motion policies (not universal multi-task ones), is a key step toward contact-centric high-level task programming in humanoids. Expanding ContactMimic to train a multi-motion, fully universal contact-conditioned policy, and scaling data augmentation to less curated, in-the-wild modalities, present exciting avenues for further work. Additionally, leveraging more sophisticated forms of contact prediction or controllers capable of reasoning over higher-order contact schedules could extend the approach to complex, multi-contact manipulation and locomotion challenges.

Figure 3: Per-motion ablation: with paired-motion augmentation, the policy modulates contact and impulse according to the contact label; without augmentation, this control collapses.

Conclusion

ContactMimic advances the state of humanoid object interaction from implicit, geometry-derived contact to explicit, controllable contact-level specification and control. Through contact-conditioned policies, contact-aware reward structuring, and data augmentation that breaks conventional geometric-contact correlations, the framework achieves fine-grained, interpretable control of diverse, contact-rich tasks in both simulation and hardware. Explicit contact-conditioning is demonstrated as a critical and generalizable principle for future progress in purposeful, interaction-aware humanoid control.