CONTACTMIMIC: Contact-Aware Imitation & Tactile Data

• CONTACTMimic is a contact-aware humanoid control framework where explicit binary contact commands guide manipulation tasks by bridging the gap between kinematic imitation and functional contact control.
• It introduces novel reward terms and trajectory augmentation strategies that ensure true contact controllability, enabling robust performance in tasks such as wiping, sitting, and object manipulation.
• The CONTACTMIMIC dataset provides high-resolution visual-tactile-action data for soft-object manipulation, combining dense tactile observations with teleoperated protocols to capture dynamic contact interactions.

CONTACTMIMIC denotes a contact-centered line of humanoid and manipulation research in which task success is defined not only by reproducing kinematics, but by reproducing or controlling physically meaningful contact. In the literature, the name appears in two principal forms: as CONTACTMIMIC, a humanoid object-interaction framework that conditions policies on explicit part-level binary contact commands during motion imitation, and as CONTACTMIMIC, a humanoid visual-tactile-action dataset for soft-object manipulation under varying pressure conditions (Li et al., 9 Jul 2026, Kwon et al., 28 Oct 2025). Taken together, these works formalize a common thesis: keypoint trajectories or vision-only observations are often insufficient for contact-rich behavior, because physically relevant interaction depends on which body parts or end-effectors touch which objects, when they touch, and under what pressure regime.

1. Scope and conceptual basis

A concise way to situate CONTACTMIMIC is to distinguish between motion similarity and contact semantics. The humanoid CONTACTMimic paper argues that tasks such as wiping a whiteboard, sitting on a chair, leaning on a backrest, lifting a box, or pushing furniture cannot be specified by keypoint tracking alone, because a robot may match the reference pose while failing to make the contact that gives the motion its functional meaning (Li et al., 9 Jul 2026). The dataset paper makes an analogous point from the sensing side: prior robot-learning datasets have emphasized rigid objects, simple grippers, and sparse or binary touch, whereas deformable-object manipulation requires dense tactile sensing and explicit pressure variation (Kwon et al., 28 Oct 2025).

Usage	Primary focus	Paper
CONTACTMimic framework	Humanoid object interaction via explicit contact control	(Li et al., 9 Jul 2026)
CONTACTMIMIC dataset	Humanoid visual-tactile-action data for soft-object manipulation	(Kwon et al., 28 Oct 2025)

This suggests a broader usage in which “CONTACTMIMIC-style” problems are those where success depends on contact-aware imitation rather than gross trajectory replay. That broader usage is explicit in later manipulation papers that describe contact-rich imitation settings in terms of forceful interaction, slip, sliding, or contact-state inference rather than geometry alone (Ablett et al., 2023, Oh et al., 23 Nov 2025).

2. CONTACTMimic as a contact-conditioned humanoid control framework

The control framework defines a policy

$$\pi_\theta(\mathbf{a}_t \mid \mathbf{p}_t, \bar{\mathbf{k}}_t, \bar{\mathbf{c}}_t),$$

where $\mathbf{p}_t$ is proprioception, $\bar{\mathbf{k}}_t$ is the reference keypoint target, and $\bar{\mathbf{c}}_t \in \{0,1\}^{|\mathcal{B}|}$ is a binary reference contact label map over contact-capable body parts (Li et al., 9 Jul 2026). The contact-capable bodies are pelvis, torso, hips, knees, ankles, shoulders, and wrists. The policy outputs target joint angles, which are converted into torques by a PD controller.

The reward augments standard tracking and regularization with two contact-aware terms: $$r_t = r_t^\text{track} + w_\text{lm} r_t^\text{lm} + w_\text{cd} r_t^\text{cd} + r_t^\text{reg}.$$ The first term, $r_t^\text{lm}$, is a contact label matching reward. With $\mathcal{S}_+ = \{(b,p): \bar{c}_{t,b,p}=1\}$ and $\mathcal{S}_- = \{(b,p): \bar{c}_{t,b,p}=0\}$, the framework defines

$$\mathrm{TPR} = \frac{1}{|\mathcal{S}_+|}\sum_{(b,p)\in\mathcal{S}_+}c_{t,b,p}, \quad \mathrm{TNR} = \frac{1}{|\mathcal{S}_-|}\sum_{(b,p)\in\mathcal{S}_-}(1-c_{t,b,p}), \quad \mathrm{FPR} = 1-\mathrm{TNR}.$$

Its default form is balanced accuracy,

$$r_t^\text{lm} = \tfrac{1}{2}(\mathrm{TPR}+\mathrm{TNR}),$$

with a sparse-contact alternative

$\mathbf{p}_t$0

The second term, $\mathbf{p}_t$1, is a contact distance reward: $\mathbf{p}_t$2 with

$\mathbf{p}_t$3

and

$\mathbf{p}_t$4

If contact is desired, the reward pulls the body part toward the object part; if contact is undesired, it penalizes being too close.

A central claim of the framework is that simply adding contact labels is not enough. In standard human-object interaction datasets, keypoint trajectories are highly correlated with contact states, so a policy can often infer contact from geometry and ignore the explicit command. CONTACTMimic addresses this with a trajectory augmentation scheme intended to break that shortcut. It uses three augmentation types: contact-label flipping, object removal, and inflated geometry during retargeting. The purpose is to create motion pairs that retain similar keypoints while differing in their contact labels, so the policy must obey the commanded contact rather than treating it as redundant (Li et al., 9 Jul 2026).

Training uses PPO in Isaac Lab with 4096 parallel environments, a 50 Hz policy rate, and 30,000 iterations. The actor and critic are MLPs with hidden sizes $\mathbf{p}_t$5 and ELU activations; the critic receives privileged base linear velocity. Domain randomization covers robot link masses, joint friction, object friction, and object mass (Li et al., 9 Jul 2026).

3. Simulation, ablations, and sim-to-real contact controllability

The framework is evaluated on the Unitree G1 humanoid with 29 DoF across 10 HUMOTO motions: Wipe whiteboard, Sit in front of table, Lean on backrest I, Lean on backrest II, Step foot on chair, Sit on table, Lean against table, Sit and squat, Kick chair, and Pick up box (Li et al., 9 Jul 2026). Reported metrics include Contact bodies, Contact impulse, Mean key-joint torque, MPJPE, and Object displacement for free-object motions.

The main empirical result is contact controllability: changing only the contact command changes behavior even when keypoints are held fixed. With contact enabled, the robot physically commits to the interaction; with contact disabled, it follows a similar pose while suppressing the relevant contact. The paper emphasizes that this allows completion of manipulation tasks without task-specific rewards, while maintaining keypoint-tracking quality comparable to a keypoint-only tracker (Li et al., 9 Jul 2026).

Comparison against BeyondMimic is used to isolate the effect of explicit contact conditioning. CONTACTMimic substantially improves contact metrics while achieving similar MPJPE. For object-manipulation motions, the keypoint-only baseline can match motion shape yet fail to manipulate the object, whereas CONTACTMimic produces the required contact and object motion (Li et al., 9 Jul 2026).

The augmentation ablation is decisive. Without trajectory augmentation, contact metrics become largely insensitive to the contact label, and ON/OFF control weakens. The Sit and squat motion shows the clearest failure: the no-augmentation model exhibits similar contact bodies and impulse whether contact is on or off. Gains from augmentation are also clear for Sit in front of table, Sit on table, Lean on backrest II, and Pick up box; Step foot on chair and Kick chair are less sensitive because geometry or brief swing dynamics already constrain contact strongly (Li et al., 9 Jul 2026).

Sim-to-real deployment is demonstrated on the real Unitree G1. The policy runs onboard at 50 Hz, a joint-level PD controller runs at 1000 Hz, and no external motion capture or vision is used at deployment. Objects are manually placed and aligned to the reference scene. The paper reports real-world results for Wipe whiteboard, Sit in front of table, Lean on backrest I, Lean on backrest II, and Sit and squat. Success rates are 5/5 and 5/5 for Wipe whiteboard, 4/5 and 5/5 for Sit in front of table, 9/10 and 10/10 for Lean on backrest I, 10/10 and 9/10 for Lean on backrest II, and 5/5 and 5/5 for Sit and squat under the two contact-command settings reported by the paper (Li et al., 9 Jul 2026). The study also shows that linear probes can predict runtime contact from proprioception and from intermediate policy features with high F1, suggesting that the policy encodes contact state internally even without explicit contact sensing.

4. CONTACTMIMIC as a humanoid visual-tactile-action dataset

The dataset version of CONTACTMIMIC is a humanoid visual-tactile-action dataset for contact-rich manipulation, specifically focused on soft-object manipulation under varying pressure conditions (Kwon et al., 28 Oct 2025). Its motivation is that previous tactile and robot-learning datasets have often centered on rigid objects, low-resolution touch, or simplified grippers, whereas soft-object manipulation produces continuously varying contact patterns in both space and time.

The collection platform is a humanoid teleoperation system described as extending the Unitree workflow. Hardware includes a humanoid robot, Inspire Hand (RH56-DFX, Inspire Robotics) dexterous hands, 1,062 high-resolution tactile sensors on each hand distributed over the fingers and palm, a head-mounted egocentric camera with resolution 848 × 480, an Intel RealSense D435 placed about 1 m to the robot’s left for a third-person view, and a piezo-resistive tactile carpet used to measure external contact pressure and generate real-time pressure heatmaps (Kwon et al., 28 Oct 2025). The robot also records arm and finger joint positions.

Data are collected by three experimenters through humanoid teleoperation. The protocol uses two soft objects, Towel and Sponge, under Strong and Weak pressure conditions, yielding four manipulation tasks: Towel Strong, Towel Weak, Sponge Strong, and Sponge Weak. The dataset reports 101.9K frames / samples of motion data, about 77–80 episodes per task, 20–30 seconds per episode, and four manipulation tasks (Kwon et al., 28 Oct 2025).

Recorded modalities include egocentric and third-person vision, dense tactile readings from 1,062 sensors per hand, tactile-carpet pressure heatmaps, arm and finger proprioception, teleoperated robot actions, and pressure/contact condition labels encoded through the task design. Tactile values are normalized from raw 0–4095 to [0,1]: $\mathbf{p}_t$6 For sparse-sensing comparison, the dense representation is reduced from 2,124 sensors to 42 sensors, approximating a 98% reduction (Kwon et al., 28 Oct 2025).

5. Benchmarks, empirical findings, and dataset limitations

The dataset benchmark uses Action Chunking Transformer (ACT) as the imitation-learning baseline, with tactile information fused into the ACT image encoder. Two variants are compared: ACT-Dense, trained with high-resolution dense tactile signals, and ACT-Sparse, trained with low-resolution sparse tactile signals (Kwon et al., 28 Oct 2025). The protocol uses approximately 80 episodes per task, an 80% training / 20% evaluation split, 100K steps of training, evaluation every 20K steps, and results over three random seeds. The metric is Mean Absolute Error (MAE) between predicted and ground-truth actions,

$\mathbf{p}_t$7

The principal findings are comparative and qualitative rather than large performance gaps. Dense tactile sensing better captures pressure-dependent structure: t-SNE visualizations show clearer clustering by task and stronger separation between strong and weak conditions than sparse tactile inputs. Soft-object manipulation induces dynamic tactile patterns that differ from rigid-object interaction, supporting the use of high-resolution tactile sensing. At the same time, optimization remains difficult: both ACT-Dense and ACT-Sparse show decreasing training loss, but test loss improvement is less pronounced, and the performance gap between dense and sparse under the current baseline is relatively small (Kwon et al., 28 Oct 2025).

Task-dependent behavior is also reported. Towel Weak shows unstable test performance, while Sponge tasks show a clearer gap between strong and weak conditions. The authors interpret this as reflecting greater uncertainty or variation in tactile signals under weak pressure, and more pronounced tactile differences for sponge manipulation. Real-world deployment from a different hand pose is feasible but challenging; tasks requiring precise contact control, such as sponge grasping, are more difficult (Kwon et al., 28 Oct 2025).

The limitations are explicit. CONTACTMIMIC presently includes only two soft objects, only strong and weak pressure regimes, and under the ACT baseline dense sensing does not yet produce dramatic gains. The paper therefore frames the dataset not as a solved benchmark, but as evidence that current optimization methods are not yet sufficient to fully exploit dense tactile data (Kwon et al., 28 Oct 2025).

6. Position within contact-rich imitation learning research

CONTACTMIMIC sits within a broader shift from kinematic imitation toward contact-aware sensing, control, and reward design. One branch emphasizes multimodal sensing at the contact interface. A see-through visuotactile sensor (STS) has been used for door-opening imitation learning with tactile force matching and learned mode switching, increasing average success by 62.5%, 30.4%, and 42.5% for force matching, mode switching, and STS policy inputs respectively, and 64.2% when all three are combined (Ablett et al., 2023). MimicTouch instead learns from human tactile-guided demonstrations with GelSight and audio, reducing imitation MSE from 0.62 without tactile or audio to 0.26 with both modalities (Yu et al., 2023). ViTaMIn pushes this direction toward robot-free visuo-tactile data collection, reporting gains on seven contact-rich tasks and improving dynamic peg insertion from 9/20 to 16/20 with tactile sensing and multimodal pretraining (Liu et al., 8 Apr 2025). MicCheck shows that even a Bluetooth pin microphone can serve as a low-cost contact sensor, achieving 92.9% accuracy on a 10-class benchmark and improving pick-and-pour success from 0.40 to 0.80 (Oh et al., 23 Nov 2025).

A second branch emphasizes contact-centric control structure. ConCent uses a single real demonstration to extract a contact event sequence and optimize contact geometry for sim-to-real RL, reaching 80.0% insertion success in a shape-sorter task, versus 20.0% without contact geometry optimization and 30.0% for unconstrained RL (Kim et al., 29 Jun 2026). WristMimic separates contact-free wrist/body tracking from contact-rich finger learning, achieving 91.1% average success over 40 test sequences, with 98.9% success on OMOMO and 83.3% on ParaHome (Yu et al., 7 Jul 2026).

These neighboring results clarify what CONTACTMIMIC is and is not. It is not merely a better keypoint tracker; the framework paper shows that explicit contact commands and contact-following rewards are necessary for contact controllability, and that contact labels alone are insufficient without augmentation to break keypoint–contact correlations (Li et al., 9 Jul 2026). It is also not equivalent to tactile sensing: the humanoid framework infers runtime contact from proprioception, whereas the dataset work exposes dense tactile structure directly (Li et al., 9 Jul 2026, Kwon et al., 28 Oct 2025). A plausible implication is that CONTACTMIMIC names a family of methods and resources centered on the same principle: contact should be represented as a first-class variable in imitation learning, whether through contact commands, dense tactile observations, force-matched replay, acoustic contact cues, or contact-event rewards.