Kinematic Interaction Transfer (KITE)

• KITE is a zero-shot cross‐embodiment manipulation framework that separates task reasoning from embodiment-specific motor control through a learned latent intent based on contact patterns.
• The framework decouples task semantics from morphology by training a shared policy for latent intent prediction and a tailored action decoder for each robot's kinematic model.
• Empirical evaluations in simulation and real-world tests show that KITE achieves high success rates in transferring interaction intent, demonstrating its robustness across diverse robot structures.

Searching arXiv for the KITE paper and closely related cross-embodiment interaction-transfer work. Kinematic Interaction Transfer across Embodiments (KITE) is a zero-shot cross-embodiment manipulation framework that decouples manipulation into embodiment-agnostic task reasoning and embodiment-specific motor control, with the two connected by a learned latent representation of interaction intent based on contact patterns. In its canonical formulation, a shared policy predicts latent intents from source demonstrations, while each embodiment uses its own intent-conditioned action decoder learned from its kinematic model; adaptation to a new embodiment therefore requires only training a new action decoder using its kinematic model, without recollecting demonstration data (Wang et al., 20 Jun 2026). In adjacent work on cross-embodiment retargeting and interaction learning, closely related objectives are realized through interaction meshes, graph-conditioned diffusion, embedded interaction graphs, calibrated force-position interfaces, particle-space world models, shared affordance spaces, and tool-mediated interfaces (Yang et al., 30 Sep 2025, Cao et al., 27 May 2025, Li et al., 28 Jul 2025, Atar et al., 14 Jun 2026, He et al., 3 Nov 2025, Seo et al., 2024). This suggests a broader research program centered on separating task semantics from morphology-specific actuation.

1. Problem formulation and conceptual scope

The core problem addressed by KITE is zero-shot cross-embodiment manipulation: a policy trained on source embodiments must be deployed on a structurally different target embodiment without additional task demonstrations (Wang et al., 20 Jun 2026). The central obstacle is coupling. Standard imitation or reinforcement policies learn directly in an embodiment’s native action space, so the learned semantics of the action space remain tied to a specific morphology. A policy trained on a parallel-jaw gripper therefore internalizes the semantics of two jaws; when transferred to a five-fingered hand, those semantics no longer hold (Wang et al., 20 Jun 2026).

Within the broader literature, this coupling appears in several forms. Retargeting pipelines that map human motion directly to robot kinematics often struggle with embodiment gaps and can produce artifacts such as foot-skating or penetration; they may also neglect human-object and human-environment interactions that are essential for expressive locomotion and loco-manipulation (Yang et al., 30 Sep 2025). Diffusion-based motion retargeting across heterogeneous embodiments faces inconsistencies in topological structure, geometrical parameters, and joint correspondence (Cao et al., 27 May 2025). Dexterous grasp transfer is further limited when force feedback remains tied to each hand’s sensing and actuation, even if motion is normalized across embodiments (Atar et al., 14 Jun 2026).

A useful distinction therefore emerges between transferring motion, transferring interaction, and transferring task intent. KITE, in the strict sense of the 2026 framework, transfers a latent interaction intent derived from contact patterns (Wang et al., 20 Jun 2026). Closely related systems instead preserve contact geometry through Laplacian mesh deformation (Yang et al., 30 Sep 2025), represent interaction as sparse inter-agent graph structure (Li et al., 28 Jul 2025), normalize action in a tool-centric space (Seo et al., 2024), or learn embodiment-invariant environment dynamics in a particle representation (He et al., 3 Nov 2025). The shared premise is that direct joint-space transfer is not, by itself, the right abstraction.

2. Canonical KITE architecture

In the 2026 formulation, KITE decomposes manipulation into two stages connected by a learned latent interaction intent $z_t$ (Wang et al., 20 Jun 2026). The embodiment-agnostic component is a shared policy $\pi$, which consumes proprioceptive-free observations such as object point clouds from RGB-D cameras and the previous latent $z_{t-1}$, and predicts the next latent intent: $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$ Because $\pi$ never sees joint angles or link geometries, it is agnostic to any particular embodiment (Wang et al., 20 Jun 2026).

The embodiment-specific component is an action decoder $g^e$ for each body $e$. Given the current configuration $q_t^e$ and the intent $z_t$, the decoder predicts the next joint configuration: $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$ At deployment on a new target embodiment, KITE swaps in $\pi$0 while reusing the same $\pi$1 (Wang et al., 20 Jun 2026).

The latent intent is grounded in contact. At each frame, the “true” interaction intent is a sparse set of contact points and local normals on the object’s surface,

$\pi$2

which is mapped by a permutation-invariant encoder $\pi$3 into a fixed vector,

$\pi$4

This design encodes where and how the manipulator should engage the object, but not which finger or link should do it (Wang et al., 20 Jun 2026).

A distinctive feature of KITE is that each decoder is trained without task demonstrations for that embodiment. Using only the robot’s URDF $\pi$5, KITE synthesizes millions of $\pi$6 triples. It samples a “contact” joint state $\pi$7 uniformly within joint limits, forward-kinematics transports a fixed template $\pi$8 of surface points to obtain $\pi$9, perturbs $z_{t-1}$0 to get $z_{t-1}$1, masks joints not affecting the chosen contact points, and defines

$z_{t-1}$2

where $z_{t-1}$3 iff joint $z_{t-1}$4 is contact-relevant. The supervised regression loss is

$z_{t-1}$5

Both $z_{t-1}$6 and all decoders $z_{t-1}$7 are trained jointly on these synthetic data; no real demonstrations enter this stage (Wang et al., 20 Jun 2026).

Training of the shared task policy occurs after $z_{t-1}$8 and $z_{t-1}$9 are fixed. Source demonstrations are converted frame by frame into contact sets $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$0, encoded as $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$1, and imitation learning is then performed in $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$2. The policy minimizes a standard imitation loss on predicting $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$3 from $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$4, and there is no additional KL term, reconstruction loss, or embodiment-specific regularizer (Wang et al., 20 Jun 2026).

3. Embodiment-invariant interfaces for interaction transfer

The main axis of variation across KITE-related research is the choice of transfer interface: latent contacts, geometric constraints, graph structure, particle dynamics, affordances, or tools. The following representations recur as candidate embodiment-invariant abstractions.

Method	Shared representation	Primary role
KITE	Latent interaction intent from contact points and normals	Decouple task reasoning from motor control
OmniRetarget	Interaction mesh with Laplacian deformation	Preserve agent-terrain-object spatial relations
G-DReaM	Embodiment graph $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$5 with joint correspondence $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$6	Unified motion retargeting across heterogeneous embodiments
BuddyImitation	Embedded Interaction Graph	Encode transferable inter-agent spatiotemporal relations
Contact-aware grasp transfer	Shared hand-pose latent $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$7 plus per-finger load descriptors	Transfer compliant interaction, not motion alone
Cross-embodiment world models	3D particles and particle displacements	Learn embodiment-invariant environment dynamics
LEGATO	Handheld gripper with tool-frame action and observation space	Unify visuomotor demonstrations across robots
Affordance Equivalence	Shared affordance representation spanning objects, actions, and effects	Translate demonstrations through object-action-effect equivalence
MOTIF	Quantized action motifs in canonical end-effector space	Few-shot transfer via spatiotemporal primitives

OmniRetarget represents source and target as an interaction mesh whose vertices include anatomical keypoints, points sampled on manipulated objects, and points sampled on the scene; edges are constructed via Delaunay tetrahedralization over the combined vertex set. The per-frame loss matches source and target Laplacians,

$$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$8

with hard enforcement of joint limits, velocity limits, non-penetration constraints $$z_t=\pi(o_t,z_{t-1}), \qquad z_t\in\mathcal Z\subset\mathbb R^m.$$9, and foot-sticking constraints. This representation is designed to preserve local shape and contact relations rather than only joint trajectories (Yang et al., 30 Sep 2025).

G-DReaM encodes each embodiment as a graph $\pi$0 with node attributes for joint rotation axes, edge attributes for parent-child link vectors, and a joint correspondence mask $\pi$1. Retargeting is cast as conditional diffusion with energy-based guidance, so that the model samples motions conditioned on the reference motion, source and target graphs, and correspondence structure (Cao et al., 27 May 2025).

BuddyImitation uses an Embedded Interaction Graph (EIG), a fully connected spatial graph whose nodes are the joints of both interacting characters and whose directed edges encode relative displacement and midpoint: $\pi$2 A time-varying sparse subgraph then becomes the imitation target, rather than raw motion matching (Li et al., 28 Jul 2025).

For dexterous grasp transfer, the contact-aware line of work separates a shared motion representation from a calibrated interaction channel. Motion is expressed in a $\pi$3-dimensional MANO latent $\pi$4, while each hand’s effort signal is calibrated into physical joint torque in N.m, mapped to fingertip forces, and compacted into per-finger load descriptors $\pi$5 (Atar et al., 14 Jun 2026). For world-model approaches, the representation is neither joints nor contacts but particles: end-effectors and objects are represented as 3D point sets, and actions are per-particle displacements, $\pi$6, so that a single transition operator can be learned across embodiments (He et al., 3 Nov 2025).

Tool and affordance interfaces occupy another branch of the design space. LEGATO uses a handheld gripper with a unified action space of 6D differential tool motion and a binary grasp bit, both expressed in the tool frame (Seo et al., 2024). Affordance Equivalence instead learns a shared affordance representation spanning multiple agents and objects, so that object, action trajectory, and effect trajectory inhabit a single latent vector in a common latent space (Aktas et al., 2024). MOTIF canonicalizes raw trajectories into end-effector pose sequences in a canonical frame and quantizes short-horizon subsequences into discrete action motifs (Zhi et al., 14 Feb 2026).

4. Learning and optimization mechanisms

KITE-related systems differ not only in representation but also in how the transfer constraint is imposed. The 2026 KITE framework learns its shared interface by synthetic kinematics supervision for the decoders and imitation learning in latent space for the task policy (Wang et al., 20 Jun 2026). The transfer bottleneck is therefore architectural: all embodiment-specific realization must pass through $\pi$7, while all task reasoning must pass through $\pi$8.

OmniRetarget imposes transfer through constrained per-frame optimization. At each time $\pi$9, it solves for robot configuration $g^e$0 by minimizing the Laplacian deformation loss plus a smoothness cost $g^e$1, subject to joint-limit and velocity limits, collision avoidance, and foot-sticking. These are hard constraints, enforced by a customized Sequential QP/SOCP solver; no soft penalties or projections are used (Yang et al., 30 Sep 2025). This is a geometric and feasibility-first formulation.

G-DReaM replaces explicit per-frame optimization with a learned conditional generative model. The forward process adds Gaussian noise, while the reverse process is parameterized by a score network. Its denoiser is a Transformer decoder over $g^e$2 tokens with spatial-temporal self-attention and multi-conditional cross-attention. Sampling is guided by explicit kinematic energy terms: joint-position similarity, forward-kinematic consistency, velocity tracking, and angle/motion norm regularization. Larger $g^e$3 in the diffusion loss forces samples into regions of low total kinematic guidance (Cao et al., 27 May 2025).

BuddyImitation uses RL with graph-based imitation objectives. The per-step imitation loss is a Frobenius-norm distance between edge-feature matrices of the demonstration and agent sparse graphs,

$g^e$4

combined with physics regularization terms such as joint torque magnitudes and root height deviation. The control stack is hierarchical: a centralized high-level network outputs a latent $g^e$5, and each low-level policy is an LSTM-augmented MLP trained with PPO or SAC (Li et al., 28 Jul 2025).

Contact-aware dexterous transfer uses a flow-matching visuomotor policy conditioned on vision, proprioception, calibrated contact, and the previous latent $g^e$6. Training applies structured visual masking, and the same calibrated signal also drives a hybrid force-position controller for demonstration collection and execution, keeping force targets consistent across training and deployment (Atar et al., 14 Jun 2026). MOTIF also adopts flow matching, but its conditioning variable is a predicted sequence of quantized motifs obtained from a frozen visual-language stack and a lightweight predictor trained with an MSE objective against the frozen motif encoder (Zhi et al., 14 Feb 2026).

Particle world models adopt supervised next-state prediction with GNN message passing, followed by model-predictive control. Given forward kinematics $g^e$7, a new embodiment maps joint sequences into particle actions, rolls out the learned world model, and optimizes joint control sequences with the Cross-Entropy Method (He et al., 3 Nov 2025). LEGATO, by contrast, places the learning burden on a recurrent visuomotor policy and uses a prioritized quadratic program over joint velocities for deployment-time whole-body motion retargeting (Seo et al., 2024).

5. Empirical validation

The canonical KITE framework was evaluated in simulation on three manipulation tasks spanning a parallel-jaw gripper, three dexterous hands, and a composite embodiment: Cube Picking, Keyboard Pressing, and Bottle Pumping. Across the 9 source-target cells, KITE was compared to OHRA, D(R,O), and SPIDER. Zero-shot success was averaged over 100 held-out episodes, and KITE (full) achieved 90–100 % success across all 9 cells. An “Oracle Intent” ablation confirmed that the action decoder alone matches or exceeds SPIDER wherever SPIDER applies, and adding small amounts of target-task data, up to 200 demos, brought no significant improvement (Wang et al., 20 Jun 2026). Real-world transfer on a physical Wuji hand succeeded in 7–8 trials out of 10 per task without real-world fine-tuning (Wang et al., 20 Jun 2026).

OmniRetarget evaluates the retargeting layer and the downstream policy layer jointly. It reports kinematic constraint metrics such as penetration duration and max depth, foot-skating duration and max skating velocity, and contact-preservation duration. On robot-object motion from OMOMO, PHC yields penetration $g^e$8 s, depth $g^e$9 cm, skating $e$0 s, contact $e$1, and RL success $e$2; GMR reports RL $e$3; VideoMimic reports RL $e$4; OmniRetarget reports penetration $e$5, skating $e$6, contact $e$7, and RL $e$8 (Yang et al., 30 Sep 2025). The resulting proprioceptive RL policies execute long-horizon parkour and loco-manipulation skills on a Unitree G1 humanoid, trained with only 5 reward terms and simple domain randomization shared by all tasks, without any learning curriculum (Yang et al., 30 Sep 2025).

G-DReaM reports full-body positional MSE at centimeter-level across eight robots, including CL: $e$9 cm$q_t^e$0, G1: $q_t^e$1, Tron1: $q_t^e$2, Atlas: $q_t^e$3, Talos: $q_t^e$4, H1: $q_t^e$5, Cassie: $q_t^e$6, and Valkyrie: $q_t^e$7. Ablations show that graph encoding plus discrete relational embeddings improves kinematic fidelity by $q_t^e$8 versus vanilla attention, and that energy guidance with $q_t^e$9 is critical (Cao et al., 27 May 2025).

BuddyImitation reports pose reconstruction error over 2 s rollouts of $z_t$0 m$z_t$1 for the full graph, $z_t$2 for a random 4-edge graph, and $z_t$3 for a learned 4-edge graph; handshake success in simulation rises from $z_t$4 for baseline IK-retargeting to $z_t$5; user-study recognition accuracy rises from $z_t$6 to $z_t$7; and semantic alignment rises from $z_t$8 to $z_t$9 (Li et al., 28 Jul 2025).

Contact-aware dexterous transfer reports average success $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$0 for a pose-only baseline and $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$1 for MARC with calibrated contact and masking, with embodiment-specific improvements of $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$2 on Inspire, $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$3 on Wuji, and $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$4 on Wuji$a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$5. Zero-shot on an unseen Wuji$a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$6 reaches mean $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$7, within $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$8 of in-distribution performance (Atar et al., 14 Jun 2026). MOTIF reports $a_t^e=g^e(z_t,q_t^e)=q_{t+1}^e.$9 percentage points in simulation few-shot transfer and a $\pi$00 percentage point gain in real-world transfer against strong baselines (Zhi et al., 14 Feb 2026). LEGATO reports $\pi$01 average success across embodiments in simulation and $\pi$02, $\pi$03, and $\pi$04 success on three real Panda tasks (Seo et al., 2024). The particle-world-model line reports that MSE decreases monotonically as the number of training embodiments increases and that mixed sim:real training at $\pi$05 yields the lowest error on held-out human data (He et al., 3 Nov 2025).

6. Limits, misconceptions, and open directions

A common misconception is that cross-embodiment transfer is solved once motion is normalized. Several of the cited systems reject that premise directly. KITE is organized around contact-pattern intent rather than joint trajectories (Wang et al., 20 Jun 2026). OmniRetarget argues that common retargeting methods neglect rich human-object and human-environment interactions (Yang et al., 30 Sep 2025). The contact-aware dexterous literature explicitly frames the issue as “transferring contact, not just motion,” emphasizing that stable manipulation requires fingers to maintain appropriate object loading as contacts slip, deform, or become visually occluded (Atar et al., 14 Jun 2026). A plausible implication is that purely kinematic alignment is often an insufficient transfer target for contact-rich manipulation.

A second misconception is that “zero-shot” eliminates embodiment-specific training entirely. In KITE, adaptation to a new embodiment requires no new task demonstrations, but it does require training a new action decoder using the target embodiment’s kinematic model (Wang et al., 20 Jun 2026). In world-model and retargeting settings, zero-shot deployment likewise presupposes either a forward-kinematic map, a collision model, or other embodiment-specific geometric information (He et al., 3 Nov 2025, Yang et al., 30 Sep 2025).

The main limitations stated in the literature are also revealing. KITE assumes that tasks can be fully described by contact patterns; sim-to-real gaps in soft compliance or tendon slack can degrade transfer because $\pi$06 sees only rigid kinematics; the one-time cost of training decoders for every new embodiment remains; and the “closest feasible contact” heuristic may fail if the robot’s pose is far from any configuration that can realize a demanded contact (Wang et al., 20 Jun 2026). OmniRetarget’s guarantees rely on driving mesh deformation loss to zero under a hard-constraint solver, which is a strong geometric premise (Yang et al., 30 Sep 2025). G-DReaM reports that complex unseen actions such as get-up remain challenging under zero-shot generalization (Cao et al., 27 May 2025). Particle-world-model approaches note sensitivity to graph density, the absence of systematic ablations on GNN depth or radius, and the present exclusion of force or tactile signals (He et al., 3 Nov 2025). MOTIF identifies fixed codebook size, reliance on progress-aware alignment, and the need to incorporate contact dynamics and rich manipulation as open problems (Zhi et al., 14 Feb 2026).

Future directions are relatively consistent across papers. KITE proposes richer interaction interfaces beyond point-normal sets, including force profiles or contact patches, and extension to tasks where non-contact intermediates such as cloth manipulation or tool usage are central (Wang et al., 20 Jun 2026). Contact-aware transfer suggests incorporating compliance models into decoder training and reusing calibrated contact primitives inside long-horizon manipulation pipelines (Atar et al., 14 Jun 2026). World-model work proposes adding force/torque or tactile feedback and extending beyond current task classes (He et al., 3 Nov 2025). MOTIF points toward multi-modal motifs, hierarchical motif chaining, adaptive codebooks, and explicit dynamic invariants (Zhi et al., 14 Feb 2026). Across these proposals, the recurring objective is to find the narrowest interface that is still sufficient for control while remaining stable across large embodiment changes.