Kinematic Retargeting

Updated 30 June 2025

Kinematic retargeting is the process of transferring motion between systems with different structures while preserving semantic intent and physical realism.
Modern methods use deep learning, mesh-based embeddings, and optimization to capture spatial relationships and ensure smooth motion.
Adaptive constraint weighting enables robust, real-time retargeting across varied morphologies and terrains, preserving critical contact details.

Kinematic retargeting is the process of adapting movement trajectories, poses, or postures from one kinematic embodiment—such as a human, digital character, or robot—to another system with potentially different bone lengths, joint structures, or morphological features, in a manner that preserves semantic intent, spatial relationships, and physical plausibility. Modern methods in this domain combine deep learning, optimization, and geometric modeling to enable real-time, semantics-preserving motion transfer across a diverse range of target morphologies, while conforming to hardware or mesh constraints.

1. Foundations and Problem Definition

Kinematic retargeting arises in contexts where a demonstration (e.g., human motion capture, animated rig, teleoperator action) must be reproduced by a target that may differ structurally—such as a robot, an avatar with a different skeleton, or a character with non-matching proportions. The challenge is not simply to interpolate or project joint angles, but to yield motions that:

Preserve high-level semantics (e.g., a hand grasping an object, a foot stepping on uneven ground),
Enforce physical plausibility (avoiding self-collisions, maintaining ground contacts),
Remain smooth and natural despite morphological differences.

This typically requires overcoming ambiguities associated with joint mappings, functional and geometric constraints, and stability requirements in physical and animated embodiments.

2. Embedding and Descriptor-Based Approaches

A central methodological advance is the use of low-dimensional mesh-based embeddings and semantic motion descriptors for robust retargeting across morphologically diverse characters (ReConForM : Real-time Contact-aware Motion Retargeting for more Diverse Character Morphologies, 28 Feb 2025). Rather than relying solely on skeletal joint positions, systems like ReConForM define a set of automatically transferred key vertices on each character’s mesh, covering limbs, hands, feet, and other contact-prone regions. These key vertices enable the extraction of motion descriptors—such as pairwise distances, directional vectors, ground heights, and sliding velocities—over time.

These descriptors form trajectory-based embeddings that abstract away the specifics of skeletal joint arrangements, allowing the retargeting process to focus on preserving the semantic structure of motion—such as maintaining hand-to-knee contact—regardless of precise skeleton or mesh topology.

3. Optimization with Adaptive Semantic Constraints

Retargeting is formulated as an optimization problem, where the goal is to minimize a weighted sum of semantic losses that encode contact relationships, floor constraints, physical plausibility, and motion smoothness. The general objective is:

$\min_{q,p} \ w_\text{reg} \mathcal{L}_\text{reg} + w_\text{smooth} \mathcal{L}_\text{smooth} + w_\text{sem} \mathcal{L}_\text{sem}$

Semantic loss ( $\mathcal{L}_\text{sem}$ ): Enforces consistency of pairwise distances, directions, collision avoidance (penetration), ground heights, and sliding measures between source and target.
Regularization ( $\mathcal{L}_\text{reg}$ ): Maintains proximity to an initialized (e.g., naively projected) pose, preventing unrealistic deviations.
Smoothness ( $\mathcal{L}_\text{smooth}$ ): Penalizes high jerk in the joint trajectories to ensure natural motion flow.

A distinguishing feature is the adaptive selection and weighting of constraints: At each frame, the algorithm determines which relationships (e.g., specific contacts or groundings) are most relevant by calculating proximity-based weights, enforcing critical constraints (such as contact or collision avoidance) while relaxing conflicting or less vital ones. For example, the weight for a contact constraint between key-vertex pairs $(i, j)$ is:

$\mathcal{W}_\text{interaction}(i, j) = \text{clamp}\left(1-\frac{\mathcal{M}_\text{dist}(i,j) - d_{\min}}{d_{\max} - d_{\min}}\right)$

where $\mathcal{M}_\text{dist}(i, j)$ is the Euclidean distance, and $d_{\min}, d_{\max}$ are thresholds (relative to character scale).

This adaptive mechanism is key to robust retargeting for characters with substantial morphological differences or atypical contacts.

4. Extensions: Multi-Character, Terrain, and Contact Scenarios

The mesh embedding and adaptive constraint framework naturally generalizes to advanced settings:

Multi-character retargeting: Cross-character constraints, such as inter-character contacts (touching, supporting, etc.), can be incorporated by extending descriptors and weights to key-vertex pairs across meshes. Additional features, such as gaze alignment, can be included to further guide multi-agent interactions.
Motion on non-flat/uneven terrain: Height and sliding descriptors are generalized to arbitrary height fields, enabling accurate retargeting for scenarios involving stairs, ramps, or terrain discontinuities. The floor proximity constraint becomes:

$\mathcal{M}_\text{height}(i) = p_i^y - f(p_i^x, p_i^z)$

where $f(x, z)$ defines the terrain surface at horizontal coordinates.

Conflict handling and interactive authoring: If the simultaneous enforcement of all constraints is physically impossible (e.g., due to mesh penetration or conflicting contact goals), conflict is detected by examining cosine similarity of constraint gradients. This allows interactive adjustment of priorities, such as favoring contact preservation or collision avoidance, according to application or user needs.

5. Evaluation and Performance

ReConForM was tested across a broad array of challenging scenarios including retargeting motions between characters with large disparities in shape, size, and skeleton, as well as on uneven grounds and multi-character set-ups (ReConForM : Real-time Contact-aware Motion Retargeting for more Diverse Character Morphologies, 28 Feb 2025). Notable findings include:

Superior contact preservation: The method achieves higher scores for foot grounding and semantic contact transfer compared to both academic and industry baselines.
Low penetration and high smoothness: Quantitative metrics for self-collision, floor penetration, and jerk are improved over state-of-the-art and commercial systems (including Mixamo, Unreal Engine, Maya MotionBuilder, and SAN/R²ET research methods).
Real-time performance: Through sparse constraint activation and efficient GPU-optimized batch processing, the method retargets sequences at over 60 frames per second on consumer hardware, supporting interactive workflows and online applications.

A large-scale user paper confirmed perceptual superiority, especially for contact-rich or morphologically mismatched retargeting tasks.

6. Broader Implications and Practical Applications

Semantic mesh-embedding approaches to kinematic retargeting support a wide spectrum of applications:

Character animation and game production: High-fidelity, semantics-preserving transfer of motion data between rigs of arbitrary topology, with reduced manual intervention.
Teleoperation and robotics: Mapping human (or simulation-based) demonstration onto physical robots or avatars with diverse limb proportions, including handling of multi-contact and manipulation scenarios.
Multi-agent and crowd simulation: Retargeting motion to heterogeneously rigged agents with consistent contact and interaction semantics.
Virtual/augmented reality embodiment: Preserving physical interactions (such as hand rests, gazes, touches) for user-driven avatars with variable body shapes on complex terrains.
Interactive authoring: Rapid prototyping and design of new characters, including visualization of how changes in configuration or morphology impact feasibility or expressivity of motion sequences.

A plausible implication is that mesh-embedding and semantics-adaptive methods may become a new standard in applications where preserving contact and interaction details across morphologically diverse characters is necessary, moving beyond the constraints of skeleton-joint-only methods.

7. Comparison with Prior Work and Current Frontiers

ReConForM advances prior approaches by:

Enabling morphology-agnostic retargeting with minimal hand-crafted correspondences, avoiding the pitfalls of joint-to-joint projection or strict kinematic chain models.
Supporting contact preservation and complex terrain handling in real time, where most previous methods either ignore contact information or lack robustness when skeletons differ strongly.
Achieving better balance between fidelity and feasibility via selective constraint weighting, as opposed to static full-set enforcement.

Remaining open challenges include handling extreme topology mismatches (e.g., missing or extra limbs), further automating the selection of key vertices or descriptors, and extending current strategies to even more complex interaction scenarios (e.g., object manipulation with deformable bodies, bimanual or collaborative manipulation with tool use).

Aspect	Description	Performance/Impact
Mesh-based embedding	Rigged key vertices transferred via optimal transport	Morphology-agnostic, real-time
Semantic motion descriptors	Pairwise distances, directions, environmental features	Contact/interaction preservation
Adaptive semantic weighting	Dynamic, frame-wise selection of constraints	Feasible, smooth, and compliant transfer
Multi-character/terrain cases	Natively supports coordination, arbitrary ground topology	Accurate retargeting in unstructured environments
Evaluation	State-of-the-art in smoothness, contact recovery, user preference	60+ FPS, outperforming industry + academic rivals

Kinematic retargeting continues to evolve toward greater generality, semantic robustness, and efficiency, with embedding and adaptive constraint methods offering promising solutions for the broadest range of applications.

PDF Markdown Chat (Upgrade)

References (1)

ReConForM : Real-time Contact-aware Motion Retargeting for more Diverse Character Morphologies (2025)