General Motion Retargeting Methods

• General Motion Retargeting (GMR) is a framework that transfers motion across different skeletal structures while retaining task intent, style, and physical plausibility.
• It employs a range of techniques—including skeleton-aware neural networks, latent-space methods, and physics-guided optimizations—to map motion between heterogeneous embodiments.
• GMR underpins applications in animation, teleoperation, and VR, enabling robust motion synthesis and effective cross-domain retargeting in diverse characters and robots.

General Motion Retargeting (GMR) addresses the problem of transferring motion from a source character or morphology (human, animal, robot, or arbitrary articulated figure) to a target with different skeletal topology, bone lengths, actuation, and physical constraints, while preserving task intent, style, and spatiotemporal plausibility. GMR underlies a range of applications including animation, teleoperation, imitation learning in robotics, and virtual character control.

1. Problem Definition and Theoretical Scope

General Motion Retargeting seeks a mapping $\mathcal{R}$ that, given a motion sequence $x^{\text{src}}$ on a source embodiment $C_{\text{src}}$, synthesizes a semantically equivalent sequence $\hat x^{\text{tgt}}$ for a target embodiment $C_{\text{tgt}}$. The challenge is to bridge differences in kinematic tree topology, degrees of freedom, joint limits, morphology, mass distribution, and possibly actuation type (e.g., human limbs, quadruped robots, crab-like bodies) (Cao et al., 27 May 2025). The field distinguishes between homeomorphic retargeting (preserving joint correspondences and kinematic chains, addressed by e.g. skeleton-aware networks (Aberman et al., 2020)) and fully heterogeneous retargeting across arbitrary topologies (e.g. human-to-quadruped, handled by graph- or template-conditioned approaches (Mourot et al., 2023, Cao et al., 27 May 2025)).

Mathematically, GMR may be formulated as trajectory optimization in the target configuration space subject to physical, geometric, and semantic constraints: $$\min_{q_{1:T}} \sum_t \mathcal{L}_{\text{pose}}(q_t) + \mathcal{L}_{\text{task}}(q_{1:T}) + \mathcal{L}_{\text{phys}}(q_{1:T})$$ where $q_t$ are target configurations, $\mathcal{L}_{\text{pose}}$ enforces body-part or end-effector alignment, $\mathcal{L}_{\text{task}}$ encodes semantic or style correspondence, and $\mathcal{L}_{\text{phys}}$ encodes feasibility (collision, balance, joint limits).

2. Algorithmic Frameworks and Methodologies

Several classes of GMR methodologies have emerged, categorized by their handling of skeleton representation, retargeting mechanism, and learning paradigm.

2.1 Skeleton- and Graph-Aware Neural Networks

Skeleton-aware networks operate by mapping both the source and target skeletons into a shared "primal" latent space defined via homeomorphic pooling of the kinematic tree, using custom temporal and graph convolutions (Aberman et al., 2020). This enables unpaired, domain-agnostic motion transfer for skeletons related by tree-chain subdivision. For skeletons with arbitrary topology, transformer-based autoencoders conditioned on explicit per-skeleton templates (canonical neutral poses) achieve topology-agnostic encoding and decoding, thus supporting retargeting across never-seen structures (Mourot et al., 2023). Graph-conditioned diffusion models, as in G-DReaM, integrate the full spatial connectivity and joint metadata of heterogeneous robots, driving the denoising process with custom energy-based retargeting losses that respect geometric and semantic mapping between incomplete and non-homeomorphic morphologies (Cao et al., 27 May 2025).

2.2 Implicit, Volumetric, and View-Canonical Representations

Implicit volumetric bottlenecks with flow-based warping allow direct manipulation in 3D feature space, enabling few-shot, subject-agnostic image- or video-based retargeting in human synthesis (Ren et al., 2021). MoCaNet demonstrates unsupervised disentanglement of structure, motion, and camera view from monocular videos, supporting 2D-to-3D retargeting even in-the-wild, without paired supervision (Zhu et al., 2021).

2.3 Latent-Space and Flow-Based Methods

Latent-space methods encode motions using vector-quantized autoencoders (VQ-VAE), then learn invertible mappings (flows) in token space for flexible, reversible, unsupervised retargeting across arbitrary pairs of characters or robots (Kim et al., 29 Sep 2025). Feature-conditioned flow-matching enables explicit trade-offs between joint-space “style” and task-space “alignment.”

2.4 Optimization- and Physics-Guided Techniques

Trajectory-level optimization is exemplified by two-stage inverse kinematics (IK) methods such as the GMR tracker for humanoids (Araujo et al., 2 Oct 2025): initial key body alignment is followed by local scaling and constrained trajectory matching, with full enforcement of joint limits and physical feasibility. Riemannian geometry-based frameworks explicitly segment motion into geodesic synergies in the space of joint angles under the inertia metric; retargeting is performed by optimizing geodesic paths in the target’s configuration manifold to reproduce task-space endpoints (Klein et al., 2022).

2.5 Contact and Geometric Constraints

Contact-aware methods employ geometry-conditioned recurrent networks and post-hoc optimization to preserve self- and ground-contacts while avoiding interpenetration, with explicit cone-field and vertex-pair penalties (Villegas et al., 2021). STaR (Seamless Spatial-Temporal aware Retargeting) frames motion as a dense point-cloud driven sequence-to-sequence learning problem, directly penalizing limb–body and limb–limb interpenetrations via local signed-distance fields and injecting multi-level trajectory consistency through pairwise motion-difference tensors (Yang et al., 9 Apr 2025).

3. Training Protocols and Objective Functions

GMR pipelines employ a spectrum of losses reflecting competing goals:

• Reconstruction losses on joint positions, quaternions, or synthesized pixels
• Adversarial losses (GAN-based) with learned discriminators in pose space (Li et al., 2023), joint-space (Zhu et al., 2021), or image space (Ren et al., 2021)
• Perceptual and semantic losses on VGG features, root/limb velocities, and action-level descriptors (Ren et al., 2021, Li et al., 2023, Kim et al., 29 Sep 2025)
• Energy-based retargeting guidance during diffusion denoising steps (e.g., spatial matching, redundancy consistency, velocity tracking) (Cao et al., 27 May 2025)
• Canonicalization losses to enforce invariances to body structure and camera view, obtaining genuine disentanglement (Zhu et al., 2021)
• Physical plausibility constraints: collision/interpenetration, bone-length consistency, joint/velocity regularization, and stochastic joint subsampling (Villegas et al., 2021, Mourot et al., 2023, Yang et al., 9 Apr 2025)

Self-supervised pipelines may bootstrap paired data via IK projection (with collision and limit checks) (Choi et al., 2021), robot-configuration-to-human pose conversion through a learned body prior (VPoser) (Figuera et al., 20 Sep 2024), or latent-embedding neighbor search in shared spaces.

4. Evaluation Protocols and Quantitative Metrics

Comparative evaluation in GMR employs a range of task- and domain-dependent metrics:

• Fidelity metrics: Mean Per-Joint Position Error (MPJPE) normalized by character height (Mourot et al., 2023), root/limb-specific errors, or framewise SSIM/LPIPS (for image-based retargeting) (Ren et al., 2021)
• Geometric plausibility: penetration and contact maintenance rates (fraction of frames or vertices in unwanted contact/interpenetration) (Villegas et al., 2021, Yang et al., 9 Apr 2025)
• Temporal consistency and smoothness: curvature of joint trajectories, multi-level motion-difference distances (Yang et al., 9 Apr 2025)
• Semantic consistency: action retrieval and clustering (Adjusted Rand Index for motion clustering) (Zhu et al., 2021), feature alignment (Kim et al., 29 Sep 2025), Fréchet Inception Distance on extracted motion features (Kim et al., 29 Sep 2025)
• Physical feasibility: collision and joint-limit violation rates (Choi et al., 2021), “naturalness” fraction (no gross artifacts), and success rates in downstream policy tracking tasks (Araujo et al., 2 Oct 2025)
• User studies and preference: subjective judgment of faithfulness and realism, often via blind pairwise comparison (Villegas et al., 2021, Araujo et al., 2 Oct 2025, Li et al., 2023)

GMR methods are further evaluated on diverse, multi-character datasets: Mixamo, AMASS, LAFAN1, iPER, Solo-Dancer, and various robot motion suites. Robustness to novel topologies and data scarcity is assessed via cross-structural transfer and zero-shot testing.

5. Advances, Limitations, and Comparative Analysis

Recent advances include:

• Topology-agnostic encoding: transformer autoencoders with explicit template conditioning enable a unified motion representation across arbitrarily structured skeletons (Mourot et al., 2023), outperforming prior per-topology models (e.g., SAN) especially for cross-structural retargeting.
• Energy-based diffusion models: G-DReaM introduces unified, graph-conditioned denoising diffusion models enabling joint-level reasoning and seamless cross-embodiment transfer (Cao et al., 27 May 2025).
• Self-supervised data curation with priors: VAE-based body priors (VPoser) and automatic filtering of physically infeasible samples provide high-quality pairing for supervised pipelines without manual intervention (Figuera et al., 20 Sep 2024).
• Robustness via contact modeling and geometric losses: Contact-aware recurrent nets and explicit limb-penetration losses (STaR) guarantee plausible, collision-free retargeting for articulated and skinned bodies (Villegas et al., 2021, Yang et al., 9 Apr 2025).
• Latent-flow modules for unsupervised, invertible mapping: MoReFlow demonstrates strong generalization and fine-grained control, even across drastically distinct morphologies (Kim et al., 29 Sep 2025).

Notable limitations persist:

• Pairwise pipelines: The majority of flow- and prior-based approaches train a separate model per character pair; many-to-many, universal models are emergent but not yet standard (Kim et al., 29 Sep 2025, Cao et al., 27 May 2025).
• Physical constraint integration: Several learned models omit explicit dynamics; physically plausible motion is often enforced by post-hoc filtering or downstream RL, rather than end-to-end differentiable simulation.
• Generalization across extreme topologies: While state-of-the-art models generalize to held-out structures (e.g., MPI-INF-3DHP in HuMoT, unknown robots in G-DReaM), convergence is slower and errors grow for non-anthropomorphic or ambiguous joint correspondences (Mourot et al., 2023, Cao et al., 27 May 2025).
• Contact-rich and interactive motions: Most pipelines focus on single-character, contact-free motion; explicit modeling of object or multi-character interaction remains challenging.

6. Applications and Future Directions

GMR unlocks a wide spectrum of applications:

• Animation and visual effects: seamless cross-character motion asset reuse and editing, robust denoising and joint upsampling, advanced image- or video-based motion synthesis (Ren et al., 2021, Mourot et al., 2023).
• Robotics and teleoperation: transferable policy training without manual retargeting, imitation learning under embodiment gaps, sim-to-real transfer of bipedal, quadrupedal, and non-humanoid controller priors (Araujo et al., 2 Oct 2025, Li et al., 2023).
• Virtual/augmented reality: cross-avatar retargeting for user-driven VR avatars, expressive adaptation to arbitrary rigs, and global view/structure disentanglement (Zhu et al., 2021).
• Motion data organization: motion normalization, clustering, retrieval, and semantic search in canonicalized or latent spaces (Zhu et al., 2021, Mourot et al., 2023).

Key future research directions include:

• Learning many-to-many, universal retargeters: scaling from character pairs to truly foundational models that generalize across species and morphologies (Cao et al., 27 May 2025).
• Contact, interaction, and multi-agent extension: integrating object, environment, and up-to-n character interactions into the retargeting process.
• Incorporation of differentiated physical and dynamic simulation: incorporating physics simulators or differentiable environments in training for stronger physical guarantees and dynamic adaptation.
• Automated joint correspondence and structural adaptation: learning correspondences and optimal mapping across arbitrary skeletons, including limbs-less or amorphous agents.

General Motion Retargeting has evolved into a mature subfield bridging animation, robotics, geometric deep learning, and differentiable optimization, with unified architectures now able to span an unprecedented range of embodiments, tasks, and motion genres. Continued advances are likely to further erase boundaries between human, animal, and robotic motion domains, both in structured environments and open real-world settings.