Open-World Motion Transfer

• Open-world motion transfer is a technique for mapping spatio-temporal motion patterns between dissimilar entities by leveraging sparse correspondences and flexible motion representations.
• Key methodologies include sparse correspondence-driven mapping, tokenization of motion sequences, and self-supervised adaptation to effectively bridge structural and semantic gaps.
• Empirical evaluations show that disentangled motion representations and generative adaptations boost fidelity and realism, enhancing applications in animation, robotics, and cross-modal synthesis.

Open-world motion transfer refers to the generalized problem of transferring spatio-temporal motion patterns from arbitrary source entities (which may have unknown, dissimilar, or out-of-distribution articulations, appearance, or category) to arbitrary target entities, potentially traversing substantial gaps in topology, shape, semantics, and context. Unlike conventional motion retargeting or style transfer, which typically assume shared skeletons, articulated structures, or style taxonomy, open-world motion transfer aims for category-agnostic, correspondence-light transfer that robustly adapts motion across modalities—including humans, animals, objects, and scenes—without the need for paired data, directly paired structure, or strong supervision.

1. Definitions, Scope, and Formalization

Open-world motion transfer encompasses the procedural mapping of source motion $M_s$ to a target representation $M_t$ whose structural, appearance, and semantic domains may differ arbitrarily. This includes but is not limited to:

• Skeleton-agnostic human/humanoid motion transfer
• Cross-categorical animal motion synthesis (e.g., human→chimpanzee, horse→dog)
• Rigid and non-rigid object motion transfer (vehicles, tools, arbitrary deformations)
• Text-conditioned motion transfer (matching a motion prior to a natural language prompt)
• Motion transfer to images, videos, or 3D reconstructed scenes, possibly with only monocular input.

Motion2Motion formalizes this in the context of animation as seeking a function $f: M_s, \Psi \mapsto M_t$ where $\Psi$ is a sparse set of correspondence constraints (e.g., a small set of bone or semantic-keypoint alignments) and $M_t$ is realized in the configuration space of the target topology (Chen et al., 18 Aug 2025). The challenge arises from the lack of fixed, dense mappings and the inherent structural mismatch between domains.

2. Representative Methodologies

Methodological paradigms for open-world motion transfer differ by application domain, representation, and the degree of structural or appearance correspondence they require. Key strategies include:

• Sparse correspondence-driven approaches (Motion2Motion): Given source and target skeletons $S_s, S_t$, and sparse correspondences $\mathcal{C}$, retarget source bone rotations and root trajectories onto matched bones, propagate to the rest of the skeleton via inverse kinematics or forward dynamics, and optimize for local posture continuity and motion realism. This facilitates transfer even across topologically non-isomorphic skeletons (Chen et al., 18 Aug 2025).
• Tokenization and latent factorization (MTVCrafter, DisMo): Encode spatio-temporal motion sequences as discrete or continuous tokens independent of appearance (e.g., VQ-VAE motion tokens in 4D space, or disentangled motion vectors from a dedicated encoder). These tokens condition video diffusion models or LoRA adapters, driving video synthesis or image animation even for novel categories or cross-modal targets (Ding et al., 15 May 2025, Ressler-Antal et al., 28 Nov 2025).
• Self-supervised and adaptation-centric approaches (SETA): Sequential self-supervised test-time adaptation disentangles appearance from pose distribution, leveraging GRAM and ReID feature losses for robust adaptation to out-of-distribution subjects and skeletons (Chen et al., 2023).
• Flow and correspondence-based transfer (RoPECraft, MotionShot): Employ dense optical flow or hybrid semantic-morphological keypoint alignment to warp positional encodings or feature grids, thereby embedding motion priors into transformer/attention architectures without weight updates and with minimal or no training overhead (Gokmen et al., 19 May 2025, Liu et al., 22 Jul 2025).
• Physics, 3D geometry, and scene-level transfer (Motion Marionette): Lift both source video and target image(s) into a unified 3D workspace (e.g., Gaussian splat representation), estimate a spatio-temporal motion prior (sequence of rigid transforms or deformation fields), and drive target deformation via analytic velocity fields with dynamical refinement (PBD) (Wang et al., 25 Nov 2025).
• Generative/discriminative hybrid frameworks (REMOT, Dance Dance Generation): Decompose synthesis into semantically aligned parts, perform GAN-based compositional fusion with explicit alignment (global/texture), and enforce appearance and pose fidelity using adversarial, perceptual, and feature-matching losses (Yang et al., 2022, Zhou et al., 2019).

3. Abstract Motion Representations and Disentanglement

Recent advances emphasize the importance of disentangling motion from appearance, structure, and content for open-world transferability:

• Dual-stream encoding (DisMo): The motion encoder $M_\theta$ outputs per-frame motion tokens $z^m_t$ from the full clip, while the content encoder $E_c$ provides static, instance-dependent features. DisMo's reconstruction objective ensures that $z^m_t$ must encode only temporal change, enforced through data augmentations and explicit flow-matching reconstruction, precluding leakage of appearance or static information into the motion channel. Adapters (e.g., LoRA) inject these motion embeddings into large frozen video generation backbones, achieving cross-domain generalization and high motion fidelity (Ressler-Antal et al., 28 Nov 2025).
• Tokenization in spatio-temporal domains (MTVCrafter, Motion Puzzle): 4DMoT in MTVCrafter quantizes motion as 4D (space+time) tokens, enabling transformer-based cross-attention with precise 4D RoPE encodings; similarly, Motion Puzzle partitions motion by body part, encoding and mixing style/content at the sub-structure level, yielding strong per-part, open-world transfer without paired or labeled data (Ding et al., 15 May 2025, Jang et al., 2022).
• Feature loss and space-time priors (Space-Time Diffusion, RoPECraft): Matching temporal feature differences or trajectories directly in the latent or attention space (e.g., SMM difference loss) allows models to preserve only relative motion, decoupling spatial layout or shape from motion though manipulation of transformer positional encodings (Yatim et al., 2023, Gokmen et al., 19 May 2025).

4. Cross-Category, Cross-Topology, and Embodiment Challenges

Open-world transfer requires robust handling of structural divergence:

• Skeleton/topology mismatch (Motion2Motion, Motion Puzzle): In Motion2Motion, sparse correspondences suffice, with local propagation (e.g., via dynamical or kinematic chains) to handle unmatched bones; cycle-consistent and root-motion preserving constraints enforce fidelity in both similar-skeleton and cross-species settings (Chen et al., 18 Aug 2025). Motion Puzzle's part-based graph convolutions and attention disentangle per-part style from kinematic chain content, ensuring that dynamic local behaviors are preserved across even highly varying actions or styles (Jang et al., 2022).
• Habit and behavioral prior preservation (Behave Your Motion): Cross-category animal motion transfer demands not only action-level content matching but also respect for species-specific habitual patterns. This is addressed with habit-preserving modules (normalizing flow priors over latent habit space $z_c$), LLM-driven semantic embeddings, and VQ-VAE backbones. For previously unobserved categories, the system leverages language-derived embeddings to select appropriate latent priors by textual proximity, ensuring plausible habits in the absence of explicit motion data (Zhang et al., 10 Jul 2025).
• Robotic embodiment and manipulation (MotionTrans, Kinesthetic Transfer): Transfer to robotic platforms introduces additional constraints (actuator space, joint limits, collision-avoidance). MotionTrans achieves human-to-robot cotraining by transforming VR-collected human motion into robot-proprioceptive state/action chunks, and leverages large dataset coverage to interpolate policy motion manifolds (Yuan et al., 22 Sep 2025). In kinesthetic transfer planning (Das et al., 13 Mar 2025), the key is extraction of critical-task frames (“C-frames”) anchored to task-relevant object locations, with motion expressed as relative SE(3) screw actions transferable across differing object geometries, with robust collison-checking and segmentation.

5. Evaluation Protocols, Metrics, and Empirical Findings

Open-world motion transfer methods are assessed using a combination of classical, feature-based, and semantic metrics:

Metric	Purpose	Representative Papers
Motion Fidelity (Chamfer, MF)	Quantitative similarity between source and output trajectories	(Ressler-Antal et al., 28 Nov 2025, Gokmen et al., 19 May 2025, Yatim et al., 2023)
Content/Input Adherence (CRA, CLIP Sim.)	Content recognition or prompt matching accuracy	(Yatim et al., 2023, Jang et al., 2022)
Style/Habit Realism (SRA, Intra-FID, Downstream FID)	Realism of habitual behaviors or style adherence	(Zhang et al., 10 Jul 2025, Jang et al., 2022)
Video-level FID, FVD, LPIPS, SSIM, PSNR	Appearance and perceptual metrics over generated clips	(Ding et al., 15 May 2025, Yang et al., 2022, Jang et al., 2022)
Temporal Consistency (TCM, SMM-diff)	Frame-to-frame smoothness, motion continuity	(Yang et al., 2022, Yatim et al., 2023)
Success Rate, Progress Score (Robotics)	Task completion and progression in manipulation	(Yuan et al., 22 Sep 2025, Das et al., 13 Mar 2025)

Empirical results demonstrate that approaches using disentangled motion encodings and domain-agnostic priors achieve higher fidelity in cross-category or skeleton-divergent situations. For example, DisMo achieves motion-fidelity 0.75, topping baselines on DAVIS/OpenVid-1M (Ressler-Antal et al., 28 Nov 2025); Behave Your Motion’s habit module produces cross-category FIDs 10× lower than kinematic or style-transfer baselines (Zhang et al., 10 Jul 2025); RoPECraft consistently outperforms strong prior methods in MF, FTD, and content-debiased FVD (Gokmen et al., 19 May 2025). Human and user studies corroborate higher subjective ratings for temporal consistency and prompt-action alignment.

6. Practical Implications, Limitations, and Future Prospects

Open-world motion transfer is enabling for digital human animation, creative content generation, robotics, cross-species analysis, and broader content creation. Transfer pipelines are growing increasingly modular and agnostic to structure/appearance, often requiring only sparse correspondences or motion priors. Notable limitations persist:

• Many frameworks still rely on pre-existing 3D models or motion capture for full generalization (e.g., SMPL for humans, implicit skeletal graphs for animals, 3DGS for rigid body transfer).
• Extremely high structural divergence (e.g., non-rigid, multi-body fusion) may degrade fidelity; generalization to highly non-convex or occluded targets remains challenging.
• Full semantic understanding of "habit" or style for previously unseen species or objects remains open, depending on LLM generalization and the breadth of habit-labeled datasets.
• Zero-shot generalization in robotics depends on sufficiently dense coverage of human and robot motion manifolds—current scaling laws suggest more tasks/data continue to improve transfer, but embodiment mismatches remain a bottleneck.

A plausible implication is that as abstract, content-invariant motion representations (e.g., DisMo’s motion tokens, habit-normalizing priors, or cross-category part style modules) reach scale alongside data-rich backbones, open-world motion transfer will become a universal primitive for animation, virtual agent control, cross-modal content generation, and multi-agent robotics.

---

Key References

• Motion2Motion: Cross-topology Motion Transfer with Sparse Correspondence (Chen et al., 18 Aug 2025)
• MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation (Ding et al., 15 May 2025)
• DisMo: Disentangled Motion Representations for Open-World Motion Transfer (Ressler-Antal et al., 28 Nov 2025)
• MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation (Liu et al., 22 Jul 2025)
• RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE Optimization on Diffusion Transformers (Gokmen et al., 19 May 2025)
• MotionTrans: Human VR Data Enable Motion-Level Learning for Robotic Manipulation Policies (Yuan et al., 22 Sep 2025)
• Behave Your Motion: Habit-preserved Cross-category Animal Motion Transfer (Zhang et al., 10 Jul 2025)
• Motion Marionette: Rethinking Rigid Motion Transfer via Prior Guidance (Wang et al., 25 Nov 2025)
• REMOT: A Region-to-Whole Framework for Realistic Human Motion Transfer (Yang et al., 2022)
• Motion Puzzle: Arbitrary Motion Style Transfer by Body Part (Jang et al., 2022)
• Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer (Yatim et al., 2023)
• Open-World Pose Transfer via Sequential Test-Time Adaption (Chen et al., 2023)
• Dance Dance Generation: Motion Transfer for Internet Videos (Zhou et al., 2019)
• Transferring Kinesthetic Demonstrations across Diverse Objects for Manipulation Planning (Das et al., 13 Mar 2025)