Papers
Topics
Authors
Recent
Search
2000 character limit reached

Any2Any: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking

Published 22 May 2026 in cs.RO and cs.AI | (2605.23733v1)

Abstract: Whole-body tracking (WBT) models have become a key foundation for humanoid robots, enabling them to imitate diverse motions with high fidelity. Training such models from scratch requires large-scale data and computation, making rapid deployment on new humanoid platforms costly. This raises a natural question: Can pretrained WBT models transfer across embodiments with minimal adaptation? To answer this question, we propose Any2Any, a paradigm that efficiently transfers an existing WBT specialist to a new humanoid embodiment with only a small amount of data and compute. Any2Any first performs kinematic alignment between source and target humanoids, aligning their input and output spaces so that the pretrained source policy can be meaningfully reused on the target embodiment.Any2Any then performs dynamics adaptation by applying lightweight parameter-efficient fine-tuning (PEFT) components to selected dynamics-sensitive modules, preserving useful behavioral priors while enabling targeted adaptation to the target robot. Extensive experiments on multiple humanoid platforms and pretrained backbones show that Any2Any substantially accelerates convergence and reduces training cost compared with training from scratch, while achieving competitive or superior tracking performance. Notably, using only 1% of the compute and data required for full training, Any2Any successfully transfers Sonic models pre-trained on Unitree G1 to LimX Oli and LimX Luna. These results suggest that pretrained WBT specialists can be efficiently reused across embodiments, providing a scalable path toward deploying humanoid whole-body control on new robots.

Authors (4)

Summary

  • The paper introduces ANY2ANY, a cross-embodiment transfer framework that adapts pretrained whole-body tracking policies with minimal data and compute.
  • It achieves kinematic alignment by remapping observations and aligning joint spaces to compensate for structural differences between robot embodiments.
  • It employs lightweight PEFT via LoRA to fine-tune dynamics-sensitive modules, reducing base-position error by 40% using only 1% of full training resources.

Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking: The ANY2ANY Framework

Context and Motivation

The development of large-scale whole-body tracking (WBT) policies for humanoid robots has enabled high-fidelity motion imitation and control, but typically demands massive datasets, intensive computation, and robot-specific tuning. This specificity creates barriers for rapid deployment across new humanoid platforms, as policies are tightly coupled to the morphology, actuator configuration, and sensor interfaces of their source robot. Previous generalization techniques—such as morphology randomization and training universal controllers—require costly retraining on multi-embodiment data, limiting practicality.

ANY2ANY addresses this challenge by proposing a parameter-efficient, post-training cross-embodiment transfer framework for WBT. Rather than retraining from scratch or building generalist models, ANY2ANY aims to efficiently adapt a pretrained WBT specialist to a new humanoid embodiment using minimal data and compute, preserving valuable behavioral priors while resolving kinematic and dynamic mismatches between source and target robots.

Technical Approach

Kinematic Alignment

ANY2ANY first establishes a unified kinematic semantic space via two-level alignment:

  • Observation Layout Alignment: Remaps and reorganizes target robot observations to match the semantic convention of the source policy, ensuring consistency in input space (e.g., base state, joint positions, action history).
  • Joint-Level Alignment: Constructs explicit mappings between source and target joint spaces using structured components:
    • Sparse scattering matrices manage joint correspondence and ordering.
    • Hip decoupling matrices compensate for structural differences (e.g., inclined hip axes).
    • Closed-chain Jacobian corrections address parallel joint coupling in robots with closed-chain mechanisms.

This alignment ensures the pretrained policy backbone receives inputs and outputs in a consistent, source-aligned joint convention, reducing structural mismatches and isolating residual dynamics gaps.

Dynamics Adaptation via Parameter-Efficient Fine-Tuning

With kinematic alignment in place, ANY2ANY treats the remaining embodiment gap as a compact, dynamics-specific residual. Rather than full-policy fine-tuning—which tends to overwrite transferable priors—the approach adapts only modules sensitive to target robot dynamics via lightweight PEFT methods (primarily LoRA).

Formally, LoRA decomposes the linear projection in each adapted module as W′=W+BAW' = W + BA, with A,BA, B as low-rank factors. This minimal intervention preserves the pretrained WBT prior in the backbone while specializing the policy to the target’s mass distribution, actuator response, and contact behavior.

Training and Evaluation

Actors are trained via PPO on retargeted AMASS motion datasets, maintaining identical reward formulations and training hyperparameters across source and target. ANY2ANY is benchmarked against training-from-scratch and other PEFT approaches (Adapter, Prefix-Tuning), using both Transformer and MLP WBT backbones.

Experimental Results

Transfer Performance

ANY2ANY demonstrates robust transferability across five source-target pairs spanning LimX Oli, LimX Luna, Unitree G1, and Unitree H1 robots, and two pretrained backbones (Sonic and Oli-WBT). Using only 1% of the compute and data required for full training, ANY2ANY achieves competitive or superior motion tracking accuracy compared to scratch-trained specialists.

Performance metrics show:

  • Substantially accelerated convergence and higher asymptotic rewards for tracking, particularly in early training stages.
  • Lower mean per-joint position error (MPJPE), base position/orientation errors, and improved velocity and acceleration smoothness, supporting real-world deployment suitability.
  • Consistent efficacy across both Transformer and MLP backbones, as well as diverse motion classes (walking, running, manipulation, squatting, bending).

Ablation Studies

Kinematic Alignment

Aligning observation and action spaces is essential; full fine-tuning without alignment shows limited convergence and lower reward. Introducing alignment enables effective reuse of motion priors across embodiments.

PEFT Variants

LoRA delivers the best balance of stability, efficiency, and tracking quality. Adapter modules underperform due to optimization challenges in closed-loop control. Prefix-Tuning fails to compensate for embodiment-level dynamics.

Scope of Adaptation

Optimal transfer is realized by injecting LoRA into the actor backbone, actor proprioception input projection, action output head, and critic backbone, targeting modules directly coupled to robot dynamics while preserving reference-motion encoding.

Data/Compute Efficiency

ANY2ANY is markedly more sample-efficient than scratch training, especially in low-data regimes, reducing the base-position error by 40% when trained with a small fraction of the dataset. Sensitivity to GPU compute and sampling budgets is reduced, suggesting that pretrained motion priors narrow the optimization search space for cross-embodiment correction.

Practical and Theoretical Implications

ANY2ANY constitutes a scalable paradigm for cross-embodiment deployment of high-fidelity whole-body controllers. This framework leverages the reusable structure of motion priors learned within WBT specialists, facilitating rapid adaptation to new robots with minimal engineering overhead.

Practically, ANY2ANY enables:

  • Cost-effective onboarding of new humanoid platforms without extensive retraining.
  • Preservation of robust whole-body coordination and balance from pretrained policies.
  • Real-world applicability across diverse humanoid morphologies.

Theoretically, the approach exemplifies how decoupling kinematics from dynamics enables modular policy adaptation, with low-dimensional correction for embodiment-specific physical properties. This paves the way for further research in foundation policy adaptation, robot-specific residual learning, and generalization across even broader classes of robotic morphologies.

Future directions may include:

  • Extending the alignment-adaptation paradigm to manipulation policy transfer.
  • Incorporating online sim-to-real adaptation for dynamic environments.
  • Exploiting synergistic combinations of PEFT with broader robot foundation models, meta-learning, or continual learning.

Conclusion

ANY2ANY introduces a principled framework for efficient post-training transfer of whole-body tracking policies across humanoid embodiments (2605.23733). By structurally aligning kinematic semantics and adapting dynamics-sensitive modules via PEFT, it preserves valuable behavioral priors and achieves rapid, accurate motion tracking on new robot platforms. This paradigm provides a practical bridge between pretrained WBT specialists and scalable cross-embodiment application, informing both practical deployment and foundational research into transferable motor control for humanoid robots.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Explain it Like I'm 14

Overview

This paper is about teaching different humanoid robots to move their whole bodies in smooth, natural ways by reusing an already trained “movement brain” instead of training a new one from scratch. The authors introduce a method called Any2Any that makes it fast and cheap to transfer a powerful whole‑body tracking model from one humanoid robot to another, even if their bodies aren’t exactly the same.

Key Questions

The paper asks three simple questions:

  • If we have a great motion model trained on one humanoid robot, can we make it work well on a different robot with only a little extra training?
  • What’s the best way to fix the differences between robots so the old model can still understand sensors and control joints correctly?
  • Which small parts of the model should we tweak to handle each robot’s unique physical “feel” without losing the useful skills it already learned?

How It Works (Methods)

Think of a whole‑body tracking (WBT) model as a smart controller that watches a “reference motion” (like a dance or a walk) and then tells the robot how to move its joints to copy that motion while staying balanced.

Any2Any reuses a pretrained WBT model by doing two main steps:

Kinematic Alignment (lining up “what” the model sees and controls)

Robots often have different numbers of joints, joint orders, and ways they list sensor data. Kinematics is about the geometry of movement—how parts are connected and move together.

  • First, the authors rearrange the target robot’s sensor inputs so they match the format the pretrained model expects. This is like re‑labeling and re‑ordering a control panel so all buttons are in familiar places.
  • Second, they map each joint on the target robot to the right joint in the source robot’s model. If some joints don’t match perfectly (like hips that are angled differently, or ankle mechanisms with special linkages), they use small conversion rules so the signals make sense in the source robot’s “joint dictionary.”

In simple terms, kinematic alignment is a translator that makes the new robot’s body look “familiar” to the old model without changing the model itself.

Dynamic Adaptation (tuning “how” the model reacts to physics)

Dynamics is about forces, masses, and how the robot actually responds when it moves and touches the ground. Even if two robots look similar, they can feel different—like two people with different strength and balance.

Instead of retraining the whole model, Any2Any uses lightweight, parameter‑efficient fine‑tuning (PEFT). The main technique here is called LoRA, which adds tiny “attach‑on” pieces to specific parts of the network:

  • LoRA lets the model learn small, low‑rank corrections to handle the new robot’s mass distribution, motor behavior, and contact with the ground.
  • Most of the original model stays frozen, so it keeps its valuable motion skills (balance, coordination, timing), and only the parts sensitive to physics get gently adjusted.

Think of LoRA like putting small, custom shock absorbers in the right spots, rather than rebuilding the entire car.

Training and Evaluation Setup

  • The team tested on several humanoid robots: LimX Oli (31 joints), LimX Luna (27 joints), Unitree G1, and Unitree H1.
  • They reused two big pretrained models:
    • Sonic (trained on Unitree G1; used ~9,000 GPU hours and ~700 hours of motion data).
    • Oli‑WBT (trained on LimX Oli with ~500 hours of motion data).
  • Motions came from the AMASS dataset, which has lots of human movements retargeted to robots.
  • They trained with PPO (a popular reinforcement learning method) in simulation, keeping rewards and setups consistent so comparisons were fair.

Main Findings

  • Fast, low‑cost transfer: Any2Any can move a trained model from one robot to another using about 1% of the original training compute and data. For example, adapting Sonic from Unitree G1 to LimX Oli or Luna took ~80 GPU hours (vs ~9,000 originally) and ~8 hours of data (vs ~700 originally).
  • Better than training from scratch: Across five transfer cases, Any2Any reached higher tracking rewards, lower errors, and faster convergence than starting with a blank model. It tracked diverse motions like walking, running, squats, bending, and simple manipulations more reliably.
  • Kinematic alignment is essential: If you skip the “translator” step, fine‑tuning struggles. Once inputs/outputs are aligned, even small updates work well.
  • LoRA works best among PEFT choices: Compared to other small‑tuning methods (Adapters, Prefix Tuning), LoRA achieved the best balance of stability, speed, and final performance.
  • Where to tweak matters: The best results came from inserting LoRA into parts most tied to physics and feedback—like the proprioception (the robot’s body‑state sensors), the action output head, and the critic (the part that judges how good the state is). The motion‑reference encoder (the “what to imitate” stream) often could stay frozen.
  • More data or GPUs help, but Any2Any needs less: It performed well even with small datasets and lower sampling rates, dropping less in performance than training‑from‑scratch when compute was cut. That means it’s more robust and sample‑efficient.

Why It Matters

Training top‑quality whole‑body control models from scratch is expensive and slow. Any2Any shows that:

  • You can reuse powerful motion skills across different humanoids without redoing everything.
  • New robots can be deployed faster, with far less data and compute.
  • This creates a practical path to scaling humanoid capabilities: build a few strong “movement brains,” then adapt them to many robots with small, smart tweaks.

In short, Any2Any makes it easier to bring natural, full‑body movement to new humanoid robots, speeding up progress in teleoperation, motion imitation, and complex tasks that combine walking and using arms—without the massive costs of starting over every time.

View Paper Prompt  View All Prompts 

Knowledge Gaps

Knowledge gaps, limitations, and open questions

Below is a single, concrete list of what remains missing, uncertain, or unexplored, framed to guide actionable future research.

  • Automation of kinematic alignment: The two-level alignment requires manual/instrumented joint correspondences, hip-axis angles, and closed-chain Jacobians. How can these mappings (Tr, Dr, Jr) be discovered automatically from CAD/URDF/measurements or learned from data for arbitrary robots without hand-crafting?
  • Coverage beyond “similar humanoid topology”: The method assumes broadly similar humanoid structures. How does it scale to large topology differences (e.g., extra/missing limbs, highly different limb lengths, articulated hands with many DoFs, passive/underactuated joints, tendon-driven or compliant mechanisms), or even to non-humanoid morphologies?
  • Handling large DoF mismatches and redundancy: The scattering/padding strategy discards unmatched joints and pads missing ones. What is the performance impact for large disparities (e.g., mapping 27-DoF to 45-DoF with many redundant DoFs)? Can alignment preserve fine-grained capabilities (e.g., dexterous hands) without losing fidelity?
  • General closed-chain treatment: The parallel-joint correction uses a closed-chain Jacobian Jr but offers no general recipe for deriving/validating Jr across diverse mechanisms (parallelogram ankles, waist, shoulder couplings). Can a systematic pipeline infer Jr from kinematic constraints or identify it from motion data?
  • Limits of “reference-stream is embodiment-agnostic” assumption: The approach freezes motion encoders, assuming kinematic features transfer. For larger embodiment gaps, when does this assumption break? What are failure modes where the reference encoder itself must adapt?
  • Joint limits, axis orientations, and reachability: Beyond hip decoupling, many joints differ in limit ranges, axis conventions, and feasible workspaces. How should alignment incorporate joint-limit maps and axis transformations to avoid infeasible or saturating actions on the target robot?
  • Sensor and actuator heterogeneity: Alignment reorders inputs but does not address sensor biases/noise, IMU placement differences, actuation latencies, or control mode mismatches (position vs torque control). What additional normalization or adaptation is needed to handle heterogeneous sensing/actuation and timing?
  • Reward and domain randomization transfer: The adaptation keeps the source reward and DR unchanged. How suboptimal is this for the target robot’s dynamics? What procedures jointly retune rewards and DR for the target without losing the source prior?
  • Sensitivity to retargeting quality: All results rely on AMASS retargeted via GMR. How does transfer degrade with less accurate retargeting, and can joint optimization of retargeting and policy adaptation outperform the current decoupled pipeline?
  • PEFT design guidelines: LoRA rank k and injection locations are chosen empirically (e.g., S7). Can we develop principled, generalizable criteria or automated module-selection (e.g., gradient/curvature-based) to determine where and how much capacity to allocate?
  • Alternative and complementary PEFT mechanisms: Only LoRA, Adapters, and Prefix were tested. How do bias-only tuning, norm-only tuning, FiLM/scale-shift layers, low-rank residual attention, or adapter fusion compare for WBT under embodiment shifts?
  • Capacity limits of linear low-rank updates: For large dynamics gaps, when does LoRA’s linear correction become insufficient? What are the trade-offs between higher LoRA ranks, nonlinear adapters, or limited full fine-tuning, and how to detect needed capacity on-the-fly?
  • Physical parameter identification vs black-box adaptation: The method models dynamics gaps implicitly via LoRA. Can we estimate a compact set of physical parameters (masses, inertias, actuator models) and condition the policy on an identified “robot embedding” to achieve more interpretable and data-efficient transfer?
  • Stability and safety guarantees: No theoretical guarantees are provided for closed-loop balance, passivity, or safety during adaptation/deployment. Can we incorporate control-theoretic constraints (e.g., CLF/CBF, passivity layers) or provide probabilistic stability guarantees under embodiment changes?
  • Sim-to-real quantification: Although deployment on hardware is mentioned, the paper lacks detailed real-world metrics (tracking errors, fall rates, energy consumption, thermal limits, wear, latency margins) and robustness tests (terrain, disturbances, contact variability). What is the actual sim-to-real performance and failure modes?
  • Real-data adaptation: The adaptation is driven by simulation and retargeted motions. Can ANY2ANY be specialized with small amounts of real robot data (e.g., a few minutes of teleop or kinesthetic demos), and how does mixed sim+real fine-tuning compare?
  • Robust contact modeling: Differences in foot geometry, sole compliance, friction, and ground interaction are not explicitly adapted. How can contact models or contact-aware features be adapted or identified to improve robustness across embodiments and surfaces?
  • Action space assumptions: The method outputs joint position offsets. How does it extend to torque/impedance controllers or mixed-mode actuation? Are there transfer-specific action regularizers needed to respect target actuator limits and bandwidth?
  • Latency and control-rate effects: The added alignment and PEFT layers change inference graphs. How do they impact real-time control at high servo rates, and what optimizations are required to meet tight latency budgets on embedded hardware?
  • Evaluation breadth and diversity: Only five transfer pairs across four humanoids are tested. How does performance change across a wider set of embodiments (different sizes, mass distributions, foot shapes), terrains, and motion classes (acrobatics, fast running, heavy manipulation)?
  • Generalization to tasks beyond tracking: The paper positions WBT as a motor prior for loco-manipulation and teleop but does not evaluate transfer on downstream tasks. How well does ANY2ANY- adapted WBT support generalization to task objectives beyond motion imitation?
  • Composability and multi-robot scaling: Each target uses its own PEFT parameters. Can we build a single multi-embodiment adaptor (e.g., with a learned robot embedding or router) that composes across many robots without per-robot PEFT sets?
  • Catastrophic interference across targets: If one wishes to sequentially adapt the same backbone to multiple robots, how to prevent interference when sharing components or merging multiple PEFT adapters (e.g., adapter fusion or parameter superposition)?
  • Hyperparameter sensitivity and reproducibility: The paper provides limited analysis of sensitivity to LoRA rank, learning rates, PPO settings, and DR ranges, and limited reporting of seed variance. What are robust defaults and variance bounds across embodiments?
  • Compute and data accounting: The “~1% compute/data” claim lacks precise accounting (wall time, frames, GPUs, environment steps, and memory). Standardized reporting is needed to permit fair comparisons and reproducibility.
  • Failure analysis: The paper emphasizes successes but lacks systematic failure cases (e.g., specific motions or embodiments where transfer fails) and diagnostics to guide corrective design (alignment issues vs dynamics capacity vs reward mismatch).
  • Automated discovery of dynamics-sensitive modules: The best injection pattern (S7) emerged empirically. Can techniques like network attribution, Fisher information, or influence functions identify dynamics-critical pathways to target with PEFT?
  • Interaction with motion diversity: Data-scaling ablations vary total frames, not motion diversity. How does the breadth of motion classes vs repeated frames affect transfer quality and sample efficiency?
  • Privileged critic transfer: The critic uses privileged info in sim. On hardware, such signals may be unavailable. How does this sim-only privilege affect learned policies after adaptation, and can we design privilege-agnostic critics for better real deployment?
  • Safety during data collection: The method assumes abundant sim data. For real data, how to ensure safe exploration and prevent hardware damage during adaptation with limited samples?
  • Long-horizon robustness and drift: Closed-loop WBT can drift over long horizons due to small model mismatches. Are there mechanisms (e.g., state-estimation adaptation, integral correction, online residual learning) to mitigate long-horizon drift after embodiment transfer?
View Paper Prompt  View All Prompts 

Practical Applications

Immediate Applications

The following applications can be deployed now with modest engineering effort, leveraging the paper’s kinematic alignment + PEFT (LoRA) approach and reported data/compute reductions (~1% of original training cost).

  • Cross-embodiment rollout of humanoid controllers across a product line (Robotics, Manufacturing)
    • What it looks like: OEMs with multiple humanoid SKUs adapt a single pretrained whole-body tracker (WBT) to new models (e.g., “G1→Oli”, “G1→Luna”) using the two-step Any2Any workflow (observation/action remapping + LoRA on dynamics-sensitive modules).
    • Tools/products/workflows: Embodiment Adapter SDK (scattering/decoupling/Jacobian mapping utilities); LoRA-based “Dynamics Tuner” that targets proprioception input, actor backbone, action head, and critic backbone; CI/CD pipelines for per-embodiment adapter checkpoints.
    • Assumptions/dependencies: Access to a strong source WBT checkpoint (e.g., SONIC); accurate kinematic specs (DoFs, joint conventions, closed-chain linkages); motion retargeting pipeline (e.g., AMASS + GMR); sim2real gap handling and safety validation on hardware.
  • Rapid porting of teleoperation and motion libraries between humanoids (Entertainment, Telepresence, Software)
    • What it looks like: Studios and teleop providers reuse large motion libraries by aligning kinematics and fine-tuning only LoRA blocks; reduces re-recording and re-training.
    • Tools/products/workflows: Motion-retarget-to-WBT workflow templates; policy zoo with per-robot LoRA adapters; VR teleop stack updated with kinematic alignment layer.
    • Assumptions/dependencies: Semantically consistent observation layouts; reliable reference motion encoders; latency and stability constraints for teleop maintained after adaptation.
  • Warehouse/factory pilots with fast controller onboarding for new hardware (Logistics, Industrial Robotics)
    • What it looks like: System integrators adapt locomotion and basic loco-manipulation tracking to site-specific humanoids in days, not weeks.
    • Tools/products/workflows: On-prem Isaac-based adaptation jobs (4×A100 or smaller GPU pools); domain randomization presets; HIL (hardware-in-the-loop) test suites for balance/stability.
    • Assumptions/dependencies: Comparable morphologies/topologies; environment-specific safety envelopes; torque/impedance controllers compatible with policy outputs.
  • Field service and maintenance: swap-in replacement units with minimal retraining (Robotics Services)
    • What it looks like: When a robot variant changes (actuator models, mass properties), deploy a small LoRA update rather than full retraining; reduces downtime.
    • Tools/products/workflows: On-device calibration runs to collect a few hours of motion data; auto-selection of LoRA ranks and injection scope (S7 profile in paper).
    • Assumptions/dependencies: Stable kinematic mapping; lightweight data collection procedures; versioned adapter checkpoints tied to hardware serials.
  • Academic replication and benchmarks for cross-embodiment transfer (Academia)
    • What it looks like: Labs replicate WBT results across different humanoids without reproducing massive pretraining; comparative studies on module injection sites and data scales.
    • Tools/products/workflows: Open benchmark suite (AMASS+GMR tasks, reward configs, Isaac Lab envs), baseline adapters for G1↔H1↔Oli↔Luna; standardized reporting (success rate, MPJPE, base pose errors).
    • Assumptions/dependencies: Licensing/access to source policies; shared evaluation protocols and seeds; transparent kinematic alignment maps.
  • Simulation platform features for “embodiment alignment” (Software, Simulation)
    • What it looks like: Isaac Lab/MuJoCo wrappers expose plug-and-play observation/action remappers (scattering, hip decoupling, closed-chain Jacobians) and PEFT hooks.
    • Tools/products/workflows: Sim plugins that auto-generate S, D, J matrices from URDFs; GUI for verifying semantic alignment; batch adaptation runners.
    • Assumptions/dependencies: High-fidelity robot descriptions (URDF/SDF); accurate mass/inertia; validated mapping parsers for vendor URDF quirks.
  • Teaching labs and low-budget programs using PEFT-based specialization (Education)
    • What it looks like: Courses run WBT adaptation on commodity GPUs (e.g., 1–2 RTX 4090s) to demonstrate whole-body control and cross-embodiment generalization.
    • Tools/products/workflows: “Any2Any Lab Kit” with toy datasets and reduced-rank LoRA configs; guided assignments for alignment vs. full fine-tuning comparisons.
    • Assumptions/dependencies: Access to simple open-source humanoid models; safe sim settings; pedagogy aligned with PPO and PEFT basics.
  • Early healthcare robotics research using standardized social/assistive motions (Healthcare Research)
    • What it looks like: Research labs port interaction/gestural behaviors across different lab humanoids used in patient-facing studies without retraining from scratch.
    • Tools/products/workflows: Pre-validated social-motion libraries with per-robot adapters; safety monitors for contact/CoM margins; auditor logs of adaptations.
    • Assumptions/dependencies: Strict IRB/safety protocols; low-force, conservative behaviors; morphology sufficiently similar to preserve motion intent.
  • Procurement and funding guidance that prioritizes model portability (Policy, Standards)
    • What it looks like: RFPs and grants encourage/require “embodiment portability” (publish alignment maps, adapter scopes, dataset scales); standardized “embodiment cards.”
    • Tools/products/workflows: Embodiment Card template (DoFs, hip axes, closed-chain notes, observation layout, action scale); reproducibility checklists.
    • Assumptions/dependencies: Community buy-in; IP-friendly disclosure of necessary kinematic/dynamic details; compatibility with vendors’ licensing.
  • ModelOps for robotics: versioning, safety-gated rollout of LoRA adapters (Software, DevOps)
    • What it looks like: Separate lifecycle for base WBT vs. per-robot LoRA weights, canary rollouts, rollback on instability.
    • Tools/products/workflows: Artifact registries tagging base checkpoint + alignment config + adapter; automated hardware regression tests (fall-recovery, foot clearance).
    • Assumptions/dependencies: Reliable telemetry; clear pass/fail gates; operator training for staged deployment.

Long-Term Applications

These concepts are plausible extensions that need further research, scaling, or standardization (e.g., broader morphologies, robust sim2real, certification).

  • Robot “app store” for motions and controllers with one-click embodiment adaptation (Robotics, Software)
    • Vision: Developers publish WBT-based skills; end users select their humanoid and auto-generate alignment + LoRA adapters.
    • Potential products: Marketplace with verified adapters, quality scores by embodiment; subscription for updates.
    • Dependencies: Broad standard for observation/action semantics; robust automated mapping from URDFs; licensing frameworks for model reuse.
  • On-robot continual calibration for wear-and-tear and payload changes (Robotics, Maintenance)
    • Vision: Periodic micro-updates to LoRA adapters compensate for actuator drift, joint friction, or changing payloads.
    • Potential products: Background “Autotune” daemon; drift detectors; safe-learning protocols for online adaptation.
    • Dependencies: Reliable safety envelopes; online data collection without risk; real-time PEFT optimization; certification for adaptive controllers.
  • Cross-domain generalist motor priors with Any2Any-style transfer beyond humanoids (Robotics, General AI)
    • Vision: A unified motion prior spanning humanoids, exoskeletons, and other morphologies with modular kinematic alignment blocks and dynamics adapters.
    • Potential products: Multi-embodiment BFM with embodiment adapters; routing policies that select adapter scopes automatically.
    • Dependencies: Scalable datasets across embodiments; stronger topology-aware representations; sim2real across diverse contacts and actuation.
  • Personalized rehabilitation and exoskeleton gait adaptation (Healthcare)
    • Vision: PEFT enables rapid personalization to an individual’s morphology and device configuration; safer, more natural assistance.
    • Potential products: Clinic-grade alignment wizards; patient-specific LoRA packages; compliance-aware safety layers.
    • Dependencies: Medical-grade validation; tight force/torque limits; interpretable safety monitors; regulatory approval (FDA/CE).
  • Safety certification frameworks for adaptive, cross-embodiment controllers (Policy, Standards)
    • Vision: New standards specify evidence for portability (test suites across embodiments, worst-case stability margins) before public deployment.
    • Potential products: Certification labs for embodiment transfer; standardized stress tests (slopes, pushes, slips) with pass/fail thresholds.
    • Dependencies: Broad stakeholder consensus; liability models for adaptive software; transparent reporting of adapter changes.
  • Morphology-in-the-loop robot co-design (Robotics, Manufacturing)
    • Vision: Joint optimization of robot kinematics and expected adapter complexity/cost; choose designs that minimize required dynamics residuals (LoRA rank).
    • Potential products: CAD plugins that predict alignment difficulty and adapter rank; BOM trade-off tools using “adaptability cost.”
    • Dependencies: Accurate predictors from URDF/CAD to adapter performance; large design-adaptation datasets.
  • Household service robots that inherit a shared WBT prior across vendors (Consumer Robotics)
    • Vision: A family of home robots reuse the same motion priors, localized with tiny adapters; faster updates and safer behaviors across brands.
    • Potential products: Vendor-neutral WBT base with certified vendor adapters; OTA updates to adapter weights.
    • Dependencies: Inter-vendor cooperation; standardized observation/action APIs; robust home-environment sim2real.
  • Energy and cost reductions via compute-light adaptation at scale (Sustainability, Finance)
    • Vision: Industry-wide shift from full retraining to PEFT for new models, cutting GPU-hours and energy consumption.
    • Potential products: ESG reporting of “embodiment-adapted training” savings; internal carbon accounting tied to adaptation metrics.
    • Dependencies: Trust in PEFT quality vs. full retraining; reproducible measurement of performance and safety parity.
  • Automated adapter scope selection and rank tuning (Software Tools, AutoML)
    • Vision: Systems that analyze embodiment gaps (Δη) and propose optimal LoRA injection sites/ranks (e.g., S7-like patterns) automatically.
    • Potential products: “Adapter Planner” that profiles proprioception/action pathways and suggests PEFT topology under data/compute budgets.
    • Dependencies: Reliable diagnostics for dynamics sensitivity; meta-learning over many transfer pairs; avoidance of overfitting.
  • Richer human-in-the-loop motion authoring across embodiments (Creative Tech, XR)
    • Vision: Creators design motions once and preview on many humanoids with faithful tracking; adapters maintain style and timing.
    • Potential products: Cross-embodiment motion editors; style-preserving alignment libraries.
    • Dependencies: High-fidelity reference encoders; metrics for perceptual motion similarity; creator-friendly tooling.

Cross-cutting assumptions and dependencies (impact many applications)

  • Morphology similarity: Best performance when target shares humanoid topology and joint semantics; extreme differences increase adapter capacity needs.
  • Data and sim stack: Access to retargeted motion data (e.g., AMASS+GMR) and high-fidelity simulation (Isaac Lab/MuJoCo) to stage adaptation safely.
  • Safety engineering: Conservative controller limits, fall-detection, and certification paths for public/clinical use.
  • IP/licensing: Legal access to source WBTs and permission to publish/share alignment maps and adapters.
  • Compute/inference: While training compute is reduced, real-time inference must meet control-rate constraints (e.g., 200–500 Hz) on target hardware.
  • Tooling maturity: Availability of robust alignment libraries (scattering/decoupling/Jacobian), adapter versioning, and test suites to reduce integration risk.
View Paper Prompt  View All Prompts 

Glossary

  • Adapter: A parameter-efficient fine-tuning module that inserts small trainable bottlenecks into a frozen network to specialize it to a new task or domain. "Adapter tuning inserts trainable bottleneck odules [41];"
  • AMASS: A large mocap-derived human motion dataset commonly used for motion imitation and retargeting to robots. "All adaptation experiments are driven by the AMASS [48] motion dataset, where human motions are retargeted to each target robot through General motion retargeting [21]."
  • Behavior foundation models (BFMs): Large pretrained models that capture reusable motor skills and priors for rapid adaptation to downstream control tasks. "Behavior foundation models (BFMs) are emerging as a promising paradigm for humanoid control [2, 3]."
  • Closed-chain mechanisms: Robot joint/link configurations that form kinematic loops (e.g., parallelograms), causing dependent joint motions. "some target robots contain closed-chain mechanisms, such as parallelogram-driven ankles or closed-loop waists,"
  • Coriolis/centrifugal coupling: Terms in robot dynamics capturing velocity-dependent forces that couple joint motions. "the Coriolis/centrifugal coupling C(q, q),"
  • Domain randomization: Training technique that randomizes environment/dynamics parameters to improve robustness and transfer. "we keep the action space, observations, reward formulation, PPO hyperparameters, reference- motion sampling, and domain randomization protocol identical to those of the corresponding source pretraining setup,"
  • Dynamics adaptation: The process of fine-tuning parts of a controller to account for differences in physical dynamics between source and target robots. "ANY2ANY then performs dynamics adaptation by ap- plying lightweight parameter-efficient fine-tuning (PEFT) components to selected dynamics-sensitive modules,"
  • Dynamics decoder: A policy submodule that decodes learned representations into action commands while modeling system dynamics. "we employ its Robot Motion Encoder, FSQ bottleneck, and dynamics decoder modules for training and evaluation."
  • Embodiment: The specific physical instantiation of a robot, including morphology, actuators, and sensors. "Beyond training cost, embodiment dependence remains a fundamental limitation of current large- scale WBT policies [11, 12]."
  • Expert routing: Mechanisms that select among multiple specialized sub-policies/experts based on input, often used in large models. "unified action/state rep- resentations with expert routing [39, 18]."
  • FSQ bottleneck: A vector-quantization-style discrete bottleneck (from Sonic) that compresses representations in the policy. "we employ its Robot Motion Encoder, FSQ bottleneck, and dynamics decoder modules for training and evaluation."
  • General motion retargeting: A method for mapping human motion to different robot kinematics while preserving key behaviors. "through General motion retargeting [21]."
  • Hip Decoupling Matrix: A structured linear transform used to decouple and align hip joint coordinates across robots with different hip-axis orientations. "(ii) Hip Decoupling Matrix."
  • Isaac Lab: An NVIDIA robotics learning framework/simulator used to train RL policies at scale. "All policies are trained with PPO in Isaac Lab."
  • Jacobian: The matrix of partial derivatives relating joint and task-space quantities; used here to correct closed-chain mappings. "we incorporate a closed-chain Jacobian correction JT ∈ RTXT into the mapping,"
  • Kinematic alignment: Mapping observations and actions between robots to ensure consistent joint semantics before learning dynamics-specific corrections. "Any2Any first performs kinematic alignment between source and target humanoids, aligning their input and output spaces so that the pretrained source policy can be meaningfully reused on the target embodiment."
  • LoRA: A low-rank adaptation method that represents weight updates with low-rank matrices for efficient fine-tuning. "LoRA represents weight updates with low-rank factors [26]."
  • Manipulator-equation: The standard form of rigid-body robot dynamics relating joint accelerations, velocities, gravity, and applied torques. "admits the standard manipulator-equation form"
  • Markov Decision Process (MDP): A formal framework for decision-making with states, actions, transitions, and rewards used in RL. "We formulate humanoid whole-body tracking (WBT) as a Markov Decision Process (MDP) M = (X, A, P,r, y),"
  • Mixture-of-experts: An architecture that combines multiple expert subnetworks with learned gating for improved capacity and specialization. "mixture-of-experts designs [9, 10]."
  • Morphology randomization: Training over varied robot morphologies to promote cross-robot generalization. "morphology randomization [13], topology-aware archi- tectures [35],"
  • MuJoCo: A physics engine commonly used for simulation and evaluation of robot control tasks. "During deployment in MuJoCo, we report the success rate of motion tracking,"
  • Parameter-efficient fine-tuning (PEFT): Techniques that adapt large pretrained models by training small added modules while freezing most weights. "applying lightweight parameter-efficient fine-tuning (PEFT) components"
  • Parallel Joint Coupling: An alignment component that handles closed-chain/parallel mechanisms by correcting joint relationships. "(iii) Parallel Joint Coupling."
  • Privileged observations: Additional training-only state information (e.g., contact forces) used to stabilize learning but not available at test time. "privileged observations (e.g., base velocity, contact forces, body mass distribution) that are available only during training."
  • Proprioception: Robot-internal sensory feedback (e.g., joint positions/velocities) providing its current physical state. "* Proprioception: the robot-centric state, including base angular velocity, projected gravity, joint positions, joint velocities, and the previous action."
  • Proximal Policy Optimization (PPO): A stable policy-gradient RL algorithm widely used for training continuous-control policies. "We adopt an actor-critic framework trained with Proximal Policy Optimization (PPO) for motion imitation."
  • Reference Motion Encoder: The policy component that encodes target/reference motion trajectories into features for control. "two-stream structure consisting of a Reference Motion Encoder and an Action Decoder"
  • Rigid-body residual: The difference in required generalized forces between two embodiments due to dynamic parameter mismatches. "rigid- body residual"
  • Sparse Scattering Matrix: A binary mapping matrix that embeds target joint values into the source joint layout to align kinematics. "(i) Sparse Scattering Matrix."
  • Teacher-student training: A training paradigm where a student model/policy learns to imitate or distill a teacher’s behavior. "teacher-student training,"
  • Whole-body tracking (WBT): A control paradigm where a humanoid reproduces full-body reference motions while maintaining balance and coordination. "Whole-body tracking (WBT) models have become a key foundation for hu- manoid robots, enabling them to imitate diverse motions with high fidelity."
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 3 tweets with 85 likes about this paper.

Sign Up for Free