- The paper demonstrates an RL-based active embodiment identification approach that autonomously infers key robot parameters, reducing simulation-to-reality gaps.
- It employs a history-augmented URMA with GRU and dual-headed networks to predict both joint-level and global parameters with considerable precision across varied robot morphologies.
- The framework achieved notable accuracy in predicting rotor inertia, joint range, and total mass, paving the way for adaptive embodiment-aware control.
Active Embodiment Identification with Reinforcement Learning for Legged Robots
The paper addresses the critical issue of embodiment uncertainty in the deployment of legged robots, particularly when transitioning from simulation-trained policies to real-world applications. Traditional RL-based controllers are typically trained with embodiment-agnostic domain randomization, resulting in conservative behaviors that underutilize robot capabilities. Embodiment-aware policies, though capable of leveraging individual robot parameters, often depend on prior knowledge of the precise embodiment, which is difficult to obtain due to manufacturing variability, wear, or deliberate morphology modification.
This work proposes a framework for active embodiment identification wherein the robot autonomously infers its own embodiment parameters through environmental interaction, moving beyond standard system identification that relies on batch optimization and passive estimation. The approach aims to close the gap between simulated and real-world embodiment knowledge, enabling more aggressive and specialized policy deployment.
Architecture and Learning Approach
A history-augmented variant of the Unified Robot Morphology Architecture (URMA) is employed to generalize across differing robot morphologies. The framework consists of three neural networks:
- Embodiment Identification Network: Utilizes an URMA joint encoder to process variable-size joint observation sets. A GRU core enables temporal information integration, while dual heads predict joint-level parameters (rotor inertia, range, nominal position, etc.) and global parameters (total mass, dimensions, trunk properties).
- Policy and Critic Networks: Implemented as URMA modules aggregating joint-wise information into fixed-size latents, allowing policy parameterization on nominal embodiment descriptors.
Training is performed in end-to-end simulation using the MJX physics engine and PPO for RL, with extreme randomization of robot embodiments in parallel environments. The identification reward is derived from exponentially normalized MSE between the network’s predictions and ground truth embodiment parameters, explicitly encouraging information-seeking actions that improve parameter estimation.
Empirical Evaluation and Results
Experiments span four robot platforms (Unitree Go2, ANYmal C, Unitree G1, Booster T1), covering both quadrupedal and humanoid morphologies. The identification network is tasked with predicting a range of URMA-based parameters, encompassing both joint-wise specifics and global robot properties.
Key quantitative results:
- Rotor Inertia: Achieved average error ~0.005 kg·m² across robots. Although not extremely precise relative to the randomization span (0.0–0.015 kg·m²), this level facilitates embodiment-aware control tuning.
- Joint Range and Nominal Position: Error averages ~0.2 rad and ~0.039 rad, respectively, across ranges of ±3.14 rad, indicating robust parameter resolution.
- Total Mass: Predicted with precision around 0.25 kg, exceptionally accurate considering maximum robot mass up to 100 kg (ANYmal C).
- Robot Dimensions and Trunk COM: Errors in millimeter regime, promising for morphology-aware control.
The learned policy stabilizes the robot in a standing pose with moderate joint actuation. This prevents early termination but inherently limits identification of hard-to-excite parameters, such as max torque and velocity, which exhibit higher prediction errors.
Implications and Future Directions
The active embodiment identification paradigm enables robots to autonomously diagnose their operational parameters post-manufacturing or after morphological changes, facilitating fine-tuning of embodiment-aware policies. This reduces risk from embodiment uncertainty and supports lifelong learning, adaptive maintenance, and robust sim-to-real transfer.
The approach integrates well with closed-loop control schemes, where inferred embodiment could be continuously updated and fed back for policy adaptation. This is especially relevant for robots exposed to dynamic physical environments or evolving operational demands.
Future research may extend the methodology to real hardware with online adaptation, allowing progressive refinement of embodiment estimates. Additional emphasis on provoking extreme dynamic responses, potentially by optimizing policy excitation strategies, could enhance accuracy for hard-to-identify parameters.
Conclusion
The paper establishes an RL-driven, active embodiment identification framework for legged robots, demonstrating accurate inference of both joint and global morphological parameters in simulation. The method circumvents limitations of prior passive identification and paves the way for adaptable embodiment-aware policy optimization. Continued progress in this direction will support more effective and autonomous robotic systems across varied operational contexts.
