Multi-Stage Inertial Parameter Estimation

• Multi-stage inertial parameter estimation is a process that sequentially determines an object's mass, center of mass, and inertia by integrating vision-based sizing with semantic priors.
• The module refines parameters using a decoupled, hierarchical cross-entropy sampling strategy to ensure physical consistency even under complex payload conditions.
• Real-time updates from the estimation module dynamically adjust robot equilibrium and manipulation controllers, enabling robust loco-manipulation in wheeled humanoids.

A multi-stage object inertial parameter estimation module is a system that sequentially infers the mass, center of mass, and moments of inertia of an unknown object being manipulated by a robot, typically for the purpose of adapting whole-body teleoperation, dynamic compensation, or robust control. In recent research on wheeled humanoid loco-manipulation, this module is realized through a pipeline that first uses real-time visual perception for object sizing, then integrates semantic priors from a vision-LLM (VLM), and finally applies a decoupled hierarchical sampling strategy for physically consistent parameter refinement. The output—estimated inertial parameters—is employed to dynamically reconfigure the robot’s equilibrium point and manipulation controllers for robust tracking and compliant interaction, especially under complex payload conditions (Baek et al., 13 Aug 2025).

1. System Architecture and Functional Role

The multi-stage estimation module is embedded in a whole-body bilateral teleoperation framework for wheeled humanoids. The human operator controls locomotion and manipulation via a human–machine interface, while the robot autonomously adapts to varying object dynamics by estimating and compensating for the unknown inertial parameters of a manipulated payload. The module functions online and in parallel with both high-fidelity simulation and hardware controllers, enabling real-time updates during task execution. Estimated parameters are directly utilized to update the equilibrium point (for posture and balance) and the manipulation controller’s feedforward/model-based compensation, facilitating stable locomotion and accurate object handling even with payloads of significant mass (up to one-third the robot’s body weight).

2. Multi-Stage Sequential Estimation Pipeline

The process is divided into three distinct, interconnected stages:

Stage 1: Vision-Based Object Size Estimation

• The physical dimensions of the object are estimated from RGB-D sensor data using an object detection algorithm (e.g., CenterSnap), which computes the axis-aligned bounding box (AABB) of the reconstructed point cloud.
• The resultant size vector $s = [a, b, c]^T$ constrains the feasible region for subsequent center of mass and inertia estimation.

Stage 2: VLM-Based Inertial Prior Generation

• A large vision–LLM interprets both the visual appearance (indicating fill state, materials, etc.) and language-based task context to produce a strong prior $m_0$ for the object’s inertial properties.
• For instance, assuming a uniform-density cuboid, mass is set as $m_0 = \rho V$ with volume $V$ from vision and density $\rho$ from the VLM’s semantic inference.
• Priors for center of mass and inertia tensor are computed via analytical formulas (parallel axis theorem, standard cuboid inertia expressions), leveraging the estimated size and mass.

Stage 3: Decoupled Hierarchical Cross-Entropy Sampling-Based Refinement

• Rather than jointly searching the entire inertial parameter space, the estimator employs a hierarchical and decoupled strategy:
  • Mass and center of mass are refined first using cross-entropy method (CEM) sampling, initialized by the VLM prior.
  • The inertia tensor is subsequently deterministically computed from the size and location of the center of mass by applying physics-based formulas (e.g., $$I_x = \frac{1}{12} m (b^2 + c^2) + m \cdot (\text{offset terms})$$).
• A multi-hypothesis scheme generates several candidate parametric samples around the prior, enhancing robustness to VLM estimation errors.
• High-fidelity simulation is performed for each hypothesis; a cost function, often based on the $L_2$ norm between simulated joint trajectories and reference data, guides the iterative refinement.

3. Estimation Strategy: Hierarchical Decoupling and Sampling Robustness

• The estimation procedure is organized hierarchically:
  • The mass and center of mass are sampled and refined first; for each, candidate values are drawn from a distribution conditioned on vision and VLM priors.
  • For each mass–CoM hypothesis, the inertia tensor is calculated according to the known geometry, ensuring physical feasibility.
• The cross-entropy method updates the sampling distribution based on performance in simulation; elite samples (lowest error) inform the next iteration.
• Decoupling the estimation reduces susceptibility to poor initial guesses and prevents mode collapse, allowing exploration of the parameter space without simultaneous optimization of highly coupled variables.

4. Integration with Control Frameworks and Real-Time Operation

Once the object’s inertial parameters are estimated, several control modules are updated:

• Locomotion Equilibrium Shift: The gravitational force contributed by the payload is used to update the robot’s equilibrium point. A moment balance (e.g., $$F_g \cdot h_R \sin(\theta_R) = -F_\text{obj} \cdot x_{ee}$$) determines the pitch angle required for balanced motion.
• Manipulation Tracking and Compliance: Manipulator inverse dynamics are recalculated by updating $M(q)$ and $b(q,\dot{q})$ in the joint torque equation

  $$\tau = K_p (q_d - q) + K_d (\dot{q}_d - \dot{q}) + M(q)\ddot{q}_d + b(q, \dot{q}),$$

  where $M(q)$ now reflects the object’s combined dynamics.

• Safety via Control Barrier Functions: Accurate inertial estimates enable proper tuning of control barrier functions for collision avoidance and compliance under uncertainties.
• The module operates asynchronously to the robot’s main control loop, with real-time update rates of approximately 0.5–1 sec per parameter estimate, leveraging sim-to-real adaptation for consistency between simulation and hardware.

5. Empirical Validation and Quantitative Performance

• Extensive empirical validation was performed via hardware and simulation experiments, demonstrating robust task execution (lifting, delivering, releasing) with heavy payloads.
• Quantitative metrics, including normalized mean absolute error (NMAE) and MAE for mass, center of mass, and inertia tensor, showed clear improvements over baselines that do not employ vision or VLM priors.
• Integration of the multi-stage estimation led to increases in manipulation tracking accuracy and improved haptic force feedback, supporting more dynamic and responsive teleoperation.
• The decoupled, hierarchical, multi-hypothesis strategy consistently resulted in stability even when semantic priors were imperfect or perception was noisy.

6. Applications, Implications, and Limitations

• The described module enables wheeled humanoids to robustly manipulate a diverse set of payloads, enhancing teleoperation for industrial, logistic, and service robotics.
• Real-time inertial parameter updates permit the operator to focus on high-level planning, relying on autonomous dynamic compensation for safe and efficient manipulation.
• The strategy of combining vision-based size estimation, VLM semantic priors, and hierarchical sampling opens further research avenues in multi-modal perception and adaptive control.
• Limitations include reliance on the validity of physical assumptions (e.g., uniform-density cuboid), the accuracy of vision and VLM outputs, and computational constraints of real-time parallel simulation; future work may target non-cuboid or deformable objects and more complex material priors.

7. Summary

The multi-stage object inertial parameter estimation module integrates real-time vision, semantic reasoning via large vision–LLMs, and sampling-based physical consistency refinement to provide robust and adaptive estimation of mass, center of mass, and inertia for unknown payloads in whole-body teleoperation. This integrated estimation directly informs adaptive control and safety modules, significantly improving manipulation tracking, equilibrium maintenance, and dynamic compliance for wheeled humanoid robots engaged in loco-manipulation tasks with heavy and variable payloads (Baek et al., 13 Aug 2025).