Thor: Towards Human-Level Whole-Body Reactions for Intense Contact-Rich Environments (2510.26280v1)

Abstract: Humanoids hold great potential for service, industrial, and rescue applications, in which robots must sustain whole-body stability while performing intense, contact-rich interactions with the environment. However, enabling humanoids to generate human-like, adaptive responses under such conditions remains a major challenge. To address this, we propose Thor, a humanoid framework for human-level whole-body reactions in contact-rich environments. Based on the robot's force analysis, we design a force-adaptive torso-tilt (FAT2) reward function to encourage humanoids to exhibit human-like responses during force-interaction tasks. To mitigate the high-dimensional challenges of humanoid control, Thor introduces a reinforcement learning architecture that decouples the upper body, waist, and lower body. Each component shares global observations of the whole body and jointly updates its parameters. Finally, we deploy Thor on the Unitree G1, and it substantially outperforms baselines in force-interaction tasks. Specifically, the robot achieves a peak pulling force of 167.7 N (approximately 48% of the G1's body weight) when moving backward and 145.5 N when moving forward, representing improvements of 68.9% and 74.7%, respectively, compared with the best-performing baseline. Moreover, Thor is capable of pulling a loaded rack (130 N) and opening a fire door with one hand (60 N). These results highlight Thor's effectiveness in enhancing humanoid force-interaction capabilities.

• The paper introduces Thor, an RL framework that decouples control into upper, waist, and lower body modules for enhanced force interaction in contact-rich tasks.
• It employs a force-adaptive torso-tilt (FAT2) reward based on ZMP analysis to mimic human strategies, achieving significant gains in balance and force output.
• Experimental results show up to 74.7% improvement over baselines in both simulation and real-world tests, underscoring its potential for industrial and rescue applications.

Thor: Human-Level Whole-Body Reactions for Intense Contact-Rich Environments

Introduction

The paper introduces Thor, a reinforcement learning (RL) framework for humanoid robots designed to achieve human-level whole-body reactions in environments characterized by intense, contact-rich interactions. The motivation stems from the limitations of traditional model-based and monolithic RL approaches, which struggle with the high-dimensionality and instability inherent to humanoid robots, especially during tasks requiring significant forceful interaction with the environment. Thor addresses these challenges by decoupling the control architecture into upper body, waist, and lower body modules, each with independent actor-critic networks, and by introducing a force-adaptive torso-tilt (FAT2) reward function grounded in biomechanical analysis and the Zero Moment Point (ZMP) criterion.

Figure 1: Humanoids performing forceful tasks: (a) opening a fire door, (b) pulling a loaded rack, (c) pushing a wheelchair, (d) wiping a whiteboard.

Decoupled Policy Architecture

Thor's architecture is motivated by the biomechanical role of the human waist as a force transmission junction. The control policy is decomposed into three sub-policies: lower body ($\pi_l$), waist ($\pi_w$), and upper body ($\pi_u$). Each sub-policy is implemented as an independent actor-critic network, sharing global observations but maintaining separate parameters and reward functions. The outputs are concatenated to form the desired joint positions for the entire robot, which are then processed by a PD controller to generate joint torques.

Figure 2: Thor's pipeline: decoupled actor-critic networks for upper body, waist, and lower body, with privileged force information in the critic and FAT2 reward shaping.

This decoupling mitigates the curse of dimensionality, enables high-frequency inference on limited onboard compute, and allows for more effective coordination during forceful interactions. The architecture is trained using PPO, with each agent's advantage and loss computed independently, and a global torque regularization term to coordinate energy consumption.

Force-Adaptive Torso-Tilt Reward (FAT2) and ZMP Analysis

A key innovation is the FAT2 reward, which encourages the robot to adaptively tilt its torso in response to external forces, mimicking human strategies for maximizing force output while maintaining balance. The reward is derived from a force and torque equilibrium analysis based on the ZMP criterion under external force. The analysis reveals that, for a given external force at the end-effectors, the optimal torso tilt angle $\beta$ can be computed as:

$$\beta = \cos^{-1} \left( \frac{|\vec{F}_h| |\vec{r}_h| \cos\varphi \cos\alpha}{|\vec{F}_g| |\vec{r}_{\text{CoM}}|} \right) \leq \beta^{\max}$$

where $|\vec{F}_h|$ is the magnitude of the interaction force, $|\vec{r}_h|$ is the distance from the support to the end-effector, $\varphi$ and $\alpha$ are geometric angles, and $|\vec{F}_g|$ is the gravitational force. The FAT2 reward penalizes deviation from this optimal tilt, promoting human-like adaptation.

Figure 3: Force interaction analysis with ZMP constraint, illustrating the equilibrium of forces and torques during pulling.

Training Methodology

Thor is trained in simulation (Isaac Gym) using a two-stage curriculum: first, the robot learns stable postures under low-force disturbances; then, the environment is made more challenging with higher force disturbances. Domain randomization is applied to the direction and magnitude of external forces to improve sim-to-real transfer. During training, privileged information (true external forces) is available to the critic; during deployment, the policy infers force interaction implicitly from proprioceptive signals.

Experimental Results

Simulation

Simulation experiments demonstrate that as the required pulling force increases, the robot adaptively tilts its torso, even allowing the CoM to move outside the support polygon, which is not typical in prior approaches. This adaptation enables the robot to generate significantly higher interaction forces.

Figure 4: Sequential simulation plots showing posture and interactive force during backward and forward motion.

Real-World Evaluation

Thor was deployed on the Unitree G1 humanoid (29 DoF, 1.32 m, 35 kg). The policy runs at 50 Hz, with a PD controller at 500 Hz. Real-world experiments measured the relationship between torso tilt and pulling force, confirming that the robot, like a human, leans to generate greater force.

Figure 5: Variation of pulling force with respect to torso tilt angle.

Thor achieved a peak pulling force of $167.7 \pm 2.4$ N (48% of body weight) during backward locomotion and $145.5 \pm 2.0$ N during forward locomotion, representing 68.9% and 74.7% improvements over the best baseline (Falcon). In single-hand tasks, Thor was able to open a fire door (60 N) and pull a 70 kg rack (130 N), tasks where baselines failed due to insufficient force or loss of balance.

Ablation studies showed that FAT2 alone accounts for 80–90% of the performance gain, but the decoupled architecture is essential for stability and mitigating anomalous behaviors (e.g., waist roll deviations) under high force.

Implications and Future Directions

Thor demonstrates that biomechanically inspired reward shaping, combined with modular RL architectures, can substantially enhance the force-interaction capabilities of humanoid robots. The explicit modeling of force and torque equilibrium, and the use of privileged information during training, enable robust sim-to-real transfer and generalization to diverse tasks.

Practically, Thor's approach enables humanoids to perform a broader range of service, industrial, and rescue tasks that require both dexterity and significant force application. Theoretically, the work suggests that further gains may be realized by integrating human demonstration data, automating hyperparameter tuning, and extending the modular architecture to additional body segments or multi-contact scenarios.

Conclusion

Thor advances the state of the art in humanoid whole-body control for contact-rich environments by introducing a decoupled RL architecture and a force-adaptive torso-tilt reward grounded in ZMP analysis. The framework achieves substantial improvements in force-interaction tasks over prior baselines, both in simulation and on real hardware. Future work should focus on leveraging human demonstrations for faster convergence and reducing reliance on manual hyperparameter tuning, as well as exploring more complex multi-contact and dynamic environments.