SHIELD: Safety on Humanoids via CBFs In Expectation on Learned Dynamics

Abstract: Robot learning has produced remarkably effective ``black-box'' controllers for complex tasks such as dynamic locomotion on humanoids. Yet ensuring dynamic safety, i.e., constraint satisfaction, remains challenging for such policies. Reinforcement learning (RL) embeds constraints heuristically through reward engineering, and adding or modifying constraints requires retraining. Model-based approaches, like control barrier functions (CBFs), enable runtime constraint specification with formal guarantees but require accurate dynamics models. This paper presents SHIELD, a layered safety framework that bridges this gap by: (1) training a generative, stochastic dynamics residual model using real-world data from hardware rollouts of the nominal controller, capturing system behavior and uncertainties; and (2) adding a safety layer on top of the nominal (learned locomotion) controller that leverages this model via a stochastic discrete-time CBF formulation enforcing safety constraints in probability. The result is a minimally-invasive safety layer that can be added to the existing autonomy stack to give probabilistic guarantees of safety that balance risk and performance. In hardware experiments on an Unitree G1 humanoid, SHIELD enables safe navigation (obstacle avoidance) through varied indoor and outdoor environments using a nominal (unknown) RL controller and onboard perception.

Analyzing SHIELD: Integrating Safety with Learned Dynamics in Humanoids

This paper introduces SHIELD, a robust framework aimed at enhancing safety in humanoid robots by capitalizing on learned dynamics and control barrier functions (CBFs). SHIELD stands for Safety on Humanoids via CBFs In Expectation on Learned Dynamics, and it effectively bridges the gap between model-based control methods and reinforcement learning (RL). The essence of SHIELD lies in its ability to integrate a safety layer atop nominal controllers, thus ensuring constraint satisfaction during operation.

Core Innovations and Methodology

The framework is bifurcated into two main components:
1. Generative Stochastic Dynamics Residual Model: SHIELD initially trains a model to predict the dynamics residuals using real-world data. It captures uncertainties and deviations from expected behavior, which is crucial for systems where precise modeling is challenging.

1. Stochastic Discrete-Time Control Barrier Functions (S-DTCBFs): The second component involves a minimally-invasive safety layer that leverages the learned model to enforce safety constraints probabilistically during runtime. This approach allows for dynamic constraint specifications without retraining the RL controller.

Key Experimental Results

Integrated with the Unitree G1 humanoid, SHIELD's capabilities were evaluated through obstacle avoidance tasks in diverse environments. The experiments underscored the framework's ability to provide probabilistic safety guarantees while achieving efficient navigation. Although actual quantitative results weren't delineated in the prompt, SHIELD demonstrated its adeptness at ensuring safety via empirically-backed risk-aware controls. These observations emphasize SHIELD’s potential for reliable deployment in real-world settings.

Implications for Future AI and Robotics Research

The introduction of SHIELD provides several significant insights for future research:
- Probabilistic Safety: The application of S-DTCBFs allows for a nuanced approach to safety, emphasizing probabilistic guarantees rather than absolute assurances. This shift could lead to innovations in how safety is conceptualized and implemented in AI-driven systems.
- Integration of Learning Models: The use of generative models to learn dynamics residuals exemplifies a method for combining learning with traditional model-based control strategies. This highlights a broader trend where hybrid strategies are increasingly necessary to navigate complex robotic environments.
- Adaptability in Uncertain Conditions: The framework's capability to adjust to dynamic constraints in real time without retraining indicates a shift toward more adaptable learning systems. This adaptability could be crucial for autonomous systems operating in unpredictable scenarios.

Theoretically, SHIELD might encourage the development of more nuanced control algorithms that can incorporate learnt model inaccuracies into their decision-making processes. Practically, the adoption of such frameworks could make the deployment of humanoid robots in human environments more plausible by ensuring safety without compromising on performance.

Conclusion

SHIELD redefines the landscape for humanoid robot safety by providing a flexible yet formally verified framework capable of integrating with complex learning-based controllers. Its balanced fusion of stochastic learning models with control barrier functions posits a promising direction for advancing humanoid robotics, bridging theoretical control guarantees with practical adaptability. Future research could extend this work to broader classes of robotic systems and explore additional applications of such hybrid safety frameworks in AI systems.