Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning

Abstract: In this article, we show that learned policies can be applied to solve legged locomotion control tasks with extensive flight phases, such as those encountered in space exploration. Using an off-the-shelf deep reinforcement learning algorithm, we trained a neural network to control a jumping quadruped robot while solely using its limbs for attitude control. We present tasks of increasing complexity leading to a combination of three-dimensional (re-)orientation and landing locomotion behaviors of a quadruped robot traversing simulated low-gravity celestial bodies. We show that our approach easily generalizes across these tasks and successfully trains policies for each case. Using sim-to-real transfer, we deploy trained policies in the real world on the SpaceBok robot placed on an experimental testbed designed for two-dimensional micro-gravity experiments. The experimental results demonstrate that repetitive, controlled jumping and landing with natural agility is possible.

• The paper develops a model-free deep reinforcement learning framework that trains quadrupedal robots to perform agile cat-like maneuvers in low-gravity settings.
• It achieves 90-degree reorientation in under 2.5 seconds in both simulations and real-world tests, outperforming traditional controller methods.
• The study demonstrates promising sim-to-real transfer for SpaceBok robots, paving the way for enhanced autonomous space exploration.

Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning

This essay reviews a study presented by Rudin et al. on the development of legged robotic systems capable of agile locomotion in low-gravity environments. The paper details the utilization of deep reinforcement learning (DRL) for training policies that enable quadrupedal robots, specifically the SpaceBok platform, to execute controlled jumping and landing maneuvers akin to natural feline movements, adapted to space exploration contexts.

Overview

The research showcases an approach leveraging model-free DRL to train a neural network policy controlling a quadruped robot. The training process involves performing tasks of varying complexity, which culminate in three-dimensional reorientation and landing behaviors on simulated low-gravity celestial bodies. A significant aspect is the sim-to-real transfer, where policies developed in simulation are tested on real hardware via the SpaceBok robot in a microgravity testbed. Experimental results confirm such systems can achieve repetitive controlled jumping and landing with inherent agility.

Numerical Results and Claims

The study highlights several key numerical results, notably the agility of the robot in achieving orientation changes of 90 degrees in less than 2.5 seconds in simulation and real-world tests alike. Furthermore, the trained policies not only outperform manually designed controllers but also manage complex tasks in inherently nonlinear dynamic environments.

Theoretical and Practical Implications

Theoretical implications of this research indicate a potential shift in the control paradigms for space-bound legged robotic systems from traditional model-based approaches to learning-based schemes. Practically, the study bolsters the case for deploying legged robots in space exploration missions where low-gravity jumping and landing could enable faster traversal and access to otherwise inaccessible terrains.

Prospective Developments in AI

Future prospects for AI in robotics hinge upon extending models such as those demonstrated here to address more complex interactions with environmental elements, such as uneven terrain or unexpected external forces. This could involve refined simulations or incorporating sensor fusion techniques to anticipate real-world challenges more accurately. Moreover, developing mechanisms to verify and ensure reliability and safety in learned policies remains crucial, especially for safety-critical applications like space exploration.

Conclusion

This paper contributes substantially to autonomous robotic navigation in low-gravity contexts. While challenges remain, particularly concerning the transferability of complex 3D maneuvers and robust state estimation, the results pave the way for future research trajectories that integrate AI with exploratory robotic missions. This development promises not only to enhance agility in legged robots but also to expand their operational envelope beyond Earth, reflecting their growing significance in planetary exploration efforts.