Learning to walk in confined spaces using 3D representation (2403.00187v1)

Abstract: Legged robots have the potential to traverse complex terrain and access confined spaces beyond the reach of traditional platforms thanks to their ability to carefully select footholds and flexibly adapt their body posture while walking. However, robust deployment in real-world applications is still an open challenge. In this paper, we present a method for legged locomotion control using reinforcement learning and 3D volumetric representations to enable robust and versatile locomotion in confined and unstructured environments. By employing a two-layer hierarchical policy structure, we exploit the capabilities of a highly robust low-level policy to follow 6D commands and a high-level policy to enable three-dimensional spatial awareness for navigating under overhanging obstacles. Our study includes the development of a procedural terrain generator to create diverse training environments. We present a series of experimental evaluations in both simulation and real-world settings, demonstrating the effectiveness of our approach in controlling a quadruped robot in confined, rough terrain. By achieving this, our work extends the applicability of legged robots to a broader range of scenarios.

• The paper presents a novel hierarchical policy framework that uses reinforcement learning and 3D volumetric representations to improve robot navigation in confined spaces.
• It implements a two-layer structure where a low-level policy manages robust locomotion and a high-level policy leverages spherical scans for effective spatial decision-making.
• Experimental results in both simulation and real-world tests demonstrate enhanced navigation performance over complex terrains compared to baseline methods.

Learning to Walk in Confined Spaces Using 3D Representation: An Overview

The paper "Learning to Walk in Confined Spaces Using 3D Representation" addresses a significant challenge in the field of robotics: enabling legged robots to navigate confined and unstructured environments effectively. This research leverages reinforcement learning and 3D volumetric representations within a two-layer hierarchical policy framework to enhance the locomotion capabilities of legged robots, particularly in environments with overhanging obstacles.

Methodological Approach

The authors develop a hierarchical policy framework to control a quadruped robot. The framework is composed of a low-level policy focused on robust locomotion across varied terrains, and a high-level policy aimed at enabling spatial awareness and navigational capabilities in complex environments.

1. Low-Level Policy:
   • Trained using the Proximal Policy Optimization (PPO) algorithm, this policy emphasizes following 6D commands (combining lateral and angular velocity with body orientation and height) to achieve smooth traversal over uneven surfaces.
   • It utilizes proprioceptive and exteroceptive inputs, including height samples around each foot, to navigate effectively.
2. High-Level Policy:
   • This policy also utilizes PPO for training and employs spherical scans to capture local geometry for effective decision-making in confined spaces.
   • Commands generated by the high-level policy direct the low-level policy, balancing spatial navigation with robust traversal.
3. Hierarchical Structure:
   • The low-level teacher policy is distilled into a student policy that can manage noisy observations.
   • Similarly, the high-level teacher policy, initially trained with spherical scans, is distilled into a student policy that interprets noisy voxel data, enabling flexibility in sensor configurations.

Experimental Evaluation

The methodology was validated both in simulation and through real-world deployments.

• Simulation: A procedural terrain generator was implemented using the Wave Function Collapse method. This allowed the creation of diverse terrain configurations, testing the policy's abilities in different confined space scenarios. Results showed high success rates in navigating complex obstacle configurations compared to baseline strategies.
• Real-World Tests: Deployments included environments resembling a collapsed building, with complex terrains comprising loose gravel and unstable structures. The robot adapted its posture dynamically, showcasing the policy's robustness and adaptability.

Implications and Future Directions

The research demonstrates significant advancements in robotic locomotion within confined and challenging environments. By enabling a legged robot to autonomously navigate and adjust its posture based on environmental cues, the paper successfully extends the operational range of such robots to scenarios where traditional platforms may fail.

Future developments could focus on enhancing cognitive capabilities for more dynamic environments and integrating advanced perception techniques for even more nuanced spatial understanding. Exploration of the integration of these systems into larger, multi-robot frameworks could pave the way for fully autonomous exploratory missions in extreme environments, including disaster sites and extraterrestrial landscapes.

In summary, this research presents a comprehensive approach to improving legged robot mobility in unstructured settings and represents a significant methodological contribution to the field of robotics.