Robust Reinforcement Learning-Based Locomotion for Resource-Constrained Quadrupeds with Exteroceptive Sensing (2505.12537v1)

Abstract: Compact quadrupedal robots are proving increasingly suitable for deployment in real-world scenarios. Their smaller size fosters easy integration into human environments. Nevertheless, real-time locomotion on uneven terrains remains challenging, particularly due to the high computational demands of terrain perception. This paper presents a robust reinforcement learning-based exteroceptive locomotion controller for resource-constrained small-scale quadrupeds in challenging terrains, which exploits real-time elevation mapping, supported by a careful depth sensor selection. We concurrently train both a policy and a state estimator, which together provide an odometry source for elevation mapping, optionally fused with visual-inertial odometry (VIO). We demonstrate the importance of positioning an additional time-of-flight sensor for maintaining robustness even without VIO, thus having the potential to free up computational resources. We experimentally demonstrate that the proposed controller can flawlessly traverse steps up to 17.5 cm in height and achieve an 80% success rate on 22.5 cm steps, both with and without VIO. The proposed controller also achieves accurate forward and yaw velocity tracking of up to 1.0 m/s and 1.5 rad/s respectively. We open-source our training code at github.com/ETH-PBL/elmap-rl-controller.

• The paper presents a robust reinforcement learning-based controller for resource-constrained quadruped robots, enabling real-time locomotion over challenging terrains using exteroceptive sensing and elevation mapping.
• The approach simplifies reinforcement learning architectures by concurrently training the control policy and state estimator, using a minimal sensor setup (stereo depth and ToF cameras) to reduce dependency on computationally expensive VIO for accurate mapping.
• Experimental results demonstrate the system's capability to navigate steps up to 17.5 cm without error and achieve an 80% success rate on 22.5 cm steps, with accurate velocity tracking up to 1.0 m/s, showing strong performance in real-world scenarios.

Robust Reinforcement Learning-Based Locomotion for Resource-Constrained Quadrupeds

The paper "Robust Reinforcement Learning-Based Locomotion for Resource-Constrained Quadrupeds with Exteroceptive Sensing" presents a novel approach to enhance quadrupedal robot locomotion on uneven terrain using reinforcement learning (RL) techniques. The research focuses on addressing the computational limitations of small-sized quadrupedal platforms when deployed in real-world environments, a challenge exacerbated by the complex nature of terrain perception.

Core Contributions

The authors introduce a robust exteroceptive RL-based controller capable of real-time locomotion for resource-constrained quadrupedal robots. This controller utilizes elevation mapping derived from depth sensor data to navigate challenging terrains. The paper highlights several contributions:

1. Concurrent Training of Policy and State Estimator: The approach simplifies RL architectures by concurrently training the control policy and the state estimator. This integration improves odometry estimation, crucial for terrain mapping.
2. Minimal Sensor Setup for Elevation Mapping: By employing a stereo depth camera and a time-of-flight (ToF) camera, the system reduces dependency on VIO—a traditional computationally expensive method for robust odometry. The sensor setup effectively maintains mapping accuracy during VIO failures, preserving computational resources.
3. Multi-Faceted Evaluation: The paper includes thorough benchmarks of the proposed system across various experimental conditions, demonstrating its efficacy and versatility.

Experimental Outcomes

The authors provide experimental evidence showcasing the robustness and versatility of the proposed controller. The system efficiently navigates steps up to 17.5 cm in height without error and manages an impressive 80% success rate on 22.5 cm steps. It achieves accurate forward and yaw velocity tracking, allowing for velocities up to 1.0 m/s and 1.5 rad/s respectively. These results are significant, illustrating a strong capability in real-world scenarios with both VIO and fallback odometry.

Moreover, the system's elevation mapping quality was evaluated using the Chamfer Distance metric, revealing substantial improvements when incorporating the ToF camera—demonstrating a reduction in mapping errors even under suboptimal odometry conditions.

Implications and Future Work

The research delineates practical and theoretical implications for advancing real-world applications of compact quadrupeds. The controller's capacity to handle rough terrain while balancing performance and computational constraints opens avenues for integration within autonomous navigation systems and other complex robotic applications.

Looking ahead, the paper suggests examining performance under different sensor failure modalities, such as low-texture environments or invisible objects—challenges inherent to real-world deployment. Additionally, expanding the locomotion capabilities to include dynamic maneuvers, such as parkour, could enhance the versatility of quadrupedal robots.

Conclusion

In summary, this paper contributes valuable insights into the domain of quadrupedal locomotion control using reinforcement learning. By addressing computational limitations and enhancing terrain navigation capabilities, it paves the way for more efficient and reliable integration of quadrupedal robots in diverse environments. The proposed system stands as a robust and practical approach, promoting ongoing advancements in robotics and AI.