Barkour: Benchmarking Animal-level Agility with Quadruped Robots (2305.14654v1)

Abstract: Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biological counterparts and show various agile skills to navigate complex environments quickly. Despite the interest, the field lacks systematic benchmarks to measure the performance of control policies and hardware in agility. We introduce the Barkour benchmark, an obstacle course to quantify agility for legged robots. Inspired by dog agility competitions, it consists of diverse obstacles and a time based scoring mechanism. This encourages researchers to develop controllers that not only move fast, but do so in a controllable and versatile way. To set strong baselines, we present two methods for tackling the benchmark. In the first approach, we train specialist locomotion skills using on-policy reinforcement learning methods and combine them with a high-level navigation controller. In the second approach, we distill the specialist skills into a Transformer-based generalist locomotion policy, named Locomotion-Transformer, that can handle various terrains and adjust the robot's gait based on the perceived environment and robot states. Using a custom-built quadruped robot, we demonstrate that our method can complete the course at half the speed of a dog. We hope that our work represents a step towards creating controllers that enable robots to reach animal-level agility.

• The paper presents the Barkour benchmark to quantify quadruped robot agility with an obstacle course mimicking dog agility competitions.
• It evaluates two policy learning approaches: specialist reinforcement learning policies for individual obstacles and a unified Transformer-based model.
• Experimental results show robots achieving an agility score up to 0.91 and completing the course at half the speed of small dogs, highlighting progress and challenges.

Insights into "Barkour: Benchmarking Animal-level Agility with Quadruped Robots"

The paper "Barkour: Benchmarking Animal-level Agility with Quadruped Robots" presents a systematic framework for evaluating and enhancing the agility of quadruped robots. Agility, characterized by swift, versatile movement capabilities to navigate complex environments, is increasingly desirable for legged robots, yet lacks a consolidated metric for assessment. This work introduces the "Barkour" benchmark as a comprehensive solution aimed at quantifying agility in legged robots through a course inspired by dog agility competitions.

The authors define Barkour to include varied obstacles that demand diverse locomotive skills, including a pause table, weave poles, an A-frame, and a broad jump, all within a confined \SI{5}{\metre} $\times$ \SI{5}{\metre} area. This environment facilitates the assessment of both control policies and robot hardware in terms of maneuverability and adaptability. The agility score, central to the Barkour benchmark, is an objective metric derived from traditional dog agility competitions, measuring performance based on completion time and penalties.

For benchmarking purposes, the paper explores two policy learning approaches: specialist and generalist policies. The specialist strategy involves training specific policies to handle individual obstacles via reinforcement learning. These include policies for omnidirectional walking, slope climbing, and jumping, tailored to align with the particular demands of the respective obstacles. The haLLMark of this approach is a hierarchical controller which deftly switches between these policies based on the robot’s position relative to the obstacles.

In contrast, the generalist approach employs a Transformer-based model, termed Locomotion-Transformer, which can tackle all obstacles using a unified policy. This strategy involves distilling specialist policies into a dataset, subsequently used to train a versatile Transformer sequence model. The Locomotion-Transformer exhibits the ability to handle varied terrain and make adjustments based on the environment and internal states, thus negating the need for discrete policy switching.

In terms of numerical results, the authors demonstrate that their methods allow a quadruped robot to complete the Barkour course at half the average speed of small dogs, reaching an agility score of 0.91 in certain instances, whilst dogs achieve a full score. This suggests promising advancement, albeit indicating room for further enhancements to achieve true animal-level agility. The specialists demonstrate specific proficiency; for example, the walking policy succeeds consistently in complex environments, yet the jumping policy, a notably challenging task due to the combination of power and precision required, achieves success only 38% of the time.

The implications of this research are notable in both theoretical methodology and practical application. For robotics research, the Barkour benchmark offers a standardized metric to compare future controllers and hardware, spurring further innovations toward achieving biomimetic agility in robots. Practically, success in such benchmarks could translate to improved efficacy in service robots navigating dynamic, unpredictable terrains.

In future directions, the paper highlights the potential removal of reliance on external positional data by adopting on-board sensors exclusively, enhancing real-world application potential. Additionally, examining the impacts of hardware innovations alongside algorithmic improvements presents an exciting avenue for closing the agility gap between robots and their animal counterparts.

In conclusion, this work establishes a clear, measurable path toward enhancing quadruped robot agility, offering standardized benchmarks essential for technological evolution in legged robotics. Through iterative enhancements grounded on such benchmarks, the robotics community stands well-positioned to make strides toward robots that exhibit movement capabilities analogous to living creatures, thus broadening the scope and utility of legged robots across various domains.