Fast and Efficient Locomotion via Learned Gait Transitions (2104.04644v3)

Abstract: We focus on the problem of developing energy efficient controllers for quadrupedal robots. Animals can actively switch gaits at different speeds to lower their energy consumption. In this paper, we devise a hierarchical learning framework, in which distinctive locomotion gaits and natural gait transitions emerge automatically with a simple reward of energy minimization. We use evolutionary strategies (ES) to train a high-level gait policy that specifies gait patterns of each foot, while the low-level convex MPC controller optimizes the motor commands so that the robot can walk at a desired velocity using that gait pattern. We test our learning framework on a quadruped robot and demonstrate automatic gait transitions, from walking to trotting and to fly-trotting, as the robot increases its speed. We show that the learned hierarchical controller consumes much less energy across a wide range of locomotion speed than baseline controllers.

Fast and Efficient Locomotion via Learned Gait Transitions

The paper "Fast and Efficient Locomotion via Learned Gait Transitions" presents a novel hierarchical framework to address the longstanding challenge of energy-efficient locomotion in quadrupedal robots. The key focus is on enabling these robots to automatically learn and transition between gaits optimally as their speed changes, paralleling the movement strategies observed in quadrupedal animals. This research has implications for the development of adaptive robotic systems that can efficiently navigate diverse environments.

Methodology

The authors introduce a framework that synergizes reinforcement learning (RL) with model-predictive control (MPC) to achieve this adaptive gait transition. The approach exploits the advantages of RL in learning versatile policies and MPC’s robustness in handling real-world constraints. The hierarchical structure of the framework includes:

1. High-Level Gait Generator: Utilizes a gait policy based on reinforcement learning to specify key parameters such as stepping frequency, swing ratio, and phase offsets, which control the gait cycles.
2. Low-Level Convex MPC Controller: Computes optimal motor commands based on gait parameters and contact schedules to achieve desired velocities efficiently.

Evolutionary strategies (ES), a form of zeroth-order optimization, were employed to train this high-level gait policy, forming the core of this framework. The learning process was driven by a simple reward function prioritizing energy minimization and velocity tracking.

Experimental Results

The framework's efficacy was validated through comprehensive simulations and real-world tests using the Unitree A1 robot. Results demonstrate:

• Energetic Efficiency: The learned controller consistently showed lower energy consumption compared to hand-tuned baseline controllers across different speeds, particularly measurable via the Cost of Transport (CoT) metric.
• Natural Gait Transitions: As the robot accelerated, automatic transitions were observed from walking to trotting and eventually to fly-trotting, mirroring natural animal behavior.
• Zero-Shot Sim-to-Real Transfer: Notably, the learned gait policy demonstrated robust performance in varied environments such as carpets and grass, without requiring fine-tuning or additional data collection post-simulation.

Implications

This research contributes significantly to the domain of robotic locomotion by addressing the challenge of energy efficiency through learned adaptive gait transitions. The hierarchical framework not only simplifies the complexity of optimizing discrete and continuous control variables but also facilitates smoother deployment in real-world scenarios.

The paper suggests potential extensions of the hierarchical control model to modulate other aspects of low-level control such as foot placement and body pose for more nuanced locomotion skills. The approach could pave the way for more agile and versatile robotics platforms capable of dynamic interactions in diverse terrains and environments.

Furthermore, the successful use of evolutionary strategies highlights their applicability in systems with complex dynamics where traditional RL methods may struggle due to non-Markovian environments. Future research might explore integrating additional sensory inputs or expanding the scope to multi-legged mobile systems beyond quadrupeds.

Overall, the insights gained from this paper contribute valuable findings towards advancing autonomous robotic systems capable of natural and efficient movement behavior, aligning closely with bio-mimetic approaches in robotics.