Multi-expert learning of adaptive legged locomotion (2012.05810v1)

Abstract: Achieving versatile robot locomotion requires motor skills which can adapt to previously unseen situations. We propose a Multi-Expert Learning Architecture (MELA) that learns to generate adaptive skills from a group of representative expert skills. During training, MELA is first initialised by a distinct set of pre-trained experts, each in a separate deep neural network (DNN). Then by learning the combination of these DNNs using a Gating Neural Network (GNN), MELA can acquire more specialised experts and transitional skills across various locomotion modes. During runtime, MELA constantly blends multiple DNNs and dynamically synthesises a new DNN to produce adaptive behaviours in response to changing situations. This approach leverages the advantages of trained expert skills and the fast online synthesis of adaptive policies to generate responsive motor skills during the changing tasks. Using a unified MELA framework, we demonstrated successful multi-skill locomotion on a real quadruped robot that performed coherent trotting, steering, and fall recovery autonomously, and showed the merit of multi-expert learning generating behaviours which can adapt to unseen scenarios.

• The paper presents a multi-expert neural network architecture (MELA) that combines specialized locomotion skills via a gating network.
• The methodology involves pre-training individual experts and co-training them for real-time sensor-based adaptations.
• Experimental results on the quadruped 'Jueying' show rapid, under-one-second corrective actions and improved adaptability.

Review of "Multi-expert learning of adaptive legged locomotion"

The paper presented in "Multi-expert learning of adaptive legged locomotion" by Chuanyu Yang et al. advocates a sophisticated approach for legged robot locomotion through a Multi-Expert Learning Architecture (MELA). This framework aims to mimic biological systems' adaptability by combining multiple expert neural networks (each representing distinct motor skills) into a comprehensive and dynamically synthesized neural network during robot operation.

Legged robot locomotion poses critical challenges due to variable and unpredictable terrains that demand rapid policy adaptations. Traditional mathematical optimization approaches, such as MPC and QP, often fall short in dynamic and contact-rich environments because they struggle with the curse of dimensionality and demand manual design of contact sequences. Herein, MELA serves as a robust alternative by facilitating seamless transitions between multiple motor skills in real-time via a Gating Neural Network (GNN).

MELA Structure and Training Paradigm

MELA is initialized with a diverse set of pre-trained expert networks, each fine-tuned for specific locomotion tasks such as trotting or fall recovery. The incorporation of the GNN allows these expert networks to be blended dynamically based on the real-time feedback received from the robot's sensors. During training, the experts are co-trained to optimize task-specific functions, whilst simultaneously training the gating network to learn how to aggregate these expert networks into a singular, adaptive policy framework.

Two critical steps guide this training process: (1) Pre-training individual skills (experts) to master specific locomotion modes, and (2) Co-training of MELA, integrating all experts to achieve a wide repertoire of adaptive skills for unseen scenarios. This framework was not only evaluated through simulations but also validated on a physical quadruped robot ('Jueying'), showing compelling results in scenarios involving turning, steering, and recovery from falls.

Performance and Results

Quantitative evaluation reveals that MELA achieves substantial improvements in adaptability and robustness over prior model-free methods. Experiments demonstrate the system's capability in providing agile adaptation and sustaining multiple locomotion modes efficiently. In particular, during target-following tasks, the framework exhibited adept transitions across diverse locomotion tasks. The robust performance, validated even on challenging and variable terrains, supports the argument for multi-expert architectures over existing single-policy frameworks.

Moreover, experiments demonstrated that the MELA-driven robot was able to perform rapid corrective actions in under a second, underscoring its potential superiority for real-time application—a critical requirement for autonomous robotic operations in unpredictable environments.

Future Implications and Considerations

The implication of such a structured framework transcends basic locomotive capabilities. By continuously blending expert policies based on evolving state feedback, it offers a promising path towards developing advanced robotic systems with generalizable skills, potentially reducing the human effort involved in manual programming of individual task-specific policies. However, the paper acknowledges the limitations in perceptual dimensions, suggesting future integration of visual and haptic feedback to improve environment-awareness and further enhance adaptability.

Overall, the presented architecture marks a significant contribution to legged robotics by leveraging computational intelligence to forge policies that effortlessly adapt and optimize their actions—a pursuit that continues to challenge both the machine learning and robotics communities. Future research may explore enhancing sensory inputs and deploying such frameworks across broader application domains, expanding the horizon of autonomous robotic capabilities.