Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots

Abstract: We introduce the Self-Adaptive Goal Generation - Robust Intelligent Adaptive Curiosity (SAGG-RIAC) architecture as an intrinsi- cally motivated goal exploration mechanism which allows active learning of inverse models in high-dimensional redundant robots. This allows a robot to efficiently and actively learn distributions of parameterized motor skills/policies that solve a corresponding distribution of parameterized tasks/goals. The architecture makes the robot sample actively novel parameterized tasks in the task space, based on a measure of competence progress, each of which triggers low-level goal-directed learning of the motor policy pa- rameters that allow to solve it. For both learning and generalization, the system leverages regression techniques which allow to infer the motor policy parameters corresponding to a given novel parameterized task, and based on the previously learnt correspondences between policy and task parameters. We present experiments with high-dimensional continuous sensorimotor spaces in three different robotic setups: 1) learning the inverse kinematics in a highly-redundant robotic arm, 2) learning omnidirectional locomotion with motor primitives in a quadruped robot, 3) an arm learning to control a fishing rod with a flexible wire. We show that 1) exploration in the task space can be a lot faster than exploration in the actuator space for learning inverse models in redundant robots; 2) selecting goals maximizing competence progress creates developmental trajectories driving the robot to progressively focus on tasks of increasing complexity and is statistically significantly more efficient than selecting tasks randomly, as well as more efficient than different standard active motor babbling methods; 3) this architecture allows the robot to actively discover which parts of its task space it can learn to reach and which part it cannot.

• The paper introduces SAGG-RIAC, an architecture that actively explores parameterized tasks to efficiently learn inverse models in complex robotic systems.
• It leverages competence progress measures to focus on tasks that maximize learning efficiency in high-dimensional, redundant sensorimotor mappings.
• Experimental results from robotic arms, quadruped locomotion, and fishing rod control validate SAGG-RIAC's superior performance over traditional methods.

Insights into Active Learning for Inverse Models in Robotics

The research presented in this paper by Baranes and Oudeyer explores novel approaches for the active learning of inverse models in robotic systems, specifically through the implementation of a new architecture termed Self-Adaptive Goal Generation - Robust Intelligent Adaptive Curiosity (SAGG-RIAC). This architecture is designed to handle high-dimensional and redundant spaces efficiently, focusing on active goal/task exploration rather than traditional random exploration or motor babbling.

Core Contributions

The paper delineates two primary contributions:

1. Active Goal Exploration: SAGG-RIAC actively samples parameterized tasks within a task space, driven by a competence progress measure. This involves self-generating goals/tasks which the robot then focuses on achieving. This method efficiently leverages the redundancy in sensorimotor mappings, strategically exploring only necessary subregions to achieve desired effects, contrasting previous approaches focused on motor babbling within actuator space.
2. Competence-Based Intrinsic Motivation: The architecture relies on evaluating competence progress to determine 'interestingness' in tasks, which actively directs the robot's learning focus towards tasks that enhance its developmental trajectory. This approach shows a departure from classical curiosity-driven methods that prioritize error minimization without direct focus on environmental or task-based constraints.

Experimental Setup and Observations

The paper meticulously details three experimental setups with varying robotic configurations:

• Robotic Arm Learning Inverse Kinematics: A 15-DOF robotic arm tasked with reaching various end-effector positions illustrated the architecture's proficiency in managing redundant kinematic spaces. The SAGG-RIAC mechanism showed superior outcomes compared to traditional methods, demonstrating efficient learning and generalization capabilities even when faced with significant dimensionality challenges.
• Quadruped Locomotion Control: A simulated quadruped's omnidirectional movement through motor synergy exploitation revealed how SAGG-RIAC guides exploration towards diverse and previously unattained stability regions within highly redundant motor spaces.
• Fishing Rod Control: In this experiment, a robotic arm controlling a fishing rod sought optimal 2D water contact positions of a flexible float. SAGG-RIAC effectively discovered spatial constraints without prior structured knowledge, highlighting its utility in handling complex, compliant environments.

Implications and Future Directions

The mathematical formulation and experimental validations affirm that active goal-directed exploration, as realized through SAGG-RIAC, significantly enhances the efficiency of learning inverse kinematic models, especially in redundancy-rich robotics. This approach departs from rigid, pre-defined task models, suggesting a paradigm that could redefine developmental robotics and cognitive development in AI systems.

However, it is noteworthy that initial task space specification still relies on prior defined parameters. Therefore, future development should aim at autonomously unveiling task spaces, perhaps benefiting from the integration of social guidance techniques or developmental constraints akin to biological maturation. Such additions could upgrade SAGG-RIAC into a comprehensive framework suitable for varied real-world applications, spanning unbounded operational domains.

The research sets a substantial precedent towards integrating active learning and intrinsic motivation into robotic systems, illustrating an essential step towards autonomous, adaptive machine behaviours akin to exploratory learning in biological entities. This paves the way for more autonomous robotic systems capable of self-directed, lifecycle-based learning and adaptation.