Autonomous development and learning in artificial intelligence and robotics: Scaling up deep learning to human--like learning (1712.01626v1)

Abstract: Autonomous lifelong development and learning is a fundamental capability of humans, differentiating them from current deep learning systems. However, other branches of artificial intelligence have designed crucial ingredients towards autonomous learning: curiosity and intrinsic motivation, social learning and natural interaction with peers, and embodiment. These mechanisms guide exploration and autonomous choice of goals, and integrating them with deep learning opens stimulating perspectives. Deep learning (DL) approaches made great advances in artificial intelligence, but are still far away from human learning. As argued convincingly by Lake et al., differences include human capabilities to learn causal models of the world from very little data, leveraging compositional representations and priors like intuitive physics and psychology. However, there are other fundamental differences between current DL systems and human learning, as well as technical ingredients to fill this gap, that are either superficially, or not adequately, discussed by Lake et al. These fundamental mechanisms relate to autonomous development and learning. They are bound to play a central role in artificial intelligence in the future. Current DL systems require engineers to manually specify a task-specific objective function for every new task, and learn through off-line processing of large training databases. On the contrary, humans learn autonomously open-ended repertoires of skills, deciding for themselves which goals to pursue or value, and which skills to explore, driven by intrinsic motivation/curiosity and social learning through natural interaction with peers. Such learning processes are incremental, online, and progressive. Human child development involves a progressive increase of complexity in a curriculum of learning where skills are explored, acquired, and built on each other, through particular ordering and timing. Finally, human learning happens in the physical world, and through bodily and physical experimentation, under severe constraints on energy, time, and computational resources. In the two last decades, the field of Developmental and Cognitive Robotics (Cangelosi and Schlesinger, 2015, Asada et al., 2009), in strong interaction with developmental psychology and neuroscience, has achieved significant advances in computational

Autonomous Development and Learning in AI and Robotics: Scaling Up Deep Learning to Human-like Learning

Pierre-Yves Oudeyer's paper, "Autonomous development and learning in artificial intelligence and robotics: Scaling up deep learning to human–like learning," emphasizes the essential distinctions between current deep learning (DL) systems and human learning capabilities. The paper articulates critical mechanisms that contribute to autonomous development in humans and suggests integrating these mechanisms could bridge the disparity between artificial and human learning.

Key Mechanisms in Autonomous Learning

The cornerstone of the paper is the identification of mechanisms that underpin autonomous learning, namely:

1. Intrinsic Motivation and Curiosity: Motivational models enable children to pursue goals and practice skills autonomously. Models driven by maximizing learning progress have shown to self-organize complex developmental structures. For instance, early infant vocal development can emerge spontaneously through intrinsically motivated exploration, influenced by the physical properties of their vocal systems.
2. Social Learning and Interaction: Humans frequently utilize social learning and natural interactions with peers, significantly contributing to incremental, online, and progressive learning. Such methods haven't been fully explored within DL applications.
3. Embodiment: Physical embodiment is another crucial factor. The interaction of a human body with its environment can naturally guide learning and exploration, whereas current DL often neglects the physicality aspect. Research has demonstrated that human-like gait patterns and motor skills can self-organize from the physical properties of robotic limbs designed to mimic human morphology.

Implications and Future Directions

Oudeyer's work points towards the integration of these mechanisms with DL, potentially leading to more human-like learning. This includes the development of models of:

• Intrinsic Motivation: Such models have shown high efficacy in exploration and learning across multiple tasks within high-dimensional spaces. By automating the generation of learning curriculums, robots can autonomously decide which tasks to pursue, leading to more efficient skill acquisition.
• Social Learning Integration: Combining social learning strategies with intrinsic motivation in DL systems promises a more holistic model of autonomous learning. This includes the ability of models to imitate and learn from human interactions and tutelage.
• Embodied Learning: Emphasizing the role of embodiment could lead to significant improvements in robotic functionalities, from enhanced locomotion skills to more adept manipulation of objects.

Practical Applications

The insights gained from these mechanisms offer intriguing practical applications:

• Enhanced Robotic Systems: Robots could autonomously adjust strategies for novel environments, particularly useful in unpredictable or hazardous conditions that require high adaptability.
• Improved Human-Robot Interaction: Robots employing these mechanisms could better understand and predict human behaviors, leading to more natural and efficient interactions.
• Multitask Learning: Implementing curiosity-driven exploration and learning progress optimization could result in more capable and versatile autonomous systems, adept at handling a wide array of tasks simultaneously.

Theoretical Contributions

From a theoretical standpoint, integrating these autonomous development principles in DL systems might lead to advancements in understanding learning and adaptation processes. It could offer:

• New Computational Models: Intrinsically motivated exploration and the interaction between social and physical learning mechanisms could yield novel models, applicable to both artificial and biological systems.
• Cross-disciplinary Insights: Investigations drawing on developmental psychology, neuroscience, and robotics could synergistically inform each discipline, fostering a more cohesive understanding of learning and development.

Conclusion

Pierre-Yves Oudeyer’s paper underscores the pivotal role of autonomous development mechanisms in achieving human-like learning in AI and robotics. By integrating intrinsic motivation, social learning, and embodiment with deep learning, the research delineates a pathway towards creating more adaptive, resilient, and autonomous artificial systems. Future progression in AI necessitates a deeper exploration of these interactions and their practical applications, potentially revolutionizing the field.