Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning (2406.09787v1)

Abstract: Biological neural networks are characterized by their high degree of plasticity, a core property that enables the remarkable adaptability of natural organisms. Importantly, this ability affects both the synaptic strength and the topology of the nervous systems. Artificial neural networks, on the other hand, have been mainly designed as static, fully connected structures that can be notoriously brittle in the face of changing environments and novel inputs. Building on previous works on Neural Developmental Programs (NDPs), we propose a class of self-organizing neural networks capable of synaptic and structural plasticity in an activity and reward-dependent manner which we call Lifelong Neural Developmental Program (LNDP). We present an instance of such a network built on the graph transformer architecture and propose a mechanism for pre-experience plasticity based on the spontaneous activity of sensory neurons. Our results demonstrate the ability of the model to learn from experiences in different control tasks starting from randomly connected or empty networks. We further show that structural plasticity is advantageous in environments necessitating fast adaptation or with non-stationary rewards.

Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning

The paper "Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning," authored by Erwan Plantec et al., explores the development of neural networks that approximate the structural and synaptic plasticity of biological systems. The researchers introduce Lifelong Neural Developmental Programs (LNDPs), which extend previous models such as Neural Developmental Programs (NDPs) by enabling continuous learning and network adaptation throughout the life of an agent.

Background and Motivation

The inspiration for this work originates from the inherent adaptability of biological neural networks, which adeptly modify both synaptic strengths and network topology to accommodate changing environments. Traditional artificial neural networks, which often operate as static, fully connected structures, are limited in this regard. The paper aims to narrow this gap by introducing a system that fosters both synaptic and structural plasticity, motivating the design through concepts from open-ended evolutionary processes and developmental neuroscience.

Principal Contributions

1. Lifelong Neural Developmental Programs (LNDPs): The LNDP framework facilitates synaptic and structural plasticity that is both activity- and reward-dependent. The architecture leverages a graph transformer layer for neuronal communication and enables a degree of self-organization and differentiation among neurons by modeling synaptic dynamics with Gated Recurrent Units (GRUs).
2. Pre-experience Spontaneous Activity: One of the innovative aspects of this work is the incorporation of a pre-experience developmental phase driven by spontaneous activity (SA). Modeled through an Ornstein-Uhlenbeck stochastic process, this phase allows networks to pre-organize into functional configurations, thus endowing agents with innate problem-solving skills before interacting with the environment.
3. Empirical Evaluation: The effectiveness of LNDPs is demonstrated across various control tasks, including CartPole and an innovative Foraging task designed with non-stationary rewards. In particular, structural plasticity proves advantageous in tasks that demand quick adaptation.

Results and Implications

The research presents strong numerical outcomes indicating that LNDPs with structural plasticity outperform their static counterparts in rapidly changing environments. Specifically, in environments requiring quick adaptation, LNDPs demonstrate superior learning capabilities. On the CartPole task, structurally plastic LNDPs show significant potential by adapting from initially non-functional network states to achieving success within a single episode.

The implications of these findings are manifold. Theoretically, LNDPs offer a promising method to emulate the adaptive efficiency of biological neural systems within artificial counterparts. Practically, this approach could lead to more robust AI systems capable of lifelong learning and adaptation, crucial features for real-world applications such as robotics and adaptive user interfaces.

Future Directions

Future work could explore integrating more biologically inspired learning rules into the LNDP framework to enhance its adaptability further or improve scalability to more complex tasks. Additionally, advancements could focus on optimizing training strategies, perhaps drawing from evolutionary or novelty-driven search algorithms to enhance the discovery of efficient neural architectures.

In conclusion, the work contributes significantly to the field of adaptive artificial intelligence by proposing mechanisms for self-organized plasticity reflecting biological principles. While challenges remain, including scaling and optimization, LNDPs represent a salient step toward neural networks capable of lifelong adaptation, potentially broadening the horizons of AI capability.