Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments (2201.00042v2)

Abstract: A key challenge for AI is to build embodied systems that operate in dynamically changing environments. Such systems must adapt to changing task contexts and learn continuously. Although standard deep learning systems achieve state of the art results on static benchmarks, they often struggle in dynamic scenarios. In these settings, error signals from multiple contexts can interfere with one another, ultimately leading to a phenomenon known as catastrophic forgetting. In this article we investigate biologically inspired architectures as solutions to these problems. Specifically, we show that the biophysical properties of dendrites and local inhibitory systems enable networks to dynamically restrict and route information in a context-specific manner. Our key contributions are as follows. First, we propose a novel artificial neural network architecture that incorporates active dendrites and sparse representations into the standard deep learning framework. Next, we study the performance of this architecture on two separate benchmarks requiring task-based adaptation: Meta-World, a multi-task reinforcement learning environment where a robotic agent must learn to solve a variety of manipulation tasks simultaneously; and a continual learning benchmark in which the model's prediction task changes throughout training. Analysis on both benchmarks demonstrates the emergence of overlapping but distinct and sparse subnetworks, allowing the system to fluidly learn multiple tasks with minimal forgetting. Our neural implementation marks the first time a single architecture has achieved competitive results on both multi-task and continual learning settings. Our research sheds light on how biological properties of neurons can inform deep learning systems to address dynamic scenarios that are typically impossible for traditional ANNs to solve.

Overview of "Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments"

The paper "Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments" introduces an innovative approach for enhancing artificial neural networks (ANNs) to effectively manage multi-task and continual learning challenges, particularly focusing on dynamic environments. The authors propose a biologically inspired architecture that incorporates the properties of active dendrites and sparse representations, drawing from insights obtained from the paper of pyramidal neurons in biological systems.

Key Contributions and Architecture

1. Active Dendrites and Sparse Representations: The core idea is to build on the traditional point neuron model by integrating active dendrites and imposing sparsity. The active dendrites, inspired by non-linear dendritic conductances found in biological neurons, allow context-dependent modulation of neural activations, enhancing the network's ability to retain task-specific information without interference.
2. Multi-Task Learning and Continual Learning Scenarios: The architecture was tested on two challenging learning scenarios. In the multi-task reinforcement learning (MTRL) setting, the model demonstrated superior performance on the MT10 benchmark, which involves a robotic arm learning ten distinct tasks simultaneously. For continual learning, the architecture was assessed using the permutedMNIST benchmark, showcasing impressive results over 100 sequential tasks.
3. Subnetwork Formation: Through the use of sparse activations and modulation by dendritic segments, distinct sparse subnetworks emerge for different tasks. This separation helps mitigate catastrophic forgetting by allowing task-specific pathways within the network that do not interfere with one another.
4. Neuroscientific Insights: The paper draws on neuroscientific observations, proposing that dendrites in pyramidal neurons enable dynamic context-specific processing. This adaptation allows biological systems to switch between different operational modes depending on the contextual signals received by these dendritic structures.

Results and Implications

The proposed Active Dendrites Networks not only surpass baseline models, such as standard multilayer perceptrons (MLPs) in multi-task RL scenarios, but they also integrate well with existing techniques like Synaptic Intelligence (SI) for continual learning. This fusion results in a significant reduction in task interference and a higher retention of learned tasks over time. Specifically, the results communicated are as follows:

• In MTRL, the networks achieved approximately 87.5% success rates across varied tasks, outperforming MLP baselines.
• In continual learning scenarios, the combination with SI yielded accuracy improvements, achieving over 90% in settings with 100 tasks.

Future Directions

The research opens several pathways for further exploration. Implementing this architecture in more complex, real-world scenarios is a logical next step. Additionally, refining methods to dynamically generate context vectors and expanding the framework to incorporate recurrent and feedback connections akin to apical dendrites will enrich the model's applicability and biological plausibility. Lastly, investigating the use of sparse dendritic segments, inspired by evidence of sparse connectivity in biological neural circuits, may further optimize the architecture.

In summary, this paper represents a significant advancement in the intersection of neuroscience and machine learning, advocating for the utility of biologically inspired mechanisms in addressing longstanding AI challenges such as catastrophic forgetting and task interference in dynamic environments.