Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Forgetting leads to chaos in attractor networks (2112.00119v1)

Published 30 Nov 2021 in q-bio.NC and cond-mat.dis-nn

Abstract: Attractor networks are an influential theory for memory storage in brain systems. This theory has recently been challenged by the observation of strong temporal variability in neuronal recordings during memory tasks. In this work, we study a sparsely connected attractor network where memories are learned according to a Hebbian synaptic plasticity rule. After recapitulating known results for the continuous, sparsely connected Hopfield model, we investigate a model in which new memories are learned continuously and old memories are forgotten, using an online synaptic plasticity rule. We show that for a forgetting time scale that optimizes storage capacity, the qualitative features of the network's memory retrieval dynamics are age-dependent: most recent memories are retrieved as fixed-point attractors while older memories are retrieved as chaotic attractors characterized by strong heterogeneity and temporal fluctuations. Therefore, fixed-point and chaotic attractors co-exist in the network phase space. The network presents a continuum of statistically distinguishable memory states, where chaotic fluctuations appear abruptly above a critical age and then increase gradually until the memory disappears. We develop a dynamical mean field theory (DMFT) to analyze the age-dependent dynamics and compare the theory with simulations of large networks. Our numerical simulations show that a high-degree of sparsity is necessary for the DMFT to accurately predict the network capacity. Finally, our theory provides specific predictions for delay response tasks with aging memoranda. Our theory of attractor networks that continuously learn new information at the price of forgetting old memories can account for the observed diversity of retrieval states in the cortex, and in particular the strong temporal fluctuations of cortical activity.

Citations (16)

Summary

We haven't generated a summary for this paper yet.