Curse of Memory in Cognition & AI

• Curse of Memory is a phenomenon where excessive retention leads to interference, exponential resource needs, and impaired learning in both human cognition and AI models.
• The concept elucidates trade-offs between memory capacity, retrieval efficiency, and controlled forgetting, emphasizing strategies like interference mitigation and stable reparameterization.
• Algorithmic and architectural remedies, such as variance-reduction techniques and decoupled memory networks, demonstrate practical solutions for managing long-horizon dependencies in continual learning.

The curse of memory refers to phenomena, spanning cognitive science and artificial intelligence, in which excessive memorization, long dependency horizons, or persistent correlations fundamentally impede efficient computation, learning, or adaptation. Contrary to the naive assumption that more memory is unconditionally beneficial, both theoretical and empirical results demonstrate that retaining too much past information can introduce severe inefficiencies, such as increased interference, exponential growth in resource requirements, persistent estimation bias, and impaired learning of new knowledge.

1. Foundational Principles in Human and Artificial Memory

The curse of memory arises from intrinsic trade-offs between memory capacity, retention, and interference. In human cognition, forgetting operates as an adaptive process to mitigate interference and maintain efficient retrieval. Quantitative models, such as Poisson/Bayesian retrieval time formalisms, show that memory systems minimizing expected search time must prune low-value or obsolete entries—otherwise both interference and retrieval latency diverge as memory grows. In deep learning, recurrent neural networks (RNNs) and state-space models (SSMs) face analogous limitations: modeling or optimizing for long-horizon dependencies incurs exponential complexity and ill-conditioning unless architectural or algorithmic interventions specifically address these memory bottlenecks (Yu et al., 2018, Li et al., 2020, Wang et al., 2023).

2. The Curse of Memory in Biological and Psychological Models

Human memory capacity and forgetting dynamics exhibit well-characterized constraints:

• Retrieval-time minimization: For $N$ stored situations with probabilities $P_1 \geq ... \geq P_N$, optimal sequential search via the rearrangement inequality yields a minimum expected retrieval time $E[T.O.] = \sum_{i=1}^N i P_i$ (Yu et al., 2018).
• Forgetting as interference mitigation: Forgetting reduces memory interference by pruning lesser-used or obsolete traces. Modeling with a Poisson encounter process and Bayesian inference, retrieval probability decays with increasing noise, directly explaining features of the Ebbinghaus forgetting curve.
• Capacity limitations: Linear interference models reproduce the Miller "magic number 7 ± 2", showing that maximal immediate recall capacity is bounded by interference between memory traces; perfect memory (no forgetting) would cause search and interference costs to diverge.
• Retroactive interference and power-law forgetting: Multi-dimensional valence models, with new items eliminating all strictly inferior traces, yield analytic retention curves exhibiting power-law decay—empirically matching human recognition data with approximately five orthogonal "importance axes" (Georgiou et al., 2019). This formalizes the trade-off between retention and necessary forgetting as an active process maintaining salient content while discarding expendable details.

3. The Curse of Memory in Recurrent and State-space Models

In RNNs and SSMs, the curse is sharply quantified by the relationship between memory horizon and computational requirements:

• Exponential model size and optimization time: For input-output functionals with memory horizon $M$, any linear RNN or vanilla SSM that $\epsilon$-approximates all such functionals requires width

$N \geq C_1 \exp(c_1 M)$

and gradient-based optimization time at least

$$T_{GF}(\epsilon) \geq C_2 \exp(c_2 M) \log(1/\epsilon)$$

for universal $C_1, C_2, c_1, c_2 > 0$ (Li et al., 2020, Wang et al., 2023). Both barriers are exponential in memory horizon, constituting the curse of memory.

• Stability boundaries in SSMs: Unreparameterized SSMs can only stably approximate functionals with exponentially decaying memory. The spectrum of recurrent weights must converge to the stability boundary when learning long-memory targets, leading to instability and ill-conditioned gradients.
• Architectural remedies: Stable reparameterizations of the recurrent weights, such as $f(w) = -1/(a w^2 + b)$, lift the exponential decay restriction and restore universal, stable approximation and optimization for functionals with polynomially decaying memory (Wang et al., 2023).

4. Algorithmic and Statistical Manifestations

Beyond architecture, the curse of memory appears in algorithmic and stochastic learning settings:

• Stochastic approximation with Markovian noise: Classic Polyak-Ruppert averaging with constant step size $\alpha$ yields unbiased optimal estimates for i.i.d. disturbances. However, under Markovian (temporally persistent) noise, an irreducible $O(\alpha)$ bias remains due to persistent cross-temporal covariances. This bias is directly sourced from solutions of the Poisson equation linked to the disturbance sequence and cannot be eliminated by averaging alone. Asymptotic covariance similarly deteriorates by an $O(\alpha)$ term, amplifiable in poorly conditioned systems (Lauand et al., 2023).
• Algorithmic implications: Standard stochastic approximation fails to deliver unbiased estimates in settings with memory (non-i.i.d. disturbance), necessitating memory-breaking interventions such as batch i.i.d. resampling or Poisson-equation-based variancereduction.

5. Lifelong Learning and Anterograde Forgetting

In settings requiring continual learning, the curse of memory not only materializes as catastrophic forgetting (loss of old knowledge) but also as anterograde forgetting (inhibition of new learning):

• Anterograde forgetting: Efforts to rigidly preserve or transfer old knowledge constrain learning capacity and introduce "conceptual confusion" by injecting irrelevant features from previous tasks. These mechanisms slow convergence and degrade new-task performance (Peng et al., 2021).
• Quantifying AF: Metrics comparing continual learning models against joint-training and from-scratch baselines reveal the net negative influence of excessive memory transfer.
• Architectural mitigation: Cycled Memory Networks (CMN) decouple short-term and long-term memory networks, employing recall-gated transfer cells and periodic consolidation. This enables isolation of new learning, controlled knowledge transfer, and effective integration—breaking the trade-off that otherwise characterizes the curse of memory.

6. Mathematical and Computational Remedies

Various mathematical and algorithmic strategies have arisen to circumvent or exploit the curse:

Domain	Limitation Imposed by Curse	Key Remedy or Principle
Biological/Human memory	Unbounded interference, slow retrieval	Active forgetting, interference-aware organization
RNN/SSM approximation (unparameterized)	Exponential width, slow optimization	Stable reparameterization, restricted functional targets
Stochastic approximation (Markovian)	Persistent estimation bias	Memory-breaking algorithms, variance-reduction using Poisson equations
Lifelong learning/continual task networks	Anterograde forgetting, capacity shrinkage	Decoupled memory systems, gated transfer, cyclical consolidation

All these approaches rest on the recognition that memory is not a monotonic blessing; optimal systems balance retention capacity, retrieval efficiency, and adaptivity by controlled forgetting or architectural decoupling.

7. Broader Implications and Open Questions

The curse of memory underscores a fundamental computational constraint: without mechanisms for forgetting, interference and inefficiency are inevitable. In both natural and artificial systems, optimal functionality emerges not from maximal memory but from discriminative retention and controlled pruning. Ongoing directions include characterization of non-diagonal stability in SSMs, learning adaptive reparameterization maps for optimal memory-approximation trade-offs, and developing robust statistical procedures under various non-i.i.d. dependencies.

The concept persists as a guiding principle for memory system design across cognitive science, neural computation, and statistical learning, justifying the systematic incorporation of forgetting mechanisms and memory-specific regularization at both algorithmic and architectural levels (Yu et al., 2018, Georgiou et al., 2019, Li et al., 2020, Lauand et al., 2023, Wang et al., 2023, Peng et al., 2021).