Exponential Dynamic Energy Network (EDEN)

• EDEN is a dynamic memory system that unifies static and sequential memory by integrating fast feature neurons with slow modulatory dynamics.
• It achieves exponential memory capacity by combining a high-capacity static energy network with asymmetrically coupled slow variables to control transitions.
• EDEN’s framework not only enables robust state transitions and analytical escape-time predictions but also mirrors biological phenomena in hippocampal replay.

The Exponential Dynamic Energy Network (EDEN) is an architecture for high-capacity sequence memory that extends the classical energy-based paradigm from static associative memory to sequential memory by incorporating multi-timescale dynamics. EDEN achieves exponential memory capacity by combining a high-capacity static energy network with a slow, asymmetrically coupled modulatory population, thereby enabling robust, controllable transitions between stored memories. This framework unifies static and sequential memory within a dynamic energy landscape and draws strong analogies with biological sequential memory phenomena observed in neural systems (Karuvally et al., 28 Oct 2025).

1. Network Structure and Dynamics

EDEN comprises two primary interacting populations:

• Fast Population: Consists of feature neurons $v \in \mathbb{R}^N$ and hidden neurons $h \in \mathbb{R}^P$. Memories are stored as binary patterns $\xi^{(\mu)} \in \{\pm1\}^N$, indexed by $\mu=1,\dots,P$, where $\xi_{i\mu} = \xi_i^{(\mu)}$ encodes the symmetric weights.
• Slow Modulatory Population: The modulatory variables $s \in \mathbb{R}^N$ provide a leaky history-dependent trace that breaks detailed balance and drives transitions between memories.

Static High-Capacity Energy Network

With the slow modulatory population fixed ($s \equiv 0$), the network dynamics are governed by the gradient flow: $$\mathcal{T}_f \, \dot v_i = \sum_{\mu=1}^P \xi_{i}^{(\mu)} p_\mu(v) - v_i,$$ where the hidden unit activation employs a softmax readout: $$h_\mu = \alpha_s \sum_{i=1}^N \xi_i^{(\mu)} v_i, \qquad p_\mu(v) = \frac{\exp(h_\mu)}{\sum_{\nu=1}^P \exp(h_\nu)}.$$ This corresponds to gradient descent on the "exponential" energy function: $$E_{\rm static}(v) = \frac{1}{2} \|v\|^2 - \frac{1}{\alpha_s} \log \left[ \sum_{\mu=1}^P \exp \left( \alpha_s \sum_i \xi_i^{(\mu)} v_i \right) \right].$$ This generalizes the Hopfield energy by introducing nonlinearity that enhances memory separation and capacity.

Slow Modulatory Dynamics and Cross-Memory Coupling

The slow population evolves according to

$h \in \mathbb{R}^P$0

implying $h \in \mathbb{R}^P$1 is a leaky integrator of the recent feature-layer activity. When $h \in \mathbb{R}^P$2, the drive to hidden units is modified as: $h \in \mathbb{R}^P$3 where $h \in \mathbb{R}^P$4 ("self-coupling") stabilizes the current pattern, and $h \in \mathbb{R}^P$5 ("cross-coupling") introduces a delayed cue biasing the transition to the next memory in the sequence.

Combined Fast and Slow Dynamics

The overall energy function becomes

$h \in \mathbb{R}^P$6

and the network realizes metastable attractor dynamics. On short timescales, the system minimizes $h \in \mathbb{R}^P$7 with respect to $h \in \mathbb{R}^P$8, while on longer timescales, $h \in \mathbb{R}^P$9 slowly tilts the energy landscape and mediates controlled transitions between attractors.

2. Short-Timescale Energy and Memory Selection

For fixed $\xi^{(\mu)} \in \{\pm1\}^N$0, the effective energy governing the fast variable is

$\xi^{(\mu)} \in \{\pm1\}^N$1

where $\xi^{(\mu)} \in \{\pm1\}^N$2 and $\xi^{(\mu)} \in \{\pm1\}^N$3. Dynamics converge to the minimum corresponding to the memory with maximal net drive $\xi^{(\mu)} \in \{\pm1\}^N$4. If $\xi^{(\mu)} \in \{\pm1\}^N$5, well-defined minima remain near individual memories for extended durations.

3. Analytical Memory Transition (Escape-Time) Theory

EDEN supports deterministic transitions between sequential memories. The escape time $\xi^{(\mu)} \in \{\pm1\}^N$6—duration before the network spontaneously transitions from memory $\xi^{(\mu)} \in \{\pm1\}^N$7 to $\xi^{(\mu)} \in \{\pm1\}^N$8—is analytically tractable. Neglecting finite-$\xi^{(\mu)} \in \{\pm1\}^N$9 effects, the escape time is: $\mu=1,\dots,P$0 or equivalently,

$\mu=1,\dots,P$1

Stochastic fluctuations introduce $\mu=1,\dots,P$2 corrections but do not alter scaling. Simulation results confirm the validity of the analytic transition time over a broad parameter range.

4. Phase Transition and Regime Boundary

A critical property of EDEN is the phase transition between static and sequential attractor dynamics, with $\mu=1,\dots,P$3 serving as the order parameter. Two regimes are observed:

• If $\mu=1,\dots,P$4, then $\mu=1,\dots,P$5 (static attractor regime, no transitions).
• If $\mu=1,\dots,P$6, then $\mu=1,\dots,P$7 and robust sequential memory transitions occur at regular intervals.

The boundary at $\mu=1,\dots,P$8 corresponds to a saddle-node bifurcation where stability of one memory ceases coincident with the emergence of the next, marking a sharp transition from static to dynamic sequence replay.

5. Capacity Analysis and Scalability

EDEN achieves exponential sequence memory capacity, significantly exceeding conventional models. The formal capacity, $\mu=1,\dots,P$9, is defined as the maximal sequence length $\xi_{i\mu} = \xi_i^{(\mu)}$0 recalling each pattern $\xi_{i\mu} = \xi_i^{(\mu)}$1 with per-bit overlap at least $\xi_{i\mu} = \xi_i^{(\mu)}$2 and probability at least $\xi_{i\mu} = \xi_i^{(\mu)}$3.

• For $\xi_{i\mu} = \xi_i^{(\mu)}$4 (dynamic regime):

$\xi_{i\mu} = \xi_i^{(\mu)}$5

where $\xi_{i\mu} = \xi_i^{(\mu)}$6 grows sub-exponentially with $\xi_{i\mu} = \xi_i^{(\mu)}$7 and setting $\xi_{i\mu} = \xi_i^{(\mu)}$8, the capacity scales as $\xi_{i\mu} = \xi_i^{(\mu)}$9. For typical $s \in \mathbb{R}^N$0, $s \in \mathbb{R}^N$1, closely approaching the information-theoretic limit $s \in \mathbb{R}^N$2.

• For reference, a Hopfield-style model with linear interactions yields only $s \in \mathbb{R}^N$3.

All connectivity matrices $s \in \mathbb{R}^N$4 are fixed once patterns are set, so retrieval and sequential transitions rely solely on network dynamics without further synaptic plasticity.

6. Biological Relevance and Neural Analogues

EDEN reproduces qualitative and quantitative features observed in biological sequence memory:

• Hidden units $s \in \mathbb{R}^N$5 activate in order, each peaking sequentially, consistent with hippocampal "time cells" as described by Umbach et al. (2020).
• Slow modulatory units $s \in \mathbb{R}^N$6 display ramping dynamics analogous to "ramping cells" in the entorhinal cortex.
• Predicted escape times for memory transitions are congruent with empirical temporal gaps measured in MEG/EEG studies of human episodic memory replay (Wimmer et al. 2020).

This biological grounding suggests EDEN as a mechanistic model for hippocampal replay, sequential working memory, and time-resolved neural coding in the cortex.

7. Interpretability, Applications, and Extensions

EDEN offers practical advantages for scalable sequence memory and interpretability:

• Retrieval and Stability: At any instant, the dynamic energy $s \in \mathbb{R}^N$7 enables visualization and prediction of current stable and incipient memories.
• Applications: Potential uses include external memory modules for AI (e.g., in language or time-series modeling), as well as theoretical models in computational neuroscience and sequential decision-making.
• Scalability: Owing to exponential capacity in network size ($s \in \mathbb{R}^N$8), EDEN can support extensive sequential recall in comparatively small networks.
• Extensions: Possible generalizations include multi-branch or graph-like sequence structures, tolerance to partial/noisy cues, continuous-valued activity patterns, adaptive scheduling of $s \in \mathbb{R}^N$9 and $s \equiv 0$0, and stochastic (temperature-driven) dynamics enabling probabilistic transitions or retrieval failure.

A plausible implication is that EDEN’s unification of static attractor memory and sequential pattern replay within a principled energy framework will facilitate future advances both in artificial memory systems and neurobiological modeling (Karuvally et al., 28 Oct 2025).