Hierarchical Associative Memory (HAM)

• Hierarchical Associative Memory (HAM) is a deep, multi-layer associative model that assembles complex memories from lower-layer primitives via both bottom-up and top-down recurrent dynamics.
• It employs layer-specific energies and a Lyapunov energy function to ensure stable convergence of neural activations to stored memory fixed points.
• Recent implementations extend HAM to continual learning, anomaly detection, and reinterpret deep architectures like Transformers through associative memory frameworks.

Hierarchical Associative Memory (HAM) denotes a family of associative-memory models in which content-addressable recall is implemented by deep, modular, recurrent dynamics rather than by a single shallow attractor layer. In the core formulation, HAM generalizes classical Hopfield networks and Dense Associative Memories to an arbitrary number of layers, allows some layers to be locally connected or convolutional, and equips the full system with a Lyapunov energy that decreases along the dynamical trajectory of the neurons’ activations (Krotov, 2021). Within this literature, “hierarchical” refers to architectural depth and modular organization, “associative” to recall from partial or corrupted cues, and “memory” to the tendency of the network to evolve toward low-energy stable fixed points that encode stored patterns (Krotov et al., 8 Jul 2025). A central claim of the original hierarchical model is that full memories are dynamically “assembled” from lower-layer primitives and higher-layer assembly rules, with rich top-down feedback participating directly in retrieval (Krotov, 2021).

1. Definition, scope, and conceptual basis

The immediate motivation for HAM was a limitation of earlier modern Hopfield-style models: they had only one hidden layer and were formulated with densely connected architectures, two features that hindered their machine-learning applicability (Krotov, 2021). HAM addresses this by introducing arbitrary depth, flexible connectivity, and a recurrent hierarchical structure in which information flows both bottom-up and top-down. In the core account, lower layers encode primitive patterns, motifs, or patch templates, while higher layers encode how those primitives are composed into more complex memories; retrieval is therefore not merely lookup, but a recurrent process of contextual assembly (Krotov, 2021).

The more recent modular-energy perspective broadens this definition. The 2025 tutorial on modern associative memory methods characterizes HAM as a generalization of classical associative memories from shallow, single-layer systems to deep, multi-layer, and possibly modular architectures. In that account, memory is not stored in a single monolithic layer but distributed over several layers, each operating at a different level or type of representation, and the model can combine different neuron Lagrangians and synaptic energies within a single energy-based formalism (Krotov et al., 8 Jul 2025). This suggests that HAM is best understood not as one fixed architecture, but as a design language for deep attractor systems.

A related, earlier precursor appears in biologically inspired vision. A 2013 extension of HMAX introduced memory and association mechanisms, distributed semantic storage, special episodic patches, and top-down adjustment into a hierarchical recognition model, with the explicit goal of mimicking active memory and association and reducing memory usage while preserving recognition performance (Hong et al., 2013). Although that work did not formulate the later energy-based HAM framework, it anticipated several recurring themes: hierarchy, distributed memory, association, and top-down regulation.

2. Energy-based formulation and recurrent dynamics

In the layered HAM formulation, the network consists of $N_{\text{layer}}$ layers, with layer $A$ containing states $x_i^A$. Each layer has a layer-specific Lagrangian $L^A$, with activations

$g_i^A = \frac{\partial L^A}{\partial x_i^A},$

and the neuron dynamics are

$$\tau_A \frac{dx_i^A}{dt} = \sum_{j=1}^{N_{A-1}} \xi^{(A,A-1)}_{ij} g_j^{A-1} + \sum_{j=1}^{N_{A+1}} \xi^{(A,A+1)}_{ij} g_j^{A+1} - x_i^A.$$

The corresponding energy is

$$E = \sum_{A=1}^{N_{\text{layer}}} \left[ \sum_{i=1}^{N_A} x_i^A g_i^A - L^A \right] - \sum_{A=1}^{N_{\text{layer}}-1} \sum_{i=1}^{N_{A+1}} \sum_{j=1}^{N_A} g_i^{A+1}\,\xi^{(A+1,A)}_{ij}\,g_j^A.$$

With symmetric interlayer weights and positive semi-definite layer Hessians, the energy is mathematically proven to decrease along trajectories; if the energy is also bounded below, trajectories converge to fixed points rather than limit cycles or chaos (Krotov, 2021).

The modular formulation in the 2025 tutorial recasts the same idea in terms of neuron energies and hypersynapses:

$$E_{\text{total}} = \sum_{\ell=1}^L E^\text{neuron}_\ell + \sum_{s=1}^S E^\text{synapse}_s.$$

Here, each neuron layer is parameterized by a convex Lagrangian $\mathcal{L}_x(x)$, with activation $\hat{x}=\nabla \mathcal{L}_x(x)$, while hypersynapses generalize ordinary pairwise synapses to hyperedges that may connect more than two layers. This abstraction is explicitly designed to recover classical Hopfield and DenseAM models as limiting cases, while also supporting arbitrary similarity functions, layerwise normalization, and multiway interactions (Krotov et al., 8 Jul 2025).

One important consequence is that HAM is not restricted to fully dense layers. The original paper permits locally connected and convolutional layers, and allows average pooling with proper feedback, while excluding max pooling because it does not admit a local energy-based feedback formulation (Krotov, 2021). This marks a significant departure from classical Hopfield networks, whose standard energy is quadratic and pairwise.

3. Storage, retrieval, continual writing, and forgetting

In the original hierarchical model, retrieval proceeds by clamping an input, possibly noisy or incomplete, to the input layer and letting recurrent dynamics propagate information upward and downward. Fixed points of the dynamics correspond to attractors that reconstruct or complete stored memories. The paper emphasizes an adiabatic regime in which higher layers equilibrate faster than lower ones, so that top layers settle quickly and provide relatively stable feedback to lower layers during retrieval (Krotov, 2021). This top-down pathway is not ancillary: it is described as helping lower-layer neurons decide on their response to the input stimuli.

Predictive-coding realizations make these mechanisms explicit. In “Associative Memories via Predictive Coding,” the model is a deep hierarchical generative network with layerwise value nodes and prediction-error nodes. Its global energy is the sum of squared prediction errors,

$A$0

and inference updates latent activities by local gradient descent,

$A$1

This model supports both autoassociative retrieval from corrupted or partial inputs and heteroassociative retrieval across modalities, such as retrieving images from descriptions and vice versa. In the reported experiments, a 10-layer predictive-coding network with 2048 hidden units retrieves more than $A$2 of Tiny ImageNet images when given half the pixels, and $A$3 when given only $A$4 of the pixels (Salvatori et al., 2021).

BayesPCN extends this line by focusing on the usually neglected write operation. It is presented as a hierarchical associative memory capable of continual one-shot memory writes without meta-learning, while also being able to gradually forget past observations to free memory (Yoo et al., 2022). Its parameters are Bayesian random variables, posterior updates are performed continually through approximate inference with a particle approximation, and forgetting is implemented by diffusion of posterior sufficient statistics back toward the prior. The abstract reports that BayesPCN can recall corrupted i.i.d. high-dimensional data observed hundreds to a thousand “timesteps” ago without a large drop in recall ability compared to state-of-the-art offline-learned parametric memory models (Yoo et al., 2022). Taken together, these predictive-coding variants show that HAM-style hierarchy can support not only recall, but also continual writing and principled forgetting.

4. Reinterpretation of modern architectures

A major recent development is the reinterpretation of contemporary deep-learning blocks as hierarchical associative memories. “Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry Breaking” proposes a framework in which the whole MetaFormer or Transformer block—including token-mixing, channel-mixing, layer normalization, and skip connections—corresponds to a single hierarchical three-layer Hopfield network (Karakida et al., 2024). In that construction, visible variables are arranged as a tokens-by-channels matrix, one hidden layer mediates token mixing, the other channel mixing, and layer normalization arises as the gradient of an appropriately chosen visible-layer Lagrangian.

The resulting update rule for the symmetric parallelized MLP-Mixer is

$A$5

This is noteworthy because it performs token and channel mixing in parallel rather than sequentially. The same paper also identifies an important tension between strict associative-memory theory and practical deep learning: symmetric interaction matrices hinder performance in image recognition tasks, whereas introducing symmetry breaking transitions the performance of the symmetric parallelized MLP-Mixer to that of the vanilla MLP-Mixer (Karakida et al., 2024). This directly challenges the common assumption that symmetry is always desirable in energy-based models.

The 2025 tutorial places this reinterpretation within a larger program. It states that Transformer attention and feed-forward modules can be recast as synaptic and neuron energies within the HAM framework, and further argues that diffusion-model training can be understood as training a hierarchical associative memory (Krotov et al., 8 Jul 2025). It also identifies HAMUX as a modular framework for building such models. A plausible implication is that HAM is becoming less a niche descendant of Hopfield networks and more a general theoretical interface between recurrent energy-based memory and mainstream deep architectures.

5. Representative systems and applied descendants

Recent systems have applied hierarchical associative principles well beyond classical pattern completion. In distributed anomaly detection, D-HTM introduces a Shared Associative Memory (SAM) that stores recurring pre-anomaly Sparse Distributed Representation windows and reuses them across entities to generate preemptive warnings. The framework combines a shared Spatial Pooler, entity-specific Temporal Memory modules, and a global SAM memory; evaluated on SMD, SMAP, MSL, and a synthetic cascade benchmark, it reports an average warning lead time of 8.1 samples prior to anomaly onset across the real-world datasets (Bera et al., 30 Jun 2026). The accompanying ablations are unusually diagnostic: a shared Spatial Pooler gives macro $A$6 versus $A$7 for raw features, and a shuffled precursor control drives $A$8 from $A$9 to $x_i^A$0 (Bera et al., 30 Jun 2026). Although D-HTM belongs to Hierarchical Temporal Memory rather than the Krotov-style HAM lineage, it demonstrates that hierarchical representation and shared associative retrieval can be operationalized in online, distributed settings.

In long-context LLM agents, HeLa-Mem proposes a bio-inspired hierarchical associative memory system built from a dynamic Hebbian episodic memory graph and a semantic memory store populated by Hebbian Distillation. Episodic edges evolve by a discrete Hebbian rule with decay, retrieval is augmented by spreading activation, and densely connected hubs are distilled into structured semantic records (Zhu et al., 18 Apr 2026). On LoCoMo, the system is reported to achieve the highest $x_i^A$1 and BLEU-1 scores across four question categories while using significantly fewer context tokens; the detailed summary gives a GPT-4o-mini Multi-hop QA $x_i^A$2 of $x_i^A$3 versus a next-best $x_i^A$4, Single-hop $x_i^A$5 of $x_i^A$6, and average token usage of about $x_i^A$7 (Zhu et al., 18 Apr 2026). On LongMemEval-S it reports best overall accuracy of $x_i^A$8 under the same retrieval budget (Zhu et al., 18 Apr 2026). These results suggest that the hierarchical-associative idea now spans graph memory, semantic consolidation, and retrieval control in agentic settings.

6. Terminology, neighboring acronyms, and unresolved issues

The acronym “HAM” is not unique, and conflating its meanings is a recurrent source of confusion. “Hierarchical Attentive Memory” is a separate 2016 memory architecture based on a binary tree whose leaves are memory cells and whose internal nodes store auxiliary information for efficient search; it achieves $x_i^A$9 memory access, can be combined with an LSTM controller, and was shown to learn sorting, merging, binary search, and classic data structures such as a stack, FIFO queue, and priority queue (Andrychowicz et al., 2016). Despite the identical acronym, that model is an external memory with hierarchical attention, not the energy-based hierarchical associative memory defined by recurrent attractor dynamics.

Likewise, “Hybrid Associative Memory” refers to a 2026 sequence layer that combines an RNN state with a selectively growing KV cache. Its core idea is that the RNN compresses the entire sequence while explicit attention memory stores only information that the RNN cannot predict well, controlled by a continuous threshold on prediction error (Lufkin et al., 20 Mar 2026). “HAM” also denotes “Hierarchy of Abstract Machines” in hierarchical reinforcement learning, where the hierarchy is a formal abstract automaton optimized in an internal or “mental” environment by Q-learning (Panov et al., 2018). These are distinct research programs.

Within the canonical hierarchical associative memory literature, several open issues remain explicit. The 2021 HAM paper states that explicit capacity analyses in multilayer HAM settings remain an open challenge (Krotov, 2021). The MetaFormer reinterpretation identifies memorization capacity and energy-landscape design under symmetry breaking as directions for further work (Karakida et al., 2024). At the same time, the modern tutorial argues that modular energies and hypersynapses give HAM a level of mathematical generality far beyond classical Hopfield networks, including support for arbitrary depth, multiway interactions, local connectivity, and compatibility with Transformers, diffusion models, and deep clustering (Krotov et al., 8 Jul 2025). This suggests that the contemporary significance of HAM lies less in a single canonical architecture than in a general theory of deep associative computation.