Hopfield Network: Memory & Dynamics

• Hopfield network is an auto-associative, content-addressable model defined by symmetric recurrent dynamics that relax states toward fixed-point attractors.
• The model employs Hebbian and optimization-based learning rules to store binary or continuous patterns, with retrieval achieved through coordinate descent on an energy function.
• Recent extensions include higher-order interactions, quantum formulations, and hardware implementations, all aimed at enhancing capacity, robustness, and computational efficiency.

A Hopfield network is an auto-associative, content-addressable memory model implemented as a symmetric recurrent neural network whose retrieval dynamics are defined by descent on an energy landscape. In its classical form, the network stores patterns as attractors of a fully connected recurrent system with binary neurons; in continuous and later variants, the same organizing idea appears in gradient systems, predictive-coding formulations, higher-order interaction models, quantum generalizations, and hardware-native implementations. Across these formulations, memory retrieval is not a feedforward mapping but a relaxation process that moves a state toward a stable fixed point or, in some generalized settings, a more elaborate attractor structure (Ganjidoost et al., 2024, Schaft, 6 Jan 2026, Burns et al., 2023).

1. Classical discrete formulation

In a common convention, a Hopfield network is a fully-connected, symmetric recurrent neural network with binary neurons $v \in \{-1,+1\}^M$, no self-connections $W_{ii}=0$, and symmetric weights $W_{ij}=W_{ji}$. It stores a finite set of memories $X_1,\dots,X_N \in \{-1,+1\}^M$ by choosing weights and biases so that each stored pattern is an attractor. The elementary asynchronous update rule is

$$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$

with synchronous updates also considered in the literature (Ganjidoost et al., 2024).

These dynamics perform coordinate descent on an energy function. One standard expression is

$E(x)= -\tfrac12 x^T W x + b^T x,$

with the equivalent component form

$$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$

Because the update can only decrease $E$, fixed points are local minima of the energy landscape. In the bipolar image-storage formulation, the same monotone descent property is written as

$E(s)= -\frac12 s^TWs,$

for symmetric $W$ with zero diagonal, implying finite-step convergence of asynchronous dynamics to a fixed-point attractor (Ganjidoost et al., 2024, Ramya et al., 2011).

The canonical storage prescription is the Hebbian outer-product rule. For binary patterns collected in $W_{ii}=0$0, one choice is

$W_{ii}=0$1

In a related bipolar convention, the rule is

$W_{ii}=0$2

Under this classical construction, random-pattern storage capacity is reported as approximately $W_{ii}=0$3 or $W_{ii}=0$4, depending on notation (Ganjidoost et al., 2024, Ramya et al., 2011).

2. Retrieval, attractors, and capacity

The defining computational role of a Hopfield network is associative recall: starting from an incomplete, noisy, or corrupted version of a stored pattern, the dynamics move the state toward the corresponding attractor. In image-recall settings, binary images are thresholded, mapped to bipolar vectors, and stored in the weight matrix; degraded test images then converge to the closest matching stored pattern, with Hamming distance

$W_{ii}=0$5

used as a natural similarity measure (Ramya et al., 2011).

A central point in the theory is that stable states are not limited to the training set. States can be grouped into learned states, spurious states, and prototype states. Learned states are those presented during training; spurious states are stable fixed points that were not learned; prototype states are stable states that were not learned but are representative for a subset of learned states. This classification matters operationally because retrieval may terminate in an attractor that is stable yet not semantically desired (McAlister et al., 4 Mar 2025).

Classical capacity figures describe only one regime. With Hebbian storage of random patterns, the widely cited limit is approximately $W_{ii}=0$6 patterns per neuron (Ganjidoost et al., 2024). Other constructions study capacity in information-theoretic or coding-theoretic terms. One binary construction partitions $W_{ii}=0$7 neurons into robust clusters of size $W_{ii}=0$8 and attains information rate $W_{ii}=0$9 with asymptotically vanishing error rates; the same cluster mechanism can be used as a noisy decoder for the discrete grid-cell code (Fiete et al., 2014). Another line of work uses probability-flow minimization to construct families of Hopfield networks that robustly store

$W_{ij}=W_{ji}$0

with $W_{ij}=W_{ji}$1 and thus $W_{ij}=W_{ji}$2, together with robustness index $W_{ij}=W_{ji}$3 under independent bit flips (Hillar et al., 2014).

Taken together, these results suggest that “capacity” is not a single invariant quantity of the model family but depends on the pattern ensemble, learning rule, architecture, and error criterion. A large count of fixed points need not imply useful associative memory if basins are small or dominated by spurious minima, whereas structured constructions can trade generality for stronger error-correcting behavior (Hillar et al., 2014, Fiete et al., 2014).

3. Learning rules and objective functions

The simplest Hopfield learning rule is one-shot Hebbian storage, but later work reformulates memory learning as optimization of explicit objectives. A recent study characterizes learning rules as descent-type algorithms for various cost functions, proposes several new cost functions suitable for learning, discusses the role of biases in the learning process, applies Newtons method for learning memories, experimentally compares various learning rules, and numerically investigates the effects of self coupling in relation to the debate over whether allowing connections of a neuron to itself enhances memory capacity (Tolmachev et al., 2020).

A particularly explicit objective-based formulation is probability-flow learning. Given a training set $W_{ij}=W_{ji}$4 and the Hamming-1 neighborhood $W_{ij}=W_{ji}$5, the empirical probability-flow loss is

$W_{ij}=W_{ji}$6

This loss is convex in $W_{ij}=W_{ji}$7 and can be minimized by first-order methods or L-BFGS. In the clique-storage construction, the resulting networks robustly store exponentially many memories in the sense described above (Hillar et al., 2014).

Predictive coding provides another route to local learning in recurrent Hopfield topologies. In that construction, each neuron is split into a value node $W_{ij}=W_{ji}$8 and an error node $W_{ij}=W_{ji}$9, with local prediction error

$X_1,\dots,X_N \in \{-1,+1\}^M$0

Once inference has settled, the synaptic updates are

$X_1,\dots,X_N \in \{-1,+1\}^M$1

Each update depends only on pre- and post-synaptic quantities, and the fixed points satisfy the same condition as in the classical Hopfield network. In this sense, predictive coding applies directly to a recurrent Hopfield topology without unrolling it in time, while preserving classical qualitative behavior (Ganjidoost et al., 2024).

These developments also clarify a frequent misconception: the Hopfield framework is not restricted to a single Hebbian formula. The invariant structure is the attractor-memory interpretation with symmetric recurrent dynamics; the learning rule can be Hebbian, convex-objective-based, Newton-type, or predictive-coding-local, provided the learned parameters create the intended energy landscape (Tolmachev et al., 2020, Hillar et al., 2014, Ganjidoost et al., 2024).

4. Continuous and system-theoretic formulations

The continuous Hopfield network replaces binary states by membrane potentials and output voltages linked by a monotone activation function. In one standard form,

$X_1,\dots,X_N \in \{-1,+1\}^M$2

where $X_1,\dots,X_N \in \{-1,+1\}^M$3 is the capacitance matrix, $X_1,\dots,X_N \in \{-1,+1\}^M$4 the leakage-resistance matrix, $X_1,\dots,X_N \in \{-1,+1\}^M$5 the synaptic matrix, and $X_1,\dots,X_N \in \{-1,+1\}^M$6 the external current (Schaft, 6 Jan 2026).

In this setting, Hopfield’s conventional “energy” is

$X_1,\dots,X_N \in \{-1,+1\}^M$7

A recent system-theoretic treatment shows that continuous Hopfield networks admit a port-Hamiltonian formulation provided the extra passivity condition

$X_1,\dots,X_N \in \{-1,+1\}^M$8

holds for all $X_1,\dots,X_N \in \{-1,+1\}^M$9 with $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$0. The same analysis shows that any Hopfield network can be represented as a gradient system, with Riemannian metric given by the inverse of the Hessian matrix of the total energy stored in the nonlinear capacitors, while Hopfield’s usual “energy” is reinterpreted as the dissipation potential. For constant input, this dissipation potential is non-increasing along trajectories, recovering convergence to critical points that constitute the network’s memories (Schaft, 6 Jan 2026).

This system-theoretic reinterpretation separates two notions that are often conflated in informal accounts: stored physical energy and Lyapunov-like descent potential. In the continuous model, the true stored energy is the Hamiltonian $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$1, whereas the function traditionally called the Hopfield energy is the quantity that certifies relaxation (Schaft, 6 Jan 2026).

Continuous generalizations also connect Hopfield networks to modern attention mechanisms. In the simplicial continuous variant, the energy is written in a softmax or log-sum-exp form,

$$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$2

with update

$$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$3

When $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$4 is the $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$5-skeleton and $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$6, this update recovers the Transformer attention update (Burns et al., 2023).

5. Higher-order and generalized memory landscapes

Several extensions alter what is stored, how it is represented, or which interactions define the attractor landscape. One such model stores each memory using two complementary encodings: a sparse, decorrelated pattern and a dense, correlated pattern. The weight matrix is a Hebbian sum over the linear combination of these two encodings. Retrieval then depends on the activity threshold: low threshold retrieves dense patterns, whereas high threshold retrieves sparse patterns. As more memories are stored, dense-example retrieval can shift toward dense-concept retrieval, while sparse retrieval preserves discrimination among examples. The same network can therefore retrieve memories at both example and concept scales and perform heteroassociation between them (Kang et al., 2023).

Higher-order interaction models generalize pairwise couplings to setwise couplings indexed by a simplicial complex. For a simplicial complex $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$7 on $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$8 neurons and simplex product $$v_i \leftarrow \operatorname{sign}\!\left(\sum_{j \neq i} W_{ij} v_j + b_i\right),$$9, the traditional simplicial energy is

$E(x)= -\tfrac12 x^T W x + b^T x,$0

The classical pairwise Hopfield network is recovered when $E(x)= -\tfrac12 x^T W x + b^T x,$1 is a $E(x)= -\tfrac12 x^T W x + b^T x,$2-skeleton. In a fully connected mixed-degree $E(x)= -\tfrac12 x^T W x + b^T x,$3-skeleton, the leading-order capacity is proportional to the number of nonzero connections,

$E(x)= -\tfrac12 x^T W x + b^T x,$4

with small retrieval errors, or

$E(x)= -\tfrac12 x^T W x + b^T x,$5

for error-free retrieval. Mixed diluted simplicial topologies can outperform purely pairwise networks even under an equal parameter budget (Burns et al., 2023).

A different generalization comes from the relativistic Hopfield model, whose Hamiltonian is

$E(x)= -\tfrac12 x^T W x + b^T x,$6

Its Taylor expansion yields a $E(x)= -\tfrac12 x^T W x + b^T x,$7-spin series with alternating signs: higher-odd terms are attractive and higher-even terms are repulsive. In the low-storage regime, the model retains the same critical point $E(x)= -\tfrac12 x^T W x + b^T x,$8 as the classical Hopfield model, while the added many-body terms deepen memory basins and destabilize spurious mixtures (Agliari et al., 2018).

These generalized models show that the Hopfield idea is compatible with markedly different representational regimes: sparse versus dense coding, pairwise versus setwise interactions, and quadratic versus many-body energy functions. A plausible implication is that “Hopfield network” designates a broader attractor-memory paradigm rather than a single fixed architecture (Kang et al., 2023, Burns et al., 2023, Agliari et al., 2018).

6. Quantum and hardware realizations

Quantum generalizations modify both representation and dynamics. In one open-quantum formulation, the classical asynchronous dynamics are embedded in a Lindblad master equation with coherent drive

$E(x)= -\tfrac12 x^T W x + b^T x,$9

and dissipative spin-flip operators chosen to satisfy detailed balance with respect to the Hopfield energy. The mean-field stationary equation coincides with the classical self-consistency relation after introducing an effective temperature

$$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$0

For small $$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$1, the model exhibits static retrieval analogous to the classical case; for sufficiently large $$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$2 at low temperature, it develops a limit-cycle phase in which periodic attractors replace static memory points. Static capacity is reduced by the factor $$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$3 for fixed $$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$4, while the capacity properties of the limit-cycle phase remain open (Rotondo et al., 2017).

A separate quantum line encodes the Hopfield weight matrix into amplitudes of quantum states. A $$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$5-neuron classical network is represented using $$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$6 qubits via

$$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$7

and the Hebbian memory matrix is stored in the density operator

$$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$8

Recall is formulated as a constrained linear system and solved with HHL-style routines, giving runtime

$$E(v)= -\tfrac12 \sum_{i\neq j} v_i W_{ij} v_j - \sum_i b_i v_i.$$9

This formulation yields logarithmic scaling in the data dimension, subject to the oracle and state-preparation assumptions of the algorithmic model (Rebentrost et al., 2017).

Physical implementations pursue Hopfield dynamics directly in matter. In a ferromagnetic thin film with electrodes, binary neuron states are represented by voltages $E$0, inter-electrode conductances determined by a magnetic domain texture play the role of synaptic weights, and training occurs through intrinsic feedback between current flow and magnetization. In a simulated 4-node network, training time per pattern is reported as $E$1–$E$2, retrieval convergence as $E$3–$E$4, and retrieval accuracy as $E$5 for single-bit or two-bit errors (Yu et al., 2021).

Hopfield-like optimization hardware has also been developed with memristive synapses. A three-terminal SONOS implementation uses device nonidealities, current leakage, and continuous conductance tuning to realize simulated annealing in a Hopfield network. For a $E$6 Max-Cut instance, the projected total is approximately $E$7 per cycle and $E$8 to solution at the reported optimum, with a projected system-level advantage of roughly $E$9 in speed and energy efficiency relative to the best CPU/GPU implementations cited there (Yi et al., 2021). A more recent hardware-aware Hopfield network uses the nonlinear current-voltage characteristics of a charge-trap memristor to reshape the energy landscape while preserving Hopfield-type energy minimization. Using a $E(s)= -\frac12 s^TWs,$0 array, it reports reliable reconstruction of corrupted patterns with empirical capacity scaling

$E(s)= -\frac12 s^TWs,$1

robustness above $E(s)= -\frac12 s^TWs,$2 even at $E(s)= -\frac12 s^TWs,$3 conductance noise, and recovery of $E(s)= -\frac12 s^TWs,$4 alphabet patterns with up to $E(s)= -\frac12 s^TWs,$5 pixel masking within $E(s)= -\frac12 s^TWs,$6–$E(s)= -\frac12 s^TWs,$7 iterations (Lee et al., 8 May 2026).

These quantum and hardware studies preserve the central Hopfield theme—memory or optimization as relaxation in an energy landscape—while shifting the substrate from abstract symmetric matrices to open quantum systems, amplitude-encoded quantum states, magnetic textures, and nonlinear memristive crossbars.