---
title: 'Hopfield-Resnet: Energy-Based Deep Architectures'
url: https://www.emergentmind.com/topics/hopfield-resnet
type: topic
---

# Hopfield-Resnet: Energy-Based Deep Architectures

Searching arXiv for the target paper and closely related equilibrium propagation work.
Hopfield-Resnet is a deep, convolutional Hopfield network trained by equilibrium propagation (EP) in which residual (skip) connections and a clipped $\mathrm{ReLU}$ nonlinearity are used to scale EP to substantially deeper architectures [2509.26003]. It was introduced in "Scaling Equilibrium Propagation to Deeper Neural Network Architectures" [2509.26003] as an architectural response to a recurring limitation of EP: prior studies had been restricted to networks containing only dense layers or relatively small architectures with a few convolutional layers followed by a final dense layer, with a significant gap in accuracy relative to similarly sized feedforward networks trained with backpropagation. In this formulation, residual interactions are incorporated directly into the Hopfield energy, and the resulting model supports networks with nearly twice the number of layers reported in prior works; the flagship configuration, Hopfield-Resnet13, achieves $93.92\%$ accuracy on CIFAR-10, approximately $3.5\%$ higher than the previous best reported EP result and comparable to a Resnet13 trained with backpropagation [2509.26003].

## 1. Conceptual basis and motivation

Hopfield-Resnet is defined within the equilibrium propagation framework, where learning is performed by contrasting equilibria of an energy-based dynamical system rather than by standard reverse-mode differentiation [2509.26003]. The model is a layered Hopfield network with neuronal state variables $\{s^n\}_{n=0}^{N-1}$, where each $s^n$ is a tensor for convolutional layers or a vector for dense layers, and with parameters $\theta$ comprising kernels, matrices, and biases. The central architectural innovation is the integration of residual connections into this Hopfield formalism, so that skip edges contribute explicit pairwise interaction terms to the energy rather than being treated as an external feedforward modification.

The stated motivation is twofold. First, EP has attractive locality properties: parameter updates depend on locally available pre- and post-synaptic states at equilibrium, which is presented as well-suited for neuromorphic hardware [2509.26003]. Second, depth had been a practical obstacle for EP-trained convolutional networks. The introduction of residual pathways and clipped activations is therefore aimed at improving stability of the relaxation dynamics and enabling substantially deeper architectures under centered equilibrium propagation (CEP) [2509.26003].

A plausible implication is that Hopfield-Resnet should be understood not merely as a residual CNN trained with an alternative objective, but as a specific energy-based construction in which residual structure alters both the equilibrium states and the local learning signals.

## 2. Energy function, state dynamics, and fixed points

For a baseline convolution-pooling-dense stack, the energy is defined as a sum of pairwise interaction terms between adjacent layers:
$$
E(x, s; \theta) = \sum_{n=0}^{N_{\mathrm{conv}}-1} \langle s^{n+1}, \mathcal{P}( W_{n+1} \star s^n ) \rangle + \sum_{n=N_{\mathrm{conv}}}^{N_{\mathrm{tot}}-1} s^{n+1\top} W_{n+1} s^n + \text{bias/regularization terms},
$$
where $W_{n+1}$ are convolution kernels or dense matrices, $\star$ denotes convolution, $\mathcal{P}$ is the pooling operator, and $\langle \cdot,\cdot\rangle$ is the spatial/channelwise inner product [2509.26003]. In Hopfield-Resnet, residual edges are added through
$$
E_{\mathrm{residual}}(s; \theta) = \sum_{(i \to j)\in \mathrm{Skips}} \langle s^j, \mathcal{P}_{i \to j}( K_{i\to j} \star s^i ) \rangle,
$$
so that the total energy is $E_{\mathrm{total}} = E + E_{\mathrm{residual}}$ [2509.26003].

The free-phase dynamics minimize this energy by relaxing the states according to the gradient flow
$$
\dot{s} = -\frac{\partial E}{\partial s},
$$
with fixed points $s^0$ satisfying
$$
\frac{\partial E}{\partial s}(x, s^0; \theta)=0
$$
[2509.26003]. In discrete time, layerwise updates apply an activation $\phi$ to pre-activations. For convolutional layers without residuals, the bidirectional Hopfield update is
$$
s^{n}_{t+1} = \phi\!\left( \mathcal{P}( W_n \star s^{n-1}_t ) + \tilde{W}_{n+1} \star \mathcal{P}^{-1}( s^{n+1}_t ) \right),
$$
and for dense layers,
$$
s^{n}_{t+1} = \phi( W_n s^{n-1}_t + W_{n+1}^{\top} s^{n+1}_t )
$$

Source: https://www.emergentmind.com/topics/hopfield-resnet