---
title: 'EqProp: Local Learning for Energy Models'
url: https://www.emergentmind.com/topics/equilibrium-propagation-eqprop
type: topic
---

# EqProp: Local Learning for Energy Models

Equilibrium Propagation (EqProp) is a learning framework for energy-based models that enables local, hardware-friendly computation of gradients for training neural and physical systems. By formulating learning as the response of physical networks to a small “nudging” perturbation, EqProp bridges the gap between backpropagation and biologically or physically plausible learning rules. EqProp has been extended to recurrent, convolutional, spiking, oscillator, Lagrangian, quantum, and analog hardware systems, with demonstrated scalability to deep architectures and robustness to hardware nonidealities.

## 1. Core Principles of Equilibrium Propagation

A system described by state variables $s$ (e.g. neuron activations), fixed inputs $x$, and parameters $\theta$ is assigned an energy $E(\theta, x, s)$. EqProp proceeds in two main phases:  
- **Free phase:** The input is clamped and the system relaxes to an energy minimum $s^0 = \arg\min_s E(\theta, x, s)$, where the output prediction is read.  
- **Nudged (weakly-clamped) phase:** A small scalar $\beta$ is introduced, coupling a supervised cost $C(s, y)$ to the energy: $E_\beta = E + \beta C$. The system is nudged to a nearby equilibrium $s^\beta = \arg\min_s (E(\theta, x, s) + \beta C(s, y))$.

The key theoretical result is that the gradient of the loss $J(\theta)=C(s^0, y)$ with respect to parameters is given by  
$$
\nabla_\theta J = \lim_{\beta\to0} \frac{1}{\beta}\left[ \frac{\partial E}{\partial \theta}(\theta, x, s^\beta) - \frac{\partial E}{\partial \theta}(\theta, x, s^0) \right]
$$
This can be evaluated by measuring local observables at the equilibrium points in both phases, yielding a learning rule that does not require explicit backpropagation through the network [1602.05179, 2503.22810, 1711.08416].

## 2. Mathematical Structure and Generalizations

The EqProp gradient derivation exploits the symmetry of mixed partial derivatives:
$$
\left.\frac{d}{d\beta}\right|_{\beta=0} \frac{\partial}{\partial\theta} E_\beta(\theta,x,s^\beta) = -\frac{\partial J}{\partial\theta}
$$
This forms the basis for the Hebbian-like local learning rule:
$$
\Delta W_{ij} = \frac{\eta}{\beta}\left[ \rho(s_i^\beta)\rho(s_j^\beta) - \rho(s_i^0)\rho(s_j^0) \right]
$$
where $\rho$ is a pointwise nonlinearity. This estimate is unbiased up to $O(\beta)$. Centered (symmetric) difference estimators with nudges at $+\beta$ and $-\beta$ further reduce bias to $O(\beta^2)$ [2101.05536].

EqProp's gradient equivalence to backpropagation is exact in the limit of small nudging ($\beta\rightarrow 0$) and for smooth energy functions with unique equilibria. For recurrent and partially asymmetric systems, EqProp matches the gradient of Backpropagation Through Time (BPTT) under infinitesimal nudging and vanishingly slow learning rates [2005.04169].

Generalizations of EqProp address:
- **Directed/Asymmetric architectures:** Via modified neuron dynamics and Oja-like plasticity [2006.08798, 2005.04169].
- **Spiking neural networks:** By mapping time-averaged spike rates and local eligibility traces to the EqProp learning rule [2010.07859, 2405.02546].
- **Sequence models:** Through convergent Hopfield or RNN layers with attention-like mechanisms [2209.09626].
- **Thermal and Quantum systems:** Extending the framework to finite-temperature Boltzmann equilibrium and quantum ground states [2405.08467, 2406.00879].

## 3. Extensions to Physical and Analog Neural Systems

EqProp is uniquely suited to in-situ learning in physical and analog substrates. Its two-phase protocol relies only on physical observables (local voltages, currents, or occupation numbers) during system relaxation.

- **Analog resistive and memristor networks:** EqProp computes the update of each conductance $g_{ij}$ via the squared difference of voltage drops across a resistor in the two phases, scaled by the nudging factor [2006.01981, 2512.12428]. Convergence is robust to strong nonlinearity in device physics, provided the programmable conductance ranges over at least an order of magnitude [2512.12428].
- **Oscillator and Ising machines:** EqProp adapts to phase or binary-spin variables. Weight gradients are obtained by measuring correlation differences of spin or oscillator observables before/after nudging. Experimental demonstrations on D-Wave quantum annealers and oscillator arrays achieve state-of-the-art accuracy on MNIST-scale tasks [2305.18321, 2504.11884].
- **Lagrangian, dynamical, or periodic systems:** The principle of EqProp translates to trajectories optimizing an action. The gradient with respect to parameters is given by the difference in the conjugate momenta (derivatives of the Lagrangian) integrated over the nudged and free trajectories [2505.07363, 2506.20402].

## 4. Algorithmic Variants and Scalability

Several algorithmic advancements have addressed the computational challenges and scalability of EqProp:

- **Discrete-time and continual updates:** Discrete EqProp simplifies the state updates for compatibility with modern hardware (e.g., CNNs) and enables “continual EqProp,” where synaptic changes occur in real-time using only locally available instantaneous states [2005.04169].
- **Bias correction:** Centered difference estimators remove leading-order bias in the EqProp gradient when using finite $\beta$ [2101.05536].
- **Deeper architectures:** Introduction of residual (skip) connections and clipped ReLU activations enables convergence and performance parity with backpropagation in networks up to ResNet13 depth, significantly expanding the range of tasks EqProp can solve [2509.26003].  
- **Sequence and attention models:** Integrating contractive Hopfield attention allows EqProp to handle complex NLP sequence tasks [2209.09626].
- **Oscillatory and holomorphic frameworks:** Holomorphic EqProp (hEP) replaces the two-phase protocol by encoding the loss gradient in the first Fourier mode of activity oscillations, supporting robust gradient estimation even for finite nudges and high noise [2209.00530].

## 5. Biological and Neuromorphic Significance

The EqProp learning rule is spatially and—through continual formulations—temporally local, requiring only pre- and post-synaptic signals. In contrast to standard backpropagation, EqProp does not require a dedicated backward pass or global memory of activations, aligning with the constraints of biologically plausible synaptic plasticity and neuromorphic implementation [1602.05179, 2010.07859]. In spike-based networks, the EqProp update rule emerges as a form of Spike-Timing Dependent Plasticity (STDP).

These features enable highly energy-efficient in-memory or event-driven learning on custom hardware such as memristors, analog crossbars, or spintronic/oscillator arrays. Empirical studies document several orders of magnitude energy savings for EqProp-based training and inference compared to GPU-based backpropagation, assuming sufficient device reliability [2010.07859, 2512.12428, 2006.01981].

Furthermore, EqProp naturally synergizes learning and synchronization in coupled-oscillator systems, providing a route to scalable, robust analog AI hardware [2504.11884].

## 6. Theoretical Connections and Limitations

EqProp unifies a spectrum of local learning principles, including:
- **Contrastive Hebbian Learning (CHL):** Recovers the CHL rule but with an infinitesimal, rather than fully clamped, output nudge for unbiased gradient estimation [1602.05179].
- **Recurrent Backpropagation:** EqProp’s transient neural activity in the nudged phase equals the error derivatives from RBP in the original backprop network, obviating the need for a separate “error circuit” [1711.08416, 2005.04169].  
- **Thermodynamic learning:** In stochastic or finite-temperature regimes, the EqProp update is a covariance between the observable and cost within the unclamped Boltzmann ensemble [2405.08467].

Key limitations include:
- Necessity for symmetric weights (though partially relaxed in recent asymmetric generalizations [2005.04169, 2006.08798]).
- Requirement for convergent dynamics and unique fixed points.
- Small nudging for unbiased gradients; finite $\beta$ can introduce bias, so robust estimation techniques are required.  
- For spiking and non-differentiable architectures, additional care in pooling/unpooling, state encoding, and gradient approximation is required [2405.02546].

## 7. Empirical Performance, Robustness, and Outlook

EqProp-trained systems match or approach the performance of backpropagation across domains:
- Dense, convolutional, and residual networks on MNIST and CIFAR-10/100 ($\leq$2–3% error gap for deep CNNs; parity in residual architectures) [2509.26003, 2101.05536].
- SNNs trained with EqProp versus BPTT achieve state-of-the-art accuracy and memory efficiency on MNIST/FashionMNIST [2405.02546].
- Robustness up to a critical noise threshold, with optimal learning observed at finite noise—corresponding to built-in regularization and hardware uncertainty tolerance [2503.22810].

Current research extends EqProp to:
- Quantum/finite-temperature and Lagrangian systems [2406.00879, 2405.08467, 2505.07363].
- Hardware demonstrations in analog, Ising, oscillator, and memristive devices [2006.01981, 2305.18321, 2512.12428, 2504.11884].
- Deep recurrent and sequence-processing networks [2209.09626].

These advances position EqProp as a leading biologically and physically plausible learning principle at the intersection of neuromorphic engineering, analog AI devices, and theoretical neuroscience.

Source: https://www.emergentmind.com/topics/equilibrium-propagation-eqprop