---
title: Back-Propagating Phase Matching in Neural Networks
url: https://www.emergentmind.com/topics/back-propagating-phase-matching
type: topic
---

# Back-Propagating Phase Matching in Neural Networks

Searching arXiv for the named papers and closely related work to ground the article in current records.
Back-propagating phase-matching arises because the small “nudge” $\beta\,\partial C/\partial s$ on the outputs is carried backwards through the same physical neuronal dynamics $\mu_\theta$ that propagated inputs forwards, and synapses measure only the local phase difference $\Delta s_i$ against presynaptic rates $\rho(s_j)$. In fixed point recurrent networks, this mechanism is implemented by a two-phase learning procedure: a free phase that converges to a prediction fixed point, and a nudged phase that converges to a nearby fixed point corresponding to smaller prediction error. In the energy-based formulation of Equilibrium Propagation, the temporal derivatives of neural activities in the second phase are equal to the error derivatives computed iteratively by Recurrent Backpropagation; in the vector-field generalization, the same scheme relaxes the requirement of an energy function and yields a local update that approximates the true gradient at a precision directly related to the degree of symmetry of the feedforward and feedback weights [1711.08416, 1808.04873].

## 1. Two-phase recurrent dynamics

In the vector-field formulation, the network state is the vector of neuron voltages $s\in\mathbb{R}^n$, the parameters are the synaptic weights $\theta=(W_{ij})$, and, for fixed input $\mathbf{x}$ and influence parameter $\beta\ge 0$, the state evolves according to the augmented vector field
$$
\frac{ds}{dt}
=
\mu_\theta^\beta(\mathbf{x},\mathbf{y},s)
=
\mu_\theta(\mathbf{x},s)
-
\beta\,\frac{\partial C}{\partial s}(\mathbf{y},s),
$$
with
$$
\mu_{\theta,i}(\mathbf{x},s)=\sum_j W_{ij}\,\rho(s_j)-s_i,
\qquad
C(\mathbf{y},s)=\tfrac12\|s_0-\mathbf{y}\|^2.
$$
The free phase is the case $\beta=0$, for which the dynamics converges to a fixed point $s_\theta^0$ satisfying
$$
\mu_\theta(\mathbf{x},s_\theta^0)=0.
$$
The nudged, or weakly clamped, phase uses $\beta>0$ small, starts from the same $s_\theta^0$, and moves to a nearby fixed point $s_\theta^\beta$ satisfying
$$
\mu_\theta(\mathbf{x},s_\theta^\beta)
-
\beta\,\frac{\partial C}{\partial s}(\mathbf{y},s_\theta^\beta)
=0.
$$
In the energy-based specialization, the same construction is written through an energy $E(\theta,s)$ with leaky gradient dynamics $\tau\,ds/dt=-\partial E(\theta,s)/\partial s$, free fixed point $s^*=s_\theta^0$ defined by $\partial E(\theta,s^*)/\partial s=0$, and augmented energy $E^\beta(\theta,s)=E(\theta,s)+\beta\,C(\theta,s)$ [1711.08416, 1808.04873].

The significance of this architecture is that prediction and error correction are both expressed as equilibrium computations of one recurrent system. The second phase is not an external reverse-mode pass; it is a nearby relaxation trajectory induced by soft clamping toward the target. This work shows that it is not required to have a side network for the computation of error derivatives, and supports the hypothesis that, in biological neural networks, temporal derivatives of neural activities may code for error signals [1711.08416].

## 2. Local plasticity and the phase-difference update

Plasticity is governed by a Hebbian/STDP-compatible rule,
$$
dW_{ij}\propto \rho(s_j)\,ds_i
\qquad\Longrightarrow\qquad
d\theta\propto \frac{\partial \mu_\theta}{\partial\theta}(s)^\top\,ds.
$$
Integrating over the small change of state $s_\theta^0\to s_\theta^\beta$, dividing by $\beta$, and letting $\beta\to 0$, one obtains the phase-difference update vector
$$
\Delta\theta
\propto
\nu(\mathbf{x},\mathbf{y},\theta)
=
\frac{\partial\mu_\theta}{\partial\theta}\bigl(\mathbf{x},s_\theta^0\bigr)^\top
\left.\frac{\partial s_\theta^\beta}{\partial\beta}\right|_{\beta=0}.
$$
Because $\partial\mu/\partial W_{ij}=\rho(s_j)$, this recovers the STDP form $dW_{ij}\propto\rho(s_j)\,ds_i$ [1808.04873].

Operationally, the crucial quantity is the difference between the two phases rather than a separately represented error vector. In the pseudocode formulation, the local weight update is
$$
\Delta W_{ij}\leftarrow \eta\cdot \rho(s^0_j)\cdot (s^\beta_i-s^0_i)/\beta.
$$
The temporal difference of each neuron’s activity carries the back-propagated error into the afferent synapses, and only local information $\{\rho(s_j),\Delta s_i\}$ is used [1808.04873].

## 3. Relation to Recurrent Backpropagation

The energy-based theory formalizes the relation to Recurrent Backpropagation through a phase-matching result. Recurrent Backpropagation defines the error-derivatives process from the projected cost $L(s,t)=C(S^0(s,t))$. At the free fixed point $s^*=s_\theta^0$, one introduces
$$
\bar S(t)\equiv \frac{\partial L}{\partial s}(s^*,t),
\qquad
\bar\Theta(t)\equiv \frac{\partial L}{\partial\theta}(s^*,t),
$$
with initial conditions and dynamics
$$
\bar S(0)=\frac{\partial C}{\partial s}(s^*),\qquad
\bar\Theta(0)=\frac{\partial C}{\partial\theta}(s^*),
$$
$$
\frac{d\bar S}{dt}=-H\,\bar S,\qquad
\frac{d\bar\Theta}{dt}=
-\frac{\partial^2E}{\partial\theta\partial s}(s^*)\,\bar S,
$$
where $H=\partial^2E/\partial s^2(s^*)$. In the nudged phase, if $S^\beta(s^*,t)$ is the flow under the augmented dynamics and
$$
\tilde S(t)\equiv -\lim_{\beta\to 0}\frac{1}{\beta}\frac{\partial S^\beta(s^*,t)}{\partial t},
$$
then
$$
\tilde S(0)=\frac{\partial C}{\partial s}(s^*),
\qquad
\frac{d\tilde S}{dt}=-H\,\tilde S.
$$
By uniqueness of solutions of the linear ODE,
$$
\tilde S(t)=\bar S(t),
\qquad
\tilde\Theta(t)=\bar\Theta(t)
\qquad
\forall\,t\ge 0.
$$
At large $t$, $\tilde\Theta(t)\to \partial J/\partial\theta$, recovering the Equilibrium Propagation learning rule [1711.08416].

This equivalence is the precise content of “phase matching” in the energy-based setting. At every moment in the second phase, the temporal derivatives of the neural activities in Equilibrium Propagation are equal to the error derivatives computed iteratively by Recurrent Backpropagation in the side network. In particular, the temporal derivatives $\dot s(t)$ encode the layerwise back-propagated errors $\lambda(t)$ [1711.08416].

## 4. Exact gradients, approximate gradients, and symmetry

For the vector-field generalization, the true gradient of the objective $J(\theta)=C(\mathbf{y},s_\theta^0)$ is
$$
\frac{\partial J}{\partial\theta}
=
-\frac{\partial C}{\partial s}\bigl(\mathbf{y},s_\theta^0\bigr)
\Bigl[\frac{\partial\mu}{\partial s}(\mathbf{x},s_\theta^0)\Bigr]^{-1}
\frac{\partial\mu}{\partial\theta}(\mathbf{x},s_\theta^0),
$$
whereas the vector-field update is
$$
\nu(\mathbf{x},\mathbf{y},\theta)
=
\frac{\partial C}{\partial s}\bigl(\mathbf{y},s_\theta^0\bigr)
\Bigl[\frac{\partial\mu}{\partial s}(\mathbf{x},s_\theta^0)^{T}\Bigr]^{-1}
\frac{\partial\mu}{\partial\theta}(\mathbf{x},s_\theta^0).
$$
Hence the error in using $\nu$ instead of $-\nabla_\theta J$ is
$$
\nu+\frac{\partial J}{\partial\theta}
=
\frac{\partial C}{\partial s}
\left(
\left[\tfrac{\partial\mu}{\partial s}\right]^{-T}
-
\left[\tfrac{\partial\mu}{\partial s}\right]^{-1}
\right)
\frac{\partial\mu}{\partial\theta}.
$$
Its norm can be bounded through
$$
\|A^{-T}-A^{-1}\|
\le
\|A^{-1}\|^2\,\|A-A^T\|,
$$
so the approximation error scales with the degree of asymmetry
$$
\left\|
\frac{\partial\mu}{\partial s}(\mathbf{x},s_\theta^0)
-
\left[\frac{\partial\mu}{\partial s}(\mathbf{x},s_\theta^0)\right]^T
\right\|,
$$
and vanishes exactly when the Jacobian is symmetric, which is the energy-based case [1808.04873].

This yields a sharp comparison among related learning schemes. Classical backpropagation computes exact gradients by forward pass plus reverse-mode differentiation, requires a separate error network or back-pass carrying signed derivatives, and imposes exact weight-symmetry. Standard Equilibrium Propagation requires an energy function $E_\theta(s)$ so that $\mu=-\partial E/\partial s$, uses the two phases, and updates exactly follow the gradient of the energy-regularized objective, but still demands symmetric weights. The vector-field generalization allows asymmetric feed-forward and feedback weights, retains the two-phase scheme and STDP-compatible local plasticity, and computes only an approximation $\nu$ to $-\nabla J$; the quality of the approximation improves as the Jacobian of $\mu$ becomes more symmetric [1808.04873].

## 5. Algorithmic realization and biological interpretation

The algorithmic outline is explicitly two-phase. Given learning rates $\epsilon$ for dynamics and $\eta$ for weights, small clamping $\beta$, cost $C$, and activation $\rho$, the procedure repeats for each minibatch $\{x,y\}$ as follows: Phase 1 is free relaxation, with $s$ initialized randomly or at zero, input units clamped to $x$, and $\beta\leftarrow 0$ until the state converges to $s^0$. Phase 2 is weakly clamped, with the same input clamped, $\beta\leftarrow \beta_{\text{small}}$, and the state initialized at $s^0$ until it reaches $s^\beta$. The local weight update is then applied at the end of the second phase, synapse by synapse, using the phase-difference formula $\Delta W_{ij}\leftarrow \eta\cdot \rho(s^0_j)\cdot (s^\beta_i-s^0_i)/\beta$ [1808.04873].

The biological motivation is explicit. Two major reasons the biological plausibility of the backpropagation algorithm has long been doubted are that neurons would need to send two different types of signal in the forward and backward phases, and that pairs of neurons would need to communicate through symmetric bidirectional connections. The vector-field generalization addresses both issues: neurons perform leaky integration, synaptic weights are updated through a local mechanism, and the backward transport of error is realized by the same physical neuronal dynamics rather than by explicit weight transposition [1808.04873].

## 6. Terminological context across disciplines

The phrase “phase matching” has established meanings outside learning theory. In nonlinear optics, energy conservation requires
$$
\omega_3=\omega_1+\omega_2
\quad\text{or}\quad
\omega_i=2\omega_p-\omega_s,
$$
and momentum conservation requires
$$
\vec k_3=\vec k_1+\vec k_2
\quad\text{or, in FWM,}\quad
\vec k_i=2\vec k_p-\vec k_s.
$$
The mismatch is the wave-vector mismatch $\Delta k$, and in a zero-index medium with effective refractive index $n(\omega)\approx 0$, the momentum per photon $k(\omega)=n(\omega)\,\omega/c$ nearly vanishes, so substituting into $\Delta k_{fw}=2k_p-k_s-k_i$ or $\Delta k_{bw}=2k_p-k_s+k_i$ yields $\Delta k\approx 0$ for either forward or backward idler generation [2102.13074].

In quantum search, “phase matching” refers to the choice of oracle and diffusion phase changes in the fully generalized Grover iterate. There, classical Grover’s algorithm and phase matching remains to be optimal till the target probability gets close 1; however, as the probability of observation approaches 1, the optimal phase changes differ from $\pi$ and no longer observe phase matching [2605.13758].

This suggests that the neural usage of back-propagating phase matching is terminologically adjacent to, but conceptually distinct from, optical phase-matching and quantum-search phase selection. In the neural setting, the matching occurs between temporal derivatives in the nudged phase and the error derivatives computed iteratively by Recurrent Backpropagation, rather than between wave vectors or between oracle–diffusion phase rotations [1711.08416, 1808.04873].

Source: https://www.emergentmind.com/topics/back-propagating-phase-matching