Hamiltonian Echo Backpropagation

• Hamiltonian Echo Backpropagation is a gradient-recovery technique that exploits time reversibility and symplectic structure to transport sensitivity information in Hamiltonian systems.
• It is implemented in both physical self-learning machines and Hamiltonian neural networks, enabling forward-only learning and constant-memory reverse propagation.
• The method shows robust performance on chaotic systems and long-horizon predictions while offering efficient, structure-preserving parameter updates.

Hamiltonian Echo Backpropagation (HEBP), also appearing in the literature as Hamiltonian Echo Backpropagation, Hamiltonian Echo Learning (HEL), Recurrent Hamiltonian Echo Learning (RHEL), and in earlier work simply as Hamiltonian Echo Backpropagation or HEB, denotes a class of gradient-recovery procedures for Hamiltonian systems that use time reversibility, symplecticity, or both to transport sensitivity information through the same dynamics that generate the forward evolution. In physical self-learning machines, the method consists of a forward pass, a small error injection, time reversal, an echo pass, and a decay step that converts a momentum kick on internal degrees of freedom into a parameter update (Lopez-Pastor et al., 2021). In Hamiltonian neural networks, it denotes a structure-preserving training paradigm in which a symplectic discretization is used both for the forward flow and for the discrete adjoint, so that the backward pass reproduces reverse-mode gradients of the solver at finite step size while supporting constant-memory reverse propagation (Choudhary et al., 25 Jun 2026). Variational treatments later identified HEL and RHEL as reversible, forward-only instances of generalized Lagrangian equilibrium propagation and established equivalence to continuous adjoint methods or Backpropagation Through Time under the stated Hamiltonian assumptions (Pourcel et al., 6 Jun 2025, Pourcel et al., 5 Jun 2025).

1. Terminological scope and lines of development

The literature uses closely related names for procedures that share a common structural idea—extracting gradients from Hamiltonian evolution by exploiting reversibility or symplectic adjoint structure—but differ in substrate, mathematical setting, and implementation. The main strands are the autonomous physical-learning protocol of nonlinear Hamiltonian devices, the discrete-adjoint training of symplectic Hamiltonian neural networks, the variational HEL/RHEL formulation for reversible temporal learning, and a distinct Heisenberg-picture usage in quantum Hamiltonian simulation (Lopez-Pastor et al., 2021, Choudhary et al., 25 Jun 2026, Pourcel et al., 6 Jun 2025, Pourcel et al., 5 Jun 2025, Fuller et al., 4 Feb 2025).

Label	Setting	Defining feature
HEB	Self-learning Hamiltonian machines	Physical echo plus autonomous decay update
HEBP	Symplectic HNN training	Discrete adjoint of symplectic RK equals backprop
HEL	Reversible Hamiltonian learning	Forward-only finite differences of Hamiltonian derivatives
RHEL	Recurrent Hamiltonian systems	Three forward passes; equivalence to BPTT
HEBP in OBP usage	Quantum Hamiltonian simulation	Heisenberg-picture operator backpropagation

This terminological spread reflects genuine generalization rather than simple renaming. In the 2021 self-learning-machine formulation, the central object is a physical device with internal degrees of freedom that serve as learnable parameters and are updated autonomously, without external computation of gradients or direct manipulation of those parameters (Lopez-Pastor et al., 2021). In the 2026 symplectic-neural-network formulation, the central object is a learnable Hamiltonian $H_\theta$ embedded in an implicit symplectic solver, with the “echo” property identified at the level of the discrete adjoint and reverse-mode automatic differentiation (Choudhary et al., 25 Jun 2026). HEL and RHEL then place these ideas inside a broader Lagrangian and Hamiltonian variational framework and emphasize forward-only learning, locality, and exact gradient equivalence in reversible systems (Pourcel et al., 6 Jun 2025, Pourcel et al., 5 Jun 2025).

A recurrent source of confusion is that not every Hamiltonian or reversible training scheme is HEBP. A separate line of work on Hamiltonian neural networks replaces iterative gradient descent with sampled hidden-layer features and a convex least-squares solve, and explicitly states that it does not mention any “echo” or “echo backpropagation” mechanism (Rahma et al., 2024). Conversely, the quantum-computing usage applies “echo” to the Heisenberg adjoint evolution of observables and not to a hardware Loschmidt echo or an autonomous physical backpropagation protocol (Fuller et al., 4 Feb 2025).

2. Hamiltonian structure and the echo principle

In its most standard form, the learnable state is a canonical phase vector $z=(q,p)\in\mathbb{R}^{2d}$ with Hamiltonian $H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$ and canonical matrix

$$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$

so that the dynamics are

$\dot z(t)=J\,\nabla H_\theta(z(t)).$

This is the basic setup used in symplectic HNN training under noisy trajectory observations, where system identification is posed from observations $z_k^{obs}=Cz_k+\varepsilon_k$ with $\varepsilon_k\sim\mathcal{N}(0,\Sigma)$ and a loss of the form $$\mathcal{L}(\theta)=\sum_{k=0}^{K}\ell(z_k^{pred}(\theta),z_k^{obs})$$ (Choudhary et al., 25 Jun 2026).

The echo principle is the claim that backward sensitivity transport can be realized by the same Hamiltonian or symplectic structure that governs the forward flow. In the discrete HNN setting, the central observation is that for symplectic Runge–Kutta schemes the discrete adjoint obtained by integrating the adjoint system with the same symplectic scheme is algebraically identical to reverse-mode differentiation through the solver. The associated forward map $\Phi_h$ is canonical, its Jacobian $D\Phi_h$ is symplectic, and the cotangent lift propagates adjoints by $z=(q,p)\in\mathbb{R}^{2d}$0, exactly the operation performed by reverse-mode AD. For symmetric schemes such as implicit midpoint, time-reversal symmetry further allows reconstruction of forward states during the backward pass, which is the basis of the constant-memory “echo” computation (Choudhary et al., 25 Jun 2026).

In the physical self-learning formulation, the same principle appears in continuous Hamiltonian dynamics with complex fields. Time reversal is the canonical momentum inversion $z=(q,p)\in\mathbb{R}^{2d}$1, or in complex notation $z=(q,p)\in\mathbb{R}^{2d}$2 and $z=(q,p)\in\mathbb{R}^{2d}$3. After a forward evolution under the Hamiltonian, a weak error signal proportional to the output gradient is injected, and the time-reversed evolution sends an error field backward through the same nonlinear medium. The advanced Green’s function needed for sensitivity propagation is thereby realized through forward propagation of the time-reversed state under the same Hamiltonian dynamics (Lopez-Pastor et al., 2021).

RHEL extends this mechanism to recurrent Hamiltonian systems. There the forward trajectory is generated by a non-dissipative Hamiltonian flow, and two echo trajectories with $z=(q,p)\in\mathbb{R}^{2d}$4 symmetry-breaking perturbations are compared. Central finite differences of the resulting echo trajectories converge to the adjoint sensitivities, so that the echo gap itself becomes the gradient carrier (Pourcel et al., 5 Jun 2025).

3. Algorithmic realizations

The physical HEB protocol for self-learning machines is defined by a fixed sequence of operations. The machine evolves forward from input to output under its Hamiltonian, producing $z=(q,p)\in\mathbb{R}^{2d}$5. A cost-function Hamiltonian $z=(q,p)\in\mathbb{R}^{2d}$6 is then applied for a short duration $z=(q,p)\in\mathbb{R}^{2d}$7, giving the error injection

$z=(q,p)\in\mathbb{R}^{2d}$8

The system is phase-conjugated, propagated through the echo pass, and the parameter field undergoes a final conjugation and a decay step. To first order in $z=(q,p)\in\mathbb{R}^{2d}$9, the echo produces

$H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$0

and after the dissipative decay step the parameter update becomes

$H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$1

which realizes stochastic gradient descent without external gradient computation or direct parameter access (Lopez-Pastor et al., 2021).

In symplectic HNN training, the forward evolution is produced by an implicit symplectic integrator, most prominently the implicit midpoint scheme

$H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$2

The discrete adjoint for a terminal loss $H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$3 is initialized by $H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$4 and propagated backward by

$H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$5

Parameter gradients accumulate as

$H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$6

For the implicit midpoint residual, the implementation avoids forming an explicit inverse and instead solves transposed linear systems such as $H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$7, which is the standard adjoint trick for implicit layers and is consistent with the discrete-adjoint recursion (Choudhary et al., 25 Jun 2026).

The practical forward solve in that setting is a predictor–corrector with fixed-point iterations. The predictor is an explicit RK2 or extrapolation step, and the corrector iterates the midpoint map until the update norm falls below a tolerance or a fixed iteration cap is reached. The contraction condition $H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$8 for $H_\theta:\mathbb{R}^{2d}\to\mathbb{R}$9-Lipschitz $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$0 gives the basic convergence criterion for the midpoint fixed-point map (Choudhary et al., 25 Jun 2026).

RHEL modifies the echo construction for sequential models. It requires one free forward pass and two echo passes with perturbations $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$1 and $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$2. The method therefore uses three forward passes irrespective of model size, does not form explicit Jacobians, and in the $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$3 limit recovers exactly the continuous adjoint in continuous time and BPTT in the discrete-time Hamiltonian Recurrent Unit setting (Pourcel et al., 5 Jun 2025). In the GLEP formalization, the corresponding HEL estimator is a finite difference of Hamiltonian parameter derivatives between the free and echo phases, with a boundary correction term that vanishes when the initial state is $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$4-independent (Pourcel et al., 6 Jun 2025).

4. Computational, geometric, and numerical properties

The principal computational claim of the symplectic HNN formulation is that HEBP turns reverse propagation through an implicit symplectic solver from an $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$5-memory computation into an $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$6-memory computation with respect to the horizon length $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$7, up to optional sparse checkpoints. Standard AD through the implicit solver stores intermediate iterates for every time step and every fixed-point or Newton iteration; HEBP instead reconstructs forward states by echoing the symmetric symplectic map or recomputing them as needed. With $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$8 the cost of evaluating $$J=\begin{pmatrix} 0 & I\ -I & 0 \end{pmatrix},$$9 and Hessian-vector products, each forward step costs $\dot z(t)=J\,\nabla H_\theta(z(t)).$0, and the total per-epoch complexity is $\dot z(t)=J\,\nabla H_\theta(z(t)).$1 (Choudhary et al., 25 Jun 2026).

The reported runtime and memory study on the coupled harmonic oscillator shows constant memory for HEBP across increasing $\dot z(t)=J\,\nabla H_\theta(z(t)).$2, whereas AD backprop grows linearly. Wall-clock times favor HEBP at small–moderate $\dot z(t)=J\,\nabla H_\theta(z(t)).$3, and for very long horizons its $\dot z(t)=J\,\nabla H_\theta(z(t)).$4 memory profile is described as decisive (Choudhary et al., 25 Jun 2026). Practical heuristics reported for the implicit midpoint implementation are $\dot z(t)=J\,\nabla H_\theta(z(t)).$5, $\dot z(t)=J\,\nabla H_\theta(z(t)).$6, and tolerances in the range $\dot z(t)=J\,\nabla H_\theta(z(t)).$7–$\dot z(t)=J\,\nabla H_\theta(z(t)).$8, with step-halving or damped Newton fallback if the implicit solve stalls (Choudhary et al., 25 Jun 2026).

The geometric motivation is long-horizon fidelity. Symplectic integrators preserve the symplectic two-form and exactly conserve a modified Hamiltonian $\dot z(t)=J\,\nabla H_\theta(z(t)).$9 over exponentially long times in backward error analysis, yielding near-conservation of energy and phase-volume preservation. In the reported HEBP experiments, energy error exhibits bounded oscillations with no secular drift over long horizons, and implicit symplectic training yields better generalization on chaotic systems such as Hénon–Heiles and on long-horizon prediction than non-symplectic baselines (Choudhary et al., 25 Jun 2026). The same work reports that post-processing the learned Hamiltonian with the $z_k^{obs}=Cz_k+\varepsilon_k$0 backward-error correction improves agreement with the true Hamiltonian without reducing $z_k^{obs}=Cz_k+\varepsilon_k$1 or switching to higher-order schemes (Choudhary et al., 25 Jun 2026).

A closely related architectural result comes from Hamiltonian deep neural networks discretized by semi-implicit Euler. There, the one-layer sensitivity matrices are proved symplectic, the backward sensitivity matrix across arbitrary depth remains symplectic, and one obtains the lower bound $z_k^{obs}=Cz_k+\varepsilon_k$2 for any submultiplicative norm, which is presented as non-vanishing gradients by design. The same work also provides an exponential upper bound on sensitivity growth in continuous time and recommends regularization of $z_k^{obs}=Cz_k+\varepsilon_k$3 and $z_k^{obs}=Cz_k+\varepsilon_k$4 to control exploding gradients (Galimberti et al., 2021). Although this is not itself labeled HEBP, it formalizes why reversible Hamiltonian discretizations are attractive substrates for echo-style backpropagation.

5. Relation to adjoint methods, equilibrium propagation, and other “echo” notions

A central theoretical clarification is that HEBP is not merely a heuristic approximation to adjoint-state methods. In the symplectic HNN setting, the discrete adjoint of the symplectic solver is exactly equal to backpropagation through that solver at finite step size. This distinction matters because the continuous-time adjoint used in Neural ODEs computes gradients of the underlying ODE, not of the discretized solver, and may therefore differ from backprop unless $z_k^{obs}=Cz_k+\varepsilon_k$5 or a discrete-adjoint coincidence holds (Choudhary et al., 25 Jun 2026).

HEL and RHEL are also situated within the broader theory of equilibrium-propagation-like learning. Generalized Lagrangian Equilibrium Propagation extends the variational formulation of EP from fixed points to trajectories, but most boundary-condition choices either leave impractical endpoint residuals or require expensive two-point boundary-value solvers. The practical exception is the Parametric Final Value Problem for time-reversible Lagrangians; under the Legendre transform this becomes HEL, and the paper identifies HEL as the only instance of GLEP it found that is forward-only, scales efficiently with two or more passes regardless of model size, and enables local learning (Pourcel et al., 6 Jun 2025). In this framework, HEB is the static-input case and RHEL is the time-varying generalization.

RHEL strengthens the adjoint connection on sequential models by proving exact equivalence to the continuous adjoint state method in continuous time and to BPTT in discrete time for Hamiltonian Recurrent Units and Hamiltonian state-space models. The method’s distinguishing claim is that these exact gradients are recovered through finite differences of physical trajectories, using three forward passes and no explicit Jacobian computation (Pourcel et al., 5 Jun 2025).

The term “echo” is not uniform across all fields. In operator backpropagation for quantum Hamiltonian simulation, the observable is pushed backward through a classically simulated portion of a circuit in the Heisenberg picture, so the backward step is an adjoint evolution of operators rather than a forward–backward hardware sequence. The authors explicitly note that this is not a Loschmidt echo on hardware; in that usage, “Hamiltonian Echo Backpropagation” means Heisenberg-picture operator backpropagation specialized to Hamiltonian simulation (Fuller et al., 4 Feb 2025). This differs conceptually from the autonomous physical echo of self-learning Hamiltonian devices and from the discrete-adjoint symplectic echo of HNN solvers.

Another boundary of the term is set by alternative backpropagation-free HNN training. The sampled-feature method of “Training Hamiltonian neural networks without backpropagation” replaces reverse-mode differentiation with hidden-layer sampling and a convex least-squares solve, and the paper explicitly states that no echo mechanism is used (Rahma et al., 2024).

6. Applications, empirical demonstrations, and limitations

The modern symplectic-HNN formulation evaluates HEBP on a range of non-separable and chaotic systems, including the double well, coupled oscillator, Hénon–Heiles system, and the Kepler two-body problem in 2D. Training and validation trajectories are generated with high-order symplectic integrators at $z_k^{obs}=Cz_k+\varepsilon_k$6, with additive Gaussian noise per step, and the reported setup uses a small MLP for $z_k^{obs}=Cz_k+\varepsilon_k$7, implicit midpoint integration, HEBP in the backward pass, mini-batches of $z_k^{obs}=Cz_k+\varepsilon_k$8, Adam with learning rate $z_k^{obs}=Cz_k+\varepsilon_k$9, a ReduceLROnPlateau scheduler, and $\varepsilon_k\sim\mathcal{N}(0,\Sigma)$0 epochs. Evaluation includes Hamiltonian identification error under three phase-space sampling distributions, long-horizon energy preservation, and runtime/memory profiling against AD backprop. The reported outcome is improved Hamiltonian prediction accuracy on non-separable and chaotic systems, near-conserved energy, constant memory profiles, and competitive or better runtime at moderate horizons; backward-error-analysis post-processing further reduces Hamiltonian error (Choudhary et al., 25 Jun 2026).

The original physical HEB demonstration uses coupled nonlinear wave fields governed by nonlinear Schrödinger-type dynamics on a $\varepsilon_k\sim\mathcal{N}(0,\Sigma)$1 grid. Inputs are encoded as Gaussian wave packets, the output is read from two output points, and the system is trained on XOR. The reported behavior is rapid reduction in cost, emergence of a structured parameter-field configuration after a few hundred HEB steps, and graceful degradation under noisy time reversal that can be mitigated by smaller learning rates or a learning-rate schedule (Lopez-Pastor et al., 2021).

RHEL applies the echo principle to sequence modeling with Hamiltonian state-space models. Reported tasks range from mid-range to long-range classification and regression, with sequence length reaching $\varepsilon_k\sim\mathcal{N}(0,\Sigma)$2, and the paper states that RHEL consistently matches the performance of BPTT across all models and tasks considered (Pourcel et al., 5 Jun 2025). The intended significance is hardware-oriented: scalable, energy-efficient physical systems for sequence modeling that compute gradients through reversible dynamics rather than explicit reverse-mode differentiation.

A distinct stationary application arises in the diffusive FitzHugh–Nagumo model. There, the steady-state response operator is self-adjoint, which enables equilibrium-propagation-style credit assignment, while deep residual topologies admit a spatial Hamiltonian and a layer-wise Hamiltonian recurrence that provides a feedforward alternative to the steady-state boundary-value formulation. On MNIST, the reported six-layer FHN architecture reaches a test error of $\varepsilon_k\sim\mathcal{N}(0,\Sigma)$3, and the Hamiltonian spatial integration is reported to track the time-relaxed steady-state solution across depth up to $\varepsilon_k\sim\mathcal{N}(0,\Sigma)$4 layers before divergence on deeper networks (Kendall, 20 May 2026). This usage broadens HEBP from strictly time-reversible temporal dynamics to stationary spatial Hamiltonian structure.

The limitations stated across the literature are consistent. Exactness requires reversible Hamiltonian dynamics or reversible symplectic discretization; strong dissipation, irreversible processes, or inaccurate time reversal break the core equivalence (Lopez-Pastor et al., 2021, Pourcel et al., 5 Jun 2025). In the implicit-solver HNN setting, stiff dynamics can slow fixed-point convergence, high observation noise can require regularization or robust losses, and singular potentials such as Kepler need increased sampling resolution or rescaling (Choudhary et al., 25 Jun 2026). Large-scale physical implementations also face engineering constraints in phase conjugation, decay mechanisms, low-loss propagation, and synchronization of echo steps (Lopez-Pastor et al., 2021). These constraints explain why HEBP is best understood not as a single algorithm, but as a family of structure-preserving gradient-computation schemes whose practical success depends on how faithfully the underlying Hamiltonian reversibility can be realized.