Hamiltonian Deep Neural Networks (HDNNs)

Updated 22 May 2026

Hamiltonian Deep Neural Networks (HDNNs) are deep learning frameworks that embed Hamiltonian dynamics, ensuring energy conservation and structural stability through symplectic methods.
They leverage canonical equations and symplectic integrators to enforce conservation laws and guarantee non-vanishing gradients for robust long-term predictions.
HDNNs incorporate techniques like parameter-cognizant channels and constraint handling, making them effective for simulating systems ranging from oscillators to chaotic and dissipative dynamics.

Hamiltonian Deep Neural Networks (HDNNs) are architectural and algorithmic frameworks for deep learning that encode the geometric, physical, and structural properties of Hamiltonian systems directly into network design, training objectives, or both. By leveraging the canonical and symplectic structure of Hamiltonian dynamics, HDNNs enable fundamentally stable, energy-conserving, and interpretable prediction of high-dimensional trajectories, surpassing the performance of generic neural networks particularly for systems dictated by conservation laws and constraints.

1. Mathematical Foundations and Canonical Principles

Hamiltonian systems are governed by a scalar function $H(q,p)$ (with $q,p\in\mathbb R^n$ ) and canonical equations $\dot q = \partial H/\partial p$ , $\dot p = -\partial H/\partial q$ , yielding a vector field $S_H(q,p) = (\partial_p H, -\partial_q H)$ that preserves both energy and phase-space volume. In HDNNs, the network is trained to parameterize $H(q,p)$ so that its symplectic gradient matches the observed or prescribed system dynamics, typically via an unsupervised loss that penalizes deviations from Hamilton's equations: $\mathcal L_{HNN} = \|\,\partial_p H_\theta(q,p) - \dot q\,\|^2 + \|\,\partial_q H_\theta(q,p) + \dot p\,\|^2$ These principles inherently enforce conservation of a learned Hamiltonian, induce exact time-reversibility under idealized conditions, and connect the network architecture to the symplectic geometry of underlying dynamics (Greydanus et al., 2019, Miller et al., 2020).

2. Network Architectures and Symplectic Discretization

HDNNs can be constructed in several forms:

Canonical HNN: A feed-forward network outputs a scalar $H_\theta(q,p)$ , with derivatives taken via automatic differentiation to yield time derivatives for supervised loss or trajectory rollout (Greydanus et al., 2019).
Equation-Driven HDNN: The trajectory $z(t)$ itself is parameterized by a network $N(t;\theta)$ , with outputs constrained to satisfy Hamilton's equations at collocation points, yielding closed-form, continuous surrogates (Mattheakis et al., 2020).
Symplectic Integrator-Embedded Nets: The network output (Hamiltonian or vector field) is propagated through a symplectic integrator (e.g., symplectic Euler, midpoint, Störmer-Verlet) directly within training, ensuring the learned map is a symplectomorphism. Integrator order and step size then become hyperparameters controlling accuracy and long-term energy conservation (Zhu et al., 2020, David et al., 2021).

Discretizing the Hamiltonian ODE with a symplectic scheme gives rise to network layers whose forward and backward sensitivity matrices are symplectic, guaranteeing non-vanishing gradients for arbitrary depth and preventing both vanishing and exploding gradients without the need for architectural hacks (Galimberti et al., 2021, Galimberti et al., 2021, Zakwan et al., 2023).

3. Extensions: Constraints, Dissipation, and Parametric Solutions

Constrained Systems

Hamilton-Dirac Neural Networks (HDNNs) generalize standard HNNs by embedding the Dirac theory of constraints and the Dirac bracket into the loss function: $q,p\in\mathbb R^n$ 0 where \begin{align*} \mathcal L_{\mathrm{eq}} & =\frac{1}{n_t n_\omega}\sum_{j,k} \big|\dot z_{\mathrm{net}}(t_j,\omega_k) - {z_{\mathrm{net}}(t_j,\omega_k), H_c}D\big|² \ \mathcal L{\mathrm{en}} & = \text{energy conservation penalty} \ \mathcal L_c & = \text{constraint violation penalty} \end{align*} This approach enforces both holonomic and non-standard constraints and yields predictions that remain on the constraint manifold and conserve energy to machine precision over long integrations, outperforming explicit solvers (Kaltsas, 2024).

Dissipative Dynamics

Dissipative HNNs (D-HNNs) augment the network by parameterizing a Rayleigh dissipation function $q,p\in\mathbb R^n$ 1, so that the learned vector field naturally Helmholtz-decomposes into conservative and dissipative components. This enables simultaneous learning of $q,p\in\mathbb R^n$ 2 and $q,p\in\mathbb R^n$ 3 for systems exhibiting friction, energy loss, or non-conservative effects, with proven interpretability and counterfactual accuracy for varying dissipation coefficients (Sosanya et al., 2022).

Parametric Hamiltonian Learning

HNNs with parameter channels can encode dependencies on system parameters or bifurcation variables (e.g., mass, eccentricity, friction). By including system-specific parameters as network inputs, the architecture becomes "parameter-cognizant," able to learn families of Hamiltonians and interpolate or extrapolate prediction to unseen parameter regimes (including bifurcation tracking and chaos detection) (Han et al., 2021, Kaltsas, 2024).

4. Universal Approximation, Training Stability, and Gradient Behavior

Rigorous universality results demonstrate that the time-discretized flow map of HDNNs is a universal approximator for continuous functions on compact domains, provided the discretization is symplectic and the activation non-polynomial and Lipschitz. Critically, semi-implicit Euler discretization (and related symplectic schemes) ensures that the backward sensitivity matrices remain symplectic: $q,p\in\mathbb R^n$ 4 for all layers $q,p\in\mathbb R^n$ 5, which mathematically guarantees non-vanishing gradients for arbitrary depth and boundedness of layerwise Jacobian products. This removes a central impediment to stable training in deep architectures and is preserved even in distributed or sparsity-constrained settings (Zakwan et al., 2023, Galimberti et al., 2021, Galimberti et al., 2021).

5. Symplectic and Structure-Preserving Neural Architectures

Recent HDNN variants explicitly constrain network outputs to be symplectomorphisms (i.e., canonical transformations). Architectures such as SymplectoNet (SpNN) are constructed using invertible coupling layers—q-shearing, p-shearing, and symplectic stretching—each parameterized by a neural network. Layer composition yields an explicit symplectomorphism, and the permutation-invariant nature of these layers ensures that, by design, the learned map preserves the symplectic form up to machine precision: $q,p\in\mathbb R^n$ 6 These networks require only standard regression losses and can approximate a broad class of unknown Hamiltonian flows and symplectic transformations, with invertibility and exact structure preservation (He et al., 2024).

6. Efficient and Non-Gradient Training Schemes

Novel training methodologies bypass standard backpropagation. By treating hidden-layer weights as fixed random features and solving only for the output layer in closed form via least squares, one achieves 100× speed-ups over gradient-based optimization, with orders-of-magnitude better accuracy in chaotic systems such as the Hénon–Heiles example. Data-driven sampling of hidden features (SWIM, A-SWIM) further concentrates model capacity in regions of phase space with steep Hamiltonian gradients. This approach preserves energy exactly and matches or exceeds gradient-trained models, provided pace scale and system dimension permit linear solves of the required size (Rahma et al., 2024).

7. Applications, Empirical Results, and Comparative Analysis

HDNNs have demonstrated superior performance in:

Long-term energy and constraint conservation, with $q,p\in\mathbb R^n$ 7 errors and constraint drifts orders of magnitude smaller than explicit solvers (e.g., RK45, LSODA) in constrained and unconstrained Hamiltonian systems (Kaltsas, 2024, Miller et al., 2020).
Accurate modeling of nonlinear oscillators (including quartic and Duffing systems), conservative and dissipative regimes, chaotic bifurcations, and real-world dynamical data such as planetary orbits, double pendulums, and ocean currents (Mattheakis et al., 2020, Miller et al., 2020, Sosanya et al., 2022).
Superior scalability in phase-space dimension, with error scaling favorably compared to conventional NNs, and power-law improvements in energy error with increasing sample size (Miller et al., 2020).
High accuracy in benchmark tasks for image classification and scientific data, with stable gradients in very deep networks (demonstrated up to 32–64 layers or on datasets such as MNIST and CIFAR-10) (Galimberti et al., 2021, Galimberti et al., 2021).
Parameter inference and semi-supervised learning, including the recovery of physical system parameters to $q,p\in\mathbb R^n$ 8 relative error purely from data (Kaltsas, 2024).

HDNNs support rigorous loss decomposition, post-training correction to physical Hamiltonians via backward error analysis, and tractable extension to graph, convolutional, or time-dependent inputs (David et al., 2021, Rahma et al., 2024).

Selected Comparative Table: Performance of Key HDNN Approaches

Approach	Symplectic Guarantee	Gradient Stability	Constraint Handling	Energy Drift	Efficiency
Canonical HNN (Greydanus et al., 2019)	Architectural	Inherited	Indirect	$q,p\in\mathbb R^n$ 9 baseline NN	Standard
Symplectic-integrator HDNN (Zhu et al., 2020, David et al., 2021)	Loss & layers	Guaranteed (symplectic BSM)	Indirect	Improved w/ order	Standard
Hamilton-Dirac NN (Kaltsas, 2024)	Loss (Dirac bracket)	Inherited	Direct	Machine precision	Expensive ( $\dot q = \partial H/\partial p$ 0min)
Fast sampling HNN (Rahma et al., 2024)	N/A (trajectory-based)	N/A	N/A	Exact (symplectic Euler)	$\dot q = \partial H/\partial p$ 1 standard
SymplectoNet (He et al., 2024)	Architectural	N/A	N/A	Exact (by construction)	Standard
D-HNN (Sosanya et al., 2022)	Architectural/Loss	Inherited	N/A	Dissipation modeled	Standard

References

(Greydanus et al., 2019) Hamiltonian Neural Networks
(Zhu et al., 2020) Deep Hamiltonian networks based on symplectic integrators
(Mattheakis et al., 2020) Hamiltonian neural networks for solving equations of motion
(Galimberti et al., 2021) Hamiltonian Deep Neural Networks Guaranteeing Non-vanishing Gradients by Design
(Galimberti et al., 2021) A unified framework for Hamiltonian deep neural networks
(Zakwan et al., 2023) Universal Approximation Property of Hamiltonian Deep Neural Networks
(Han et al., 2021) Adaptable Hamiltonian neural networks
(Kaltsas, 2024) Constrained Hamiltonian Systems and Physics-Informed Neural Networks: Hamilton-Dirac Neural Networks
(Rahma et al., 2024) Training Hamiltonian neural networks without backpropagation
(Sosanya et al., 2022) Dissipative Hamiltonian Neural Networks: Learning Dissipative and Conservative Dynamics Separately
(David et al., 2021) Symplectic Learning for Hamiltonian Neural Networks
(He et al., 2024) Deep Neural Networks with Symplectic Preservation Properties
(Miller et al., 2020) Mastering high-dimensional dynamics with Hamiltonian neural networks

HDNNs constitute a mathematically principled, physically interpretable, and empirically superior paradigm for learning and simulating dynamical systems with conservation laws and symplectic structure, enabling robust generalization, stable training at depth, and application to a wide class of scientific and engineering domains.