Learning Deterministic and Stochastic Forced Hamiltonian Systems

Abstract: We develop a geometric framework for learning deterministic and stochastic forced Hamiltonian systems with neural networks. Motivated by the Lagrange-d'Alembert principle and the theory of variational integrators, we introduce the notion of a Lagrange-d'Alembert map and establish a $C<sup>r$ convergence theorem for first-order one-step methods. Building on these results, we propose Generalized Forced Hamiltonian Neural Networks (GFHNNs), a class of structure-preserving neural networks obtained by concatenating Lagrange-d'Alembert-Euler maps, and prove a universal approximation theorem for this architecture. We further extend the framework to parameter-dependent systems, leading to Parametric Generalized Forced Hamiltonian Neural Networks (PGFHNNs). By interpreting the multiple Stratonovich integrals appearing in the Stratonovich-Taylor expansion as parameters, the same framework can be applied to stochastic forced Hamiltonian systems whenever information about the underlying Wiener process is available. Our numerical experiments demonstrate that the proposed geometric architectures provide significantly improved long-time stability and accuracy compared to non-geometric residual neural networks, while requiring substantially less training data to achieve a comparable level of performance.

• The paper introduces a geometric framework using Lagrange-d'Alembert maps for learning forced Hamiltonian systems with neural networks, focused on enhancing the convergence of discrete forces in a $C^r$ topology.
• GFHNNs leverages this framework efficiently, replacing symplectic blocks with explicit Lagrange-d'Alembert-Euler maps and achieving superior long-time behavior and data efficiency in numerical experiments compared to ResNets, specifically for damping and time-dependent oscillators.
• Parametric GFHNNs are extended to handle time-dependent and Stratonovich SDE stochastic systems, demonstrating applicability in broader contexts with accurate stochastic flow representations.

Geometric foundation: Lagrange-d'Alembert maps and convergence

The paper develops a geometric framework for learning forced Hamiltonian systems with neural networks, built on the Lagrange-d'Alembert principle rather than on symplecticity alone. Forced Hamiltonian flows on $T^*Q$ are not symplectic in general, but they satisfy a variational principle that yields type-I generating functions and exact discrete forces, so that the flow is encoded by $p_a = -D_1 S - f^-$ and $p_b = D_2 S + f^+$. The authors formalize this by introducing the notion of a Lagrange-d'Alembert map: any diffeomorphism of $T^*Q$ arising as the time-$T$ map of a time-dependent forced Hamiltonian system satisfying standing regularity assumptions ($H \in C^{r+2}$, hyperregularity of $H$, and global existence of the flow). The set of such maps is denoted $LdA^{r+1}(T^*Q)$. Notably, a Lagrange-d'Alembert map does not uniquely determine its generating system, since different Hamiltonian/force pairs can generate the same flow.

The central analytic tool is a $C^r$ convergence theorem for first-order one-step methods. Under a first-order consistency condition on all derivatives up to order $r$ and a stability bound $p_a = -D_1 S - f^-$0, the method converges to the exact flow in the strong $p_a = -D_1 S - f^-$1 topology on compact sets at rate $p_a = -D_1 S - f^-$2. The proof proceeds via discrete Grönwall estimates applied recursively to derivative errors using the Faà di Bruno formula. Both the Lagrange-d'Alembert-Euler scheme (a forcing-extended symplectic Euler) and the Lie-Trotter splitting $p_a = -D_1 S - f^-$3 are verified to satisfy these assumptions for small $p_a = -D_1 S - f^-$4. This theorem is what licenses the later universal approximation statements, which require density in $p_a = -D_1 S - f^-$5, not merely pointwise convergence.

Generalized Forced Hamiltonian Neural Networks

The GFHNN architecture extends Generalized Hamiltonian Neural Networks (GHNNs), which unify SympNets and HénonNets as concatenations of symplectic integrator blocks with separable learned Hamiltonians. A GFHNN replaces each symplectic block with an explicit Lagrange-d'Alembert-Euler map $p_a = -D_1 S - f^-$6,

$p_a = -D_1 S - f^-$7

with $p_a = -D_1 S - f^-$8, $p_a = -D_1 S - f^-$9, and $p_b = D_2 S + f^+$0 represented by multilayer perceptrons. Training minimizes a mean-squared loss against observed one-step transitions $p_b = D_2 S + f^+$1.

The universal approximation result is established through a three-stage argument:

1. Any Lagrange-d'Alembert map can be approximated arbitrarily well in $p_b = D_2 S + f^+$2 by compositions of Lie-Trotter integrators, directly from the $p_b = D_2 S + f^+$3 convergence theorem.
2. Each Lie-Trotter step decomposes into a forcing update (itself an LDE map) and a symplectic Euler step; Turaev's polynomial approximation result for symplectic maps expresses the latter as compositions of Hénon-like maps, which factor exactly into four LDE blocks.
3. By Hornik's theorem, MLPs approximate the kinetic, potential, and forcing functions in $p_b = D_2 S + f^+$4/$p_b = D_2 S + f^+$5 uniformly, so $p_b = D_2 S + f^+$6 can be approximated by $p_b = D_2 S + f^+$7.

Combining these yields the main theorem: finite compositions of neural-network-parameterized LDE maps are dense in $p_b = D_2 S + f^+$8 in $p_b = D_2 S + f^+$9 for every compact $T^*Q$0. Because each block is explicit, trajectory generation requires only forward evaluations — no iterative nonlinear solves, unlike implicit schemes or SGFNNs.

Parametric extension

For systems depending on parameters $T^*Q$1, the paper defines augmented LDE maps acting trivially on $T^*Q$2 and proves the analogous universal approximation theorem for Parametric GFHNNs (PGFHNNs), extending Horn's PGHNN framework to forced systems. The parametric convergence argument reduces to the non-parametric case by augmenting the state with a constant parameter channel.

Two reformulations then bring broader classes of problems into scope:

Time-dependent systems: fixing $T^*Q$3, the map $T^*Q$4 of a time-dependent system becomes a parametric Lagrange-d'Alembert map with time $T^*Q$5 as the parameter, via the reparametrized Hamiltonian $T^*Q$6.

Stochastic forced Hamiltonian systems: Stratonovich SDEs of the form $T^*Q$7, with analogous momentum equations including both deterministic and noise-driven forcing, possess a stochastic variational principle generalizing Lagrange-d'Alembert. The key observation is that the stochastic midpoint method's flow satisfies $T^*Q$8, where $T^*Q$9 is a deterministic parametric Lagrange-d'Alembert map generated by a parameter-dependent discrete Lagrangian and forces. Hence PGFHNNs apply whenever Wiener process paths are known. For higher strong-order schemes, higher multiple Stratonovich integrals $T$0 enter the parameter vector; formally the exact stochastic flow is parametrized by the full infinite sequence of such integrals appearing in the Stratonovich–Taylor expansion.

Numerical results

Experiments (implemented in GeometricMachineLearning.jl, tanh activations, Adam optimizer) compare GFHNNs/PGFHNNs against ResNets with matched trainable parameter counts (~72–186).

Experiment	Architecture	Parameters	Data size	Result
Linearly damped oscillator ($T$1)	GFHNN vs ResNet	74 vs 72	400 ICs	GFHNN lower averaged trajectory/Hamiltonian error across all $T$2
Quadratically damped pendulum	GFHNN vs ResNet	comparable	400 ICs	ResNet fails to reproduce dissipation at $T$3
Time-dependent damped oscillator ($T$4, $T$5)	PGFHNN vs parametric ResNet	186 vs 180	625 vs 4761 ICs	ResNet needed a 7.6× larger dataset to match accuracy
Forced Kubo oscillator ($T$6)	PGFHNN vs parametric ResNet	186 vs 180	100 ICs × 10 paths	PGFHNN superior over 2401 test trajectories

Two findings deserve emphasis. First, the advantage grows with the ratio of simulation horizon to the system's characteristic timescale: for weakly damped cases ($T$7), damping occurs over intervals orders of magnitude longer than $T$8, and only the geometric model tracks it accurately over long times. Second, the data efficiency claim is concrete: in the time-dependent experiment, the ResNet required $T$9 versus the PGFHNN's 625 to reach comparable precision. The paper is candid that performance depends on initial condition — for one specific initial condition in the weakly damped case, the ResNet slightly outperformed the GFHNN — which motivates its averaged-error evaluation protocol over held-out test sets.

Limitations and open questions

Several restrictions are stated plainly. The configuration space is assumed to be a vector space; extension to manifolds and Lie groups remains open, with robotics cited as a target application. Hyperregularity of the Hamiltonian is imposed for tractability of the generating-function formulation; degenerate Hamiltonians would require verifying admissible boundary conditions and choosing an appropriate generating-function type. The global-existence assumption on flows guarantees compact containment needed for the universal approximation proof. In the stochastic setting, the framework requires knowledge of the underlying Wiener process realizations, which limits direct application to settings such as Monte Carlo surrogate modeling; combining the approach with autoencoder-based denoising to infer latent random variables when paths are unobservable is identified as an open direction. Finally, the theory covers only first-order building blocks; higher-order structure-preserving architectures are not treated.

Conclusion

This work supplies a mathematically grounded route from variational mechanics to machine learning: the $H \in C^{r+2}$0 convergence theorem for first-order one-step methods, combined with Turaev's symplectic approximation results and Hornik universality, yields provable density of explicit LDE-block networks in the space of Lagrange-d'Alembert maps, in both deterministic and parametric/stochastic settings. The empirical evidence supports the structural claim that geometry preservation translates into better long-time behavior and lower data requirements relative to non-geometric baselines, while leaving manifold extensions and unobserved-noise inference as clearly posed open problems.