VaRONet: Neural Operator for Hamiltonian Systems

• VaRONet is a neural operator architecture that uses a variational LSTM seq-to-seq framework to predict complete phase-space trajectories from discretized potential functions.
• The model bypasses traditional step-based integration, reducing cumulative error and achieving accuracy competitive with RK4, TraONet, and MambONet across varied potentials.
• Its design leverages latent variable reparameterization and dual LSTM decoders to capture complex dynamics, ensuring robust performance even on nearly discontinuous potentials.

VaRONet is a neural operator architecture introduced within the Neural Hamilton framework to directly learn the mapping from a potential function $V(q)$ to Hamiltonian system trajectories, without explicitly solving Hamilton’s equations via numerical time integration. It adapts a variational LSTM sequence-to-sequence ("seq-to-seq") model to the operator-learning setting, enabling the prediction of entire phase-space trajectories $(q(t), p(t))$ for a given discretized potential. Designed to prevent the cumulative error propagation typical of step-based integrators, VaRONet demonstrates state-of-the-art accuracy competitive with the fourth-order Runge–Kutta (RK4) algorithm and advanced neural operators such as MambONet and TraONet, across a diverse range of physical systems and potential classes (Kim et al., 2024).

1. Operator-Learning Formulation

VaRONet is formulated within an operator-learning perspective for classical Hamiltonian systems. For an $m$-degree-of-freedom system governed by

$$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$

the operator of interest $\mathcal{F}$ maps a potential $V : [0, 1] \to \mathbb{R}$ (assumed $C^2$-smooth) to a trajectory $x(t) = (q(t), p(t))$ over $t \in [0, T]$:

$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$

Instead of learning a timestep-based generator or an equation solver, VaRONet directly learns a parameterized neural surrogate $(q(t), p(t))$0 that approximates $(q(t), p(t))$1, predicting the full trajectory at once for arbitrary potentials.

2. Model Architecture

VaRONet adapts the Variational Autoencoder with LSTM decoder (VAE-LSTM) scheme to the operator-learning paradigm. The architecture comprises several key stages:

1. Encoder-Branch (Input Discretization):
   • The potential $(q(t), p(t))$2 is sampled on $(q(t), p(t))$3 equally spaced points in $(q(t), p(t))$4, forming a vector $(q(t), p(t))$5.
   • A feedforward network $(q(t), p(t))$6 with four hidden layers of 512 units each processes $(q(t), p(t))$7 into a feature vector $(q(t), p(t))$8.
2. Variational Bottleneck:
   • Two subnetworks $(q(t), p(t))$9 produce mean and log-variance:

   $m$0

• The latent variable is sampled with reparameterization:

  $m$1

1. Latent-to-Decoder State Initialization:

   • Two MLPs $m$2 map $m$3 to initial hidden states:

   $m$4

2. Dual LSTM Decoders ("Trunk"):

   • Time points $m$5, matching the sampling of $m$6, are used as input sequences.
   • Two separate 4-layer LSTMs (512 hidden units each) are initialized with $m$7 and output sequences $m$8.
3. Output Projection:
   • Linear decoder heads project each LSTM output state to the scalar position $m$9 and momentum $$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$0 at time $$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$1.

The data flow can be summarized as: $(q(t), p(t))$03

3. Mathematical Foundations and Training Objective

The neural operator $$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$2 is optimized to approximate the mapping

$$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$3

with the following components:

• Encoder/Latent:

$$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$4

$$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$5

• Dual Decoder Recursion:

$$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$6

$$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$7

• Loss Function:

$$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$8

where $$\dot q_i = \frac{\partial H}{\partial p_i}, \quad \dot p_i = -\frac{\partial H}{\partial q_i}, \quad H(q, p) = \sum_{i=1}^m \frac{p_i^2}{2 m_i} + V(q),$$9 for best results on the standard dataset.

Notably, VaRONet does not implement explicit Hamiltonian or energy-conservation constraints; physicality emerges through empirical correspondence on large training sets.

4. Training Protocol and Data Generation

Potential Generation:

• Potentials are defined on $\mathcal{F}$0, baseline $\mathcal{F}$1.
• The number of Gaussian random field (GRF) control points, $\mathcal{F}$2, is sampled per potential.
• Each GRF uses a squared-exponential kernel with length scale $\mathcal{F}$3, values are normalized to $\mathcal{F}$4.
• Points are interpolated with a clamped cubic B-spline (degree 3, endpoints $\mathcal{F}$5) to guarantee $\mathcal{F}$6 smoothness.

Trajectory Labeling:

• Trajectories are generated by numerically solving $\mathcal{F}$7 with fine-step ODE integrators (e.g., RK4).
• Output is resampled via cubic Hermite spline at $\mathcal{F}$8 uniform time points.

Dataset Splits:

• "Standard": 10,000 potentials; "Extended": 100,000 potentials; 80/20 train/validation, with a 4,000-instance held-out test set.

Optimization:

• AdamW optimizer, $\mathcal{F}$9, batch size 100.
• Learning rate: ExpHyperbolicLR (warmup/slow decay), initial $V : [0, 1] \to \mathbb{R}$0, infimum $V : [0, 1] \to \mathbb{R}$1.
• Hyperparameters: LSTM decoder 4 layers × 512 units, encoder 4 layers × 512 units, latent dimension $V : [0, 1] \to \mathbb{R}$2, $V : [0, 1] \to \mathbb{R}$3, 250 epochs, metrics averaged over 5 seeds.

5. Quantitative Evaluation and Comparative Results

Test-Set Accuracy (MSE on 100 time-points):

Model	Standard ($V : [0, 1] \to \mathbb{R}$4)	Extended ($V : [0, 1] \to \mathbb{R}$5)
RK4	$V : [0, 1] \to \mathbb{R}$6	$V : [0, 1] \to \mathbb{R}$7
DeepONet	$V : [0, 1] \to \mathbb{R}$8	$V : [0, 1] \to \mathbb{R}$9
TraONet	$C^2$0	$C^2$1
VaRONet	$C^2$2	$C^2$3
MambONet	$C^2$4	$C^2$5

Computational Efficiency (single-threaded, per potential):

Model	Time (s)
RK4	$C^2$6
DeepONet	$C^2$7
TraONet	$C^2$8
VaRONet	$C^2$9
MambONet	$x(t) = (q(t), p(t))$0

Physically Relevant Potentials ($x(t) = (q(t), p(t))$1 with $x(t) = (q(t), p(t))$2 training):

Potential	RK4	DeepONet	TraONet	VaRONet	MambONet
SHO	$x(t) = (q(t), p(t))$3	$x(t) = (q(t), p(t))$4	$x(t) = (q(t), p(t))$5	$x(t) = (q(t), p(t))$6	$x(t) = (q(t), p(t))$7
Double-well	$x(t) = (q(t), p(t))$8	$x(t) = (q(t), p(t))$9	$t \in [0, T]$0	$t \in [0, T]$1	$t \in [0, T]$2
Morse	$t \in [0, T]$3	$t \in [0, T]$4	$t \in [0, T]$5	$t \in [0, T]$6	$t \in [0, T]$7
MFF (C⁰)	$t \in [0, T]$8	$t \in [0, T]$9	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$0	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$1	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$2
SMFF (≈C²)	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$3	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$4	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$5	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$6	$$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$7

Ablation / Sensitivity:

• Increasing dataset size improves accuracy: TraONet reaches $$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$8 loss at $$\mathcal{F}: V(\cdot) \longmapsto (q(t), p(t))_{t \in [0, T]}.$$9 potentials, rivaling MambONet on $(q(t), p(t))$00 potentials.
• VaRONet exhibits higher variance in loss, especially for non-smooth potentials, relative to TraONet or MambONet.

6. Error Propagation and Physical Consistency

VaRONet diverges from traditional integrators by predicting all timepoints in parallel, removing stepwise dependency and eliminating the primary pathway for error accumulation. In contrast to the iterative nature of RK4 and similar algorithms, which propagate local errors forward in time causing drift, VaRONet conditions its prediction solely on the input potential and sampled latent, so a local mistake at time $(q(t), p(t))$01 does not corrupt future steps. This architecture leads to notably lower cumulative trajectory error and robust performance, even on nearly discontinuous potentials. VaRONet does not impose explicit energy or Hamiltonian constraints; rather, empirical conservation emerges through successful operator learning over large datasets.

7. Significance and Position Within the Field

VaRONet represents a significant advancement in neural operator learning for dynamical systems governed by Hamiltonian mechanics. Its variational-LSTM sequence-to-sequence construction enables it to capture variability in physical responses and generalize to out-of-distribution potentials. While not as fast as fully feedforward alternatives (e.g., TraONet), nor matching the hybrid Mamba operator (MambONet) in all metrics, VaRONet occupies a distinct position by delivering near-RK4 accuracy and high expressivity with only $(q(t), p(t))$02 training instances. Detailed implementation resources, including data generation and reproducible training recipes, are made available in the corresponding code repository (Kim et al., 2024).