Hamiltonian World Models: Structured Latent Dynamics

• Hamiltonian World Models are generative simulators that map observations into structured latent (q, p) states governed by scalar energy functions.
• They extend classical Hamiltonian mechanics by incorporating dissipation, control inputs, and residual corrections to model complex real-world dynamics.
• These models integrate encoder-decoder architectures, symplectic integrators, and planning modules to achieve energy-consistent, long-horizon predictions and efficient policy optimization.

Hamiltonian World Models are learned, action-conditioned generative simulators whose core latent state lives in a structured phase space $(q,p)$ and whose transition dynamics are induced by a scalar “Hamiltonian” energy function $H(q,p)$. In the formulation explicitly proposed under this name, an HWM factorizes into an encoder $E_\theta$ mapping past observations into a latent phase state, a Hamiltonian-inspired transition $T_H$ rolling that state forward under actions, a decoder $D_\theta$ rendering future observations from the latent rollout, and a planner that uses the generated rollouts to select actions (Cui et al., 1 May 2026). The broader literature uses this template in several forms: direct latent Hamiltonian dynamics from images, canonical ODE auto-encoders for reinforcement learning, constrained Cartesian formulations with explicit holonomic constraints, symplectic latent reduction for high-dimensional systems, and port- or soft-Hamiltonian extensions for dissipative and controlled environments (Toth et al., 2019, Feng et al., 2022, Finzi et al., 2020, Friedl et al., 29 Sep 2025, Luan et al., 18 May 2026, Tang et al., 7 May 2026).

1. Emergence of the paradigm

Early neural Hamiltonian models were formulated as scalar-energy learners on phase-space coordinates, with dynamics obtained by automatic differentiation through Hamilton’s equations. A canonical HNN replaces the unknown Hamiltonian with a neural network $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$ and predicts the vector field as $\dot z = J \nabla_z H_\theta(z)$, which in the error-free ODE solver limit conserves the learned energy exactly (Gruver et al., 2022). The methodological survey on neural Hamiltonian dynamics organized subsequent developments around generalized Hamiltonian systems, symplectic integration, generalized input form, and extended problem settings such as dissipation, control, constraints, graphs, and high-dimensional sensory observations (Chen et al., 2022).

A decisive extension from state-space learning to world modeling appeared in Hamiltonian Generative Networks. HGN introduced a model capable of consistently learning Hamiltonian dynamics from high-dimensional observations such as images, using an encoder to infer an initial latent $(q_0,p_0)$, a Hamiltonian network to generate latent dynamics, and a decoder that reconstructs frames from the position component $q_t$ (Toth et al., 2019). This moved Hamiltonian learning from derivative regression toward sequential latent simulation.

Subsequent work connected Hamiltonian latent dynamics to model-based reinforcement learning. NODA used an auto-encoder to discover latent canonical variables $(q,p)$ from high-dimensional observations and a neural ODE module to learn their time-evolution, with a decoder reconstructing next-state observations and a reward predictor attached to the same latent (Feng et al., 2022). Later formulations made the planner-facing latent itself explicitly Hamiltonian or port-Hamiltonian, as in DreamSAC, PH-Dreamer, and HaM-World, where imagined rollouts are part of policy optimization and action search rather than solely a predictive benchmark (Tang et al., 8 Mar 2026, Luan et al., 18 May 2026, Tang et al., 7 May 2026).

2. Canonical latent dynamics and their extensions

The standard conservative formulation represents the latent state as $H(q,p)$0 and models the Hamiltonian as

$H(q,p)$1

with canonical equations

$H(q,p)$2

In compact form, with $H(q,p)$3 and $H(q,p)$4, this is $H(q,p)$5 (Chen et al., 2022). This phase-space view underlies both low-dimensional HNNs and latent world models.

For action-conditioned and non-ideal environments, the conservative flow is typically relaxed rather than abandoned. The conceptual HWM formulation augments pure Hamiltonian flow with a dissipation term $H(q,p)$6, a structured control input map $H(q,p)$7, and a residual correction $H(q,p)$8:

$H(q,p)$9

This is explicitly motivated by friction, contact, non-conservative forces, and deformable objects in real-world robotic scenes (Cui et al., 1 May 2026). Port-Hamiltonian modeling makes the same extension explicit in continuous time by writing

$E_\theta$0

so that energy storage, exchange, dissipation, and power injection are all represented in the dynamics (Luan et al., 18 May 2026).

Several recent architectures soften the exact Hamiltonian constraint while preserving a canonical substructure. HaM-World decomposes the latent into $E_\theta$1, where $E_\theta$2 are canonical coordinates and $E_\theta$3 carries semantic, dissipative, or non-conservative information. Its update mixes a learned residual dynamics core with the Hamiltonian vector field through a coefficient $E_\theta$4, so that $E_\theta$5 recovers forward-Euler controlled Hamiltonian flow and $E_\theta$6 recovers an unconstrained MLP residual plus control (Tang et al., 7 May 2026). This suggests a graded notion of Hamiltonian world modeling in which exact conservative mechanics is one endpoint rather than a universal assumption.

3. Structural and geometric priors

A central theme in the literature is reduction of regression complexity by imposing structure on the learned energy. Separable Hamiltonian Neural Networks encode the additive decomposition

$E_\theta$7

either by architectural factorization into two MLPs or by a penalty on the mixed Hessian $E_\theta$8. On five systems—nonlinear pendulum, anisotropic oscillator, Hénon–Heiles, Toda lattice, and coupled oscillator—every form of separable HNN outperformed the baseline HNN in both energy error and vector-field error, and on Hénon–Heiles the HNN-OI model reduced endpoint MSE in $E_\theta$9 by approximately $T_H$0 and energy drift by approximately $T_H$1 (Khoo et al., 2023).

Constraint handling is another major source of inductive bias. “Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints” embeds systems in Cartesian coordinates and enforces holonomic constraints with time-dependent Lagrange multipliers, rather than learning in generalized coordinates. The Hamiltonian is written as

$T_H$2

with constrained equations

$T_H$3

together with $T_H$4 (Finzi et al., 2020). On the gyroscope task, CHNN was reported as more than $T_H$5 more data-efficient and approximately $T_H$6 more accurate at fixed data than HNN, while on N-pendulums, magnetic pendulum, and rigid rotor CHNN/CLNN achieved $T_H$7–$T_H$8 orders of magnitude lower $T_H$9 and dramatically lower test variance (Finzi et al., 2020). The explicit separation between energy and constraints is presented there as the reason the network can learn a much simpler scalar function $D_\theta$0.

High-dimensional Hamiltonian learning has increasingly been addressed by differential-geometric latent reduction. GeoHNN enforces two structures simultaneously: the Riemannian geometry of inertia by parameterizing inertia matrices in the SPD manifold via the exponential map, and the symplectic geometry of phase space via a constrained autoencoder with layer-wise biorthogonal constraints (Aboussalah et al., 21 Jul 2025). On deformable cloth, the constrained encoder improved position prediction error from approximately $D_\theta$1 to approximately $D_\theta$2, momentum error from approximately $D_\theta$3 to approximately $D_\theta$4, and reconstruction error from approximately $D_\theta$5 to approximately $D_\theta$6 for positions and from approximately $D_\theta$7 to approximately $D_\theta$8 for momenta (Aboussalah et al., 21 Jul 2025).

RO-HNN pursues the same objective through a geometrically-constrained symplectic autoencoder that learns a low-dimensional, structure-preserving symplectic submanifold, coupled with a Hamiltonian model on the latent manifold and a second-order Strang-splitting symplectic integrator (Friedl et al., 29 Sep 2025). It was evaluated on a $D_\theta$9-DoF coupled pendulum, a $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$0-DoF thin cloth with damping, and a $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$1-DoF particle vortex, where full-order HNNs were reported to fail to train or diverge, while RO-HNN matched the oracle low-dimensional HNN on the pendulum and generalized well on cloth and vortex (Friedl et al., 29 Sep 2025).

4. Inference from observations, rollout losses, and integration

Hamiltonian world models usually combine a latent physics core with an observation model. In HGN, the encoder takes a stack of frames $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$2, infers a latent $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$3, and a decoder reconstructs each frame from the rolled-out latent positions $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$4. Training uses a VAE objective,

$H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$5

with no extra “dynamics-matching” loss (Toth et al., 2019). On ideal pendulum, mass-spring, two-body, and three-body datasets, HGN with leapfrog achieved test-set pixel MSE on the order of $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$6, while HNN baselines on pixels often collapsed to a constant image with MSE greater than $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$7 (Toth et al., 2019).

Auto-encoding remains common when the observation space is not itself canonical. NODA uses an encoder $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$8, a neural ODE block evolving the latent $H_\theta(q,p):\mathbb R^{2n}\to\mathbb R$9 under actions, and a decoder $\dot z = J \nabla_z H_\theta(z)$0 reconstructing next observations; the training loss combines auto-encoder reconstruction, one-step state prediction, and reward prediction with mixing weight $\dot z = J \nabla_z H_\theta(z)$1 (Feng et al., 2022). DreamSAC similarly builds on an RSSM, but replaces the standard dynamics predictor with a Hamiltonian dynamics prior over object-centric slot latents, each split into generalized coordinates and canonical momenta (Tang et al., 8 Mar 2026).

Training objectives vary with the supervision available. The survey distinguishes derivative-matching losses such as

$\dot z = J \nabla_z H_\theta(z)$2

from rollout losses of the form

$\dot z = J \nabla_z H_\theta(z)$3

as well as explicit regularizers for symplecticity, degeneracy conditions, or holonomic constraints (Chen et al., 2022). GeoHNN and RO-HNN add reconstruction and latent-matching objectives to multi-step latent rollouts, whereas DreamSAC adds a viewpoint-robust InfoNCE term with cosine similarity and temperature $\dot z = J \nabla_z H_\theta(z)$4 to the standard ELBO (Aboussalah et al., 21 Jul 2025, Friedl et al., 29 Sep 2025, Tang et al., 8 Mar 2026).

Integration is a constitutive component rather than a numerical afterthought. HGN recommends the symplectic leapfrog integrator, emphasizing exact volume preservation and reversibility (Toth et al., 2019). GeoHNN trains either with direct derivative matching or with a symplectic integrator such as symplectic Euler (Aboussalah et al., 21 Jul 2025). RO-HNN uses a second-order Strang-splitting symplectic integrator (Friedl et al., 29 Sep 2025). DreamSAC uses explicit Euler during training for gradient stability and a symplectic Leapfrog integrator during imagined rollouts (Tang et al., 8 Mar 2026). PH-Dreamer uses RK4 for its shadow port-Hamiltonian transition (Luan et al., 18 May 2026). The consistent pattern is that the learned scalar energy and the integration scheme are jointly responsible for long-horizon behavior.

5. Planning and control

In model-based reinforcement learning, the latent Hamiltonian is used not only to predict but also to support policy improvement. NODA formalizes this by placing the learned latent transition inside the one-step model

$\dot z = J \nabla_z H_\theta(z)$5

and proves uniform bounds for multi-step transition errors and value errors under certain conditions (Feng et al., 2022). Empirically, NODA-SAC was reported to match or exceed SAC, especially in early-training returns, and NODA-Dreamer outperformed baselines on average return over $\dot z = J \nabla_z H_\theta(z)$6 frames on DeepMind Control Suite tasks (Feng et al., 2022).

DreamSAC adds an explicit exploration mechanism based on conservation-law violation in the current model. Its intrinsic reward is

$\dot z = J \nabla_z H_\theta(z)$7

annealed against an RND bonus during unsupervised pretraining (Tang et al., 8 Mar 2026). On OOD downstream tasks after $\dot z = J \nabla_z H_\theta(z)$8 pretrain and $\dot z = J \nabla_z H_\theta(z)$9 adapt, it reported $(q_0,p_0)$0 on Reacher-hard under unseen view, $(q_0,p_0)$1 on FetchReach under unseen object, $(q_0,p_0)$2 on Walker-walk under unseen gravity, and $(q_0,p_0)$3 on Cheetah-run under unseen friction (Tang et al., 8 Mar 2026).

PH-Dreamer adopts a port-Hamiltonian latent regularizer and an explicit kinematics-aware energy estimator. Its projected latent phase space volume was reduced by $(q_0,p_0)$4–$(q_0,p_0)$5 across tasks, final online returns averaged $(q_0,p_0)$6 versus $(q_0,p_0)$7 for R2Dreamer, total energy consumption was reduced by up to $(q_0,p_0)$8, and mean squared jerk by up to $(q_0,p_0)$9 (Luan et al., 18 May 2026). HaM-World, by contrast, places the Hamiltonian prior directly in the planner-facing latent used by Cross-Entropy Method action search. On four DeepMind Control Suite tasks it reached the highest Avg. AUC of training return, $q_t$0 with a reported $q_t$1 over the best baseline TD-MPC2, reduced imagined-rollout MSE to $q_t$2 of TD-MPC2, and achieved the highest return in every one of $q_t$3 OOD perturbation conditions (Tang et al., 7 May 2026).

These results indicate that, in the control setting, Hamiltonian world models are used in at least three distinct ways: as latent transition regularizers, as explicit energy estimators for auxiliary constraints or exploration bonuses, and as planner-facing dynamics cores. A plausible implication is that the term now denotes a design family rather than a single algorithmic recipe.

6. Empirical properties, controversies, and open problems

The main empirical arguments for Hamiltonian world models are long-horizon stability, data efficiency, and physically consistent latent trajectories. CHNN attributes $q_t$4–$q_t$5 gains in accuracy and data efficiency to Cartesian coordinates with explicit constraints (Finzi et al., 2020). HGN emphasizes reversibility, smooth interpolation in time, and speed control by scaling the integration step (Toth et al., 2019). GeoHNN, RO-HNN, PH-Dreamer, and HaM-World all report improved long-term stability or physically consistent rollouts in higher-dimensional or control-oriented settings (Aboussalah et al., 21 Jul 2025, Friedl et al., 29 Sep 2025, Luan et al., 18 May 2026, Tang et al., 7 May 2026).

A significant methodological controversy concerns which inductive bias is actually responsible for these gains. “Deconstructing the Inductive Biases of Hamiltonian Neural Networks” argues that, contrary to conventional wisdom, the improved generalization of HNNs is the result of modeling acceleration directly and avoiding artificial complexity from the coordinate system, rather than symplectic structure or energy conservation (Gruver et al., 2022). On synthetic Hamiltonian benchmarks, NODE + SO matched or exceeded standard HNN, pure HNNs did not conserve the true energy any better than NODEs once rollout error was accounted for, and symplectic regularization yielded negligible generalization gain (Gruver et al., 2022). This does not negate Hamiltonian world modeling, but it does narrow the claim: some reported benefits may derive from second-order mechanical structure or coordinate choice rather than from exact conservative mechanics alone.

The open problems are correspondingly concrete. The survey notes that classical HNN only covers conservative systems; dissipative or controlled variants add complexity, partial observability remains challenging, chaotic or stiff systems remain hard, and additional compute and memory overhead arise from symplectic layers and higher-order differentiation (Chen et al., 2022). DreamSAC lists conservative dynamics assumption, numerical stiffness for hard contacts, training-versus-inference discrepancy from Euler versus Leapfrog, increased compute from symplectic rollout, and limited applicability of fixed-slot object encoders to fluid or cloth dynamics (Tang et al., 8 Mar 2026). The conceptual HWM perspective adds friction, contact, non-conservative forces, and deformable objects as persistent obstacles for physically meaningful long-horizon prediction from pixels (Cui et al., 1 May 2026).

Taken together, the literature presents Hamiltonian World Models as a structured latent-dynamics program: observations are mapped into a phase-space representation, latent evolution is organized around energy geometry, and additional modules absorb controls, dissipation, memory, constraints, or observation complexity. Their scope now extends from conservative toy systems to high-dimensional deformable dynamics and planner-facing RL models, but their validity still depends on how well the Hamiltonian core is matched to the true non-ideal structure of the environment.