---
title: Hamiltonian/Lagrangian GNNs
url: https://www.emergentmind.com/topics/hamiltonian-lagrangian-gnns
type: topic
---

# Hamiltonian/Lagrangian GNNs

Hamiltonian and Lagrangian Graph Neural Networks (GNNs) are a class of neural architectures that integrate structure-preserving dynamics from Hamiltonian or Lagrangian mechanics with message-passing on graphs. These models combine the inductive biases of geometric physics—such as energy conservation and symplecticity—with the expressive relational structure of GNNs, enabling robust learning of high-dimensional dynamical systems, node embeddings, and interpretable law discovery.

## 1. Theoretical Foundations: Hamiltonian and Lagrangian Dynamics on Graphs

Hamiltonian GNNs (HamGNN, HGNN, HDG, SympGNN) encode the state of each node as a pair of position and momentum (or velocity) vectors, forming a global phase-space variable $(q,p)$ across all nodes. The joint evolution of these states is then dictated by a learnable Hamiltonian $H(q,p)$ parameterized as a neural network utilizing message-passing layers. Classical Hamilton’s equations,
\[
  \dot{q} = +\frac{\partial H}{\partial p}, \qquad
  \dot{p} = -\frac{\partial H}{\partial q},
\]
are enforced, with gradients computed via back-propagation through the entire GNN structure. Symplectic integrators (e.g., velocity Verlet, explicit Euler, splitting integrators) are utilized to ensure approximate conservation of the Hamiltonian over time [2307.05299, 2303.01030, 2305.18965, 2408.16698].

Lagrangian GNNs (LGNN, Lagrangian Propagation GNN) instead parameterize a scalar Lagrangian $\mathcal{L}(q,\dot q)$, typically as the difference of kinetic and potential energies, using separate message-passing GNNs for each. The system evolution is governed by discrete or continuous Euler–Lagrange equations,
\[
  \frac{d}{dt}\Bigl(\frac{\partial \mathcal{L}}{\partial \dot{q}}\Bigr) - \frac{\partial \mathcal{L}}{\partial q} = 0,
\]
with gradients and Hessians evaluated through autodiff [2209.11588, 2002.07684]. The constraint-based Lagrangian GNN formulates state evolution and convergence as a saddle-point problem, updating both node states and Lagrange multipliers in a unified optimization loop.

## 2. Architectural Principles and Variants

Hamiltonian/Lagrangian GNNs adhere to physics-enforced modularity. Common architectural features include:

- **Graph representation**: Nodes correspond to particles, articulated joints, or abstract graph vertices; edges represent pairwise interactions or physical constraints, with edge features (types, distances/angles).
- **Hamiltonian/Lagrangian decomposition**: Energies are decomposed into local node-wise and edge-wise contributions, each parameterized by dedicated MLPs or GNNs. For separable Hamiltonians, kinetic $T(p)$ and potential $V(q)$ are encoded separately [2408.16698, 2307.05299].
- **Message passing**: Multi-layer message-passing aggregates neighbor information explicitly into node and edge embeddings, which feed into the energy networks. Activation functions (e.g., squareplus) and learned type embeddings are employed [2307.05299, 2209.11588].
- **Integrator design**: Layer-wise updates correspond to explicit or symplectic integrators for the governing ODEs (e.g., velocity Verlet, explicit or symplectic Euler, high-order Runge–Kutta, learned splitting integrators) [2408.16698, 2305.18965].
- **Symplectic structure learning**: Some Hamiltonian GNNs (e.g., SAH-GNN) relax the canonical symplectic form, optimizing the symplectic structure $\Omega$ over the symplectic Stiefel manifold via Riemannian optimization, increasing expressivity and adaptability to graph geometry [2309.04885].

The following table summarizes prototypical architectures:

| Model           | Energy Parametrization      | Integration      | Message Passing | Symplectic Adaptation      |
|-----------------|----------------------------|------------------|----------------|---------------------------|
| HGNN [2307.05299]     | $T(p)$, $V(q)$ (MLPs)   | Velocity Verlet  | Yes            | Fixed canonical $J$       |
| SympGNN [2408.16698]  | $T$, $V$ (MLP/quad)     | Learned splitting| Yes            | Perm-equivariant modular  |
| SAH-GNN [2309.04885]  | $H(q,p)$ (MLP)          | RK4, symplectic  | Yes            | Learnable $\Omega$ on Sp  |
| LGNN [2209.11588]     | $\widehat T$, $\widehat V$ (GNNs)| EL Discrete    | Yes            | N/A (Lagrangian)          |


## 3. Training Methodology and Physics-Informed Objectives

Hamiltonian/Lagrangian GNNs are typically trained using standard trajectory-based loss functions. The principal approach is to:

- Observe state pairs (e.g., $(q(t), p(t))$ or $(q, \dot q)$) from ground-truth simulations at consecutive time steps.
- Predict the next-step state using the learned model and minimize the mean squared error (MSE) between predicted and observed trajectories.
- No explicit loss is applied for energy or momentum conservation; the model structure and the physics-enforced flow automatically ensure these invariants are preserved up to integrator error [2307.05299, 2303.01030, 2408.16698].

Lagrangian GNNs (LGNN) minimize acceleration error, leveraging automatic differentiation of the Lagrangian to compute $\widehat{\ddot{q}}$ via the EL equations [2209.11588].

When applied to node classification, the Hamiltonian layers act as depth-robust embedding generators, feeding final $q$ (or $q,p$) states into standard linear or MLP decoders with cross-entropy loss.

Regularization is generally unnecessary; the architectural constraint of energy function learning provides strong physical inductive bias, potentially reducing the need for problem-specific hyperparameter tuning [2303.01030, 2307.05299].

## 4. Empirical Performance and Generalization

Hamiltonian/Lagrangian GNNs demonstrate:

- **Accurate long-horizon dynamics**: HGNN achieves phase-space rollout matching for $10^5$ integration steps, with energy/momentum conservation error remaining $\ll 1$; fine-grained force and energy MSEs on the order $10^{-8}$–$10^{-4}$ in standard test systems [2307.05299].
- **Zero-shot scalability**: Models trained on small $n$ generalize to $10-50\times$ larger systems (e.g., pendulums, spring chains, Lennard–Jones fluids, articulated rigid-body chains) with comparable prediction error [2307.05299, 2209.11588].
- **Hybrid compositionality**: Additivity of learned Hamiltonians allows simulating new hybrid systems by summing component Hamiltonians for previously unseen compositionality [2307.05299].
- **Robustness to adversarial perturbations and over-smoothing**: Hamiltonian GNNs resist degradation in node classification under adversarial topology/feature attacks, outperforming traditional GCNs and continuous GNNs by up to 40 points in accuracy under white-box attacks (HANG, HANG-quad) [2310.06396]. Energy conservation and phase-space volume preservation explain empirical stability and resistance to over-smoothing at layer depths $T=32,64$ [2303.01030, 2408.16698].
- **System identification**: SympGNN achieves state-of-the-art identification and multi-step prediction error in high-dimensional spring and 2000-particle LJ systems, maintaining energy drift $<0.5\%$ over $100$-step rollouts compared to $O(10\%)$ in non-structure-preserving models [2408.16698].
- **Node classification and heterophily**: LA-SympGNN and HamGNN match or surpass GCN/GAT and hyperbolic GNN benchmarks on a wide variety of graphs, including those with heterogeneous and mixed geometries, and resist accuracy collapse in high-depth or low-homophily regimes [2408.16698, 2305.18965].

## 5. Interpretability and Law Discovery

A distinctive feature of Hamiltonian/Lagrangian GNNs is their support for symbolic law recovery via post-hoc regression on the learned energy functions:

1. After training, sampled input features are passed through the model to evaluate node, edge, or overall energy terms.
2. Symbolic regression (e.g., via PySR) is applied to extract compact, closed-form expressions for kinetic and potential energies.
3. The learned laws recover canonical forms to high precision (e.g., $T\approx 0.5 m|\dot x|^2$, $V_{ij}\approx 0.5(r_{ij}-1)^2$, Lennard-Jones $r^{-12}$/$r^{-6}$ potentials) even in complex multi-component systems [2307.05299].

This approach enables interpretable, data-driven discovery of governing symbolic interaction laws directly from trajectories—unlike standard black-box GNNs.

## 6. Advancements in Geometric and Stability Structure

Recent work explores relaxation of the canonical symplectic structure to achieve greater adaptability:

- **Learnable symplectic structure (SAH-GNN)**: Instead of fixing the symplectic $J$ matrix, a symplectic matrix $\Omega$ is optimized via Riemannian gradient updates, ensuring that energy conservation and geometric structure persist across diverse graph types [2309.04885].
- **Permutation equivariance and symmetry**: SympGNN realizes symplectic integration that is also permutation equivariant—achieved by parametrizing $T$ and $V$ energies as sum or graph operator-based functionals—enabling direct application to many-body and generalized graph scenarios [2408.16698].
- **Lagrangian parameter manifold learning**: The Lagrangian perspective suggests generalizing to learning Riemannian metrics or mass matrices as positive-definite matrix variables, extending geometric learning beyond energy structure [2309.04885].

These developments enable adaptation to latent curved geometries, hierarchical or composite topologies, as well as controlled stability and invariance properties for broader classes of physical and combinatorial systems.

## 7. Practical Applications and Extensions

Hamiltonian/Lagrangian GNNs have demonstrated effectiveness in:

- **Physical system identification**: Molecular dynamics, coupled oscillators, many-body atomic systems, articulated multibody simulation, and granular or soft-body media [2307.05299, 2209.11588, 2408.16698].
- **Symbolic law discovery**: Automated inference of physical laws from trajectory data or atomistic simulation, subject only to graph connectivity choices [2307.05299].
- **Robust node embedding and classification**: Node classification and link prediction on graphs with widely varying geometry, homophily, depth, and adversarial robustness constraints [2303.01030, 2310.06396, 2305.18965, 2408.16698].
- **Generalization and compositional hybrid modeling**: Accurate transfer to systems with more components or new hybrid configurations unseen in training [2307.05299, 2209.11588].

Potential extensions include equivariant Hamiltonian/Lagrangian layers for explicit $E(3)$ invariance, incorporation of dissipation and stochasticity, data-driven constraint learning, and application to cellular, biological, or continuum systems with adaptive, latent topology [2307.05299, 2309.04885, 2209.11588].

---

In summary, Hamiltonian and Lagrangian GNNs collectively establish a framework for graph-based learning of complex dynamical systems and robust embeddings, distinguished by physical interpretability, exact conservation principles, inductive scalability, and extendability to broad downstream scientific and machine learning tasks [2307.05299, 2303.01030, 2310.06396, 2309.04885, 2002.07684, 2305.18965, 2209.11588, 2408.16698].

Source: https://www.emergentmind.com/topics/hamiltonian-lagrangian-gnns