Hamiltonian/Lagrangian GNNs

• Hamiltonian/Lagrangian GNNs are neural architectures that fuse classical mechanics principles with graph-based learning to capture energy conservation and system dynamics.
• They utilize physics-informed methods like Hamilton’s equations and Euler–Lagrange formulations to ensure stable long-horizon predictions and interpretable law discovery.
• Empirical results demonstrate their effectiveness in complex system simulation, scalable node classification, and robust embedding generation across diverse applications.

Hamiltonian and Lagrangian Graph Neural Networks (GNNs) are a class of neural architectures that integrate structure-preserving dynamics from Hamiltonian or Lagrangian mechanics with message-passing on graphs. These models combine the inductive biases of geometric physics—such as energy conservation and symplecticity—with the expressive relational structure of GNNs, enabling robust learning of high-dimensional dynamical systems, node embeddings, and interpretable law discovery.

1. Theoretical Foundations: Hamiltonian and Lagrangian Dynamics on Graphs

Hamiltonian GNNs (HamGNN, HGNN, HDG, SympGNN) encode the state of each node as a pair of position and momentum (or velocity) vectors, forming a global phase-space variable $(q,p)$ across all nodes. The joint evolution of these states is then dictated by a learnable Hamiltonian $H(q,p)$ parameterized as a neural network utilizing message-passing layers. Classical Hamilton’s equations,

$$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$

are enforced, with gradients computed via back-propagation through the entire GNN structure. Symplectic integrators (e.g., velocity Verlet, explicit Euler, splitting integrators) are utilized to ensure approximate conservation of the Hamiltonian over time (Bishnoi et al., 2023, Kang et al., 2023, Kang et al., 2023, Varghese et al., 2024).

Lagrangian GNNs (LGNN, Lagrangian Propagation GNN) instead parameterize a scalar Lagrangian $\mathcal{L}(q,\dot q)$, typically as the difference of kinetic and potential energies, using separate message-passing GNNs for each. The system evolution is governed by discrete or continuous Euler–Lagrange equations,

$$\frac{d}{dt}\Bigl(\frac{\partial \mathcal{L}}{\partial \dot{q}}\Bigr) - \frac{\partial \mathcal{L}}{\partial q} = 0,$$

with gradients and Hessians evaluated through autodiff (Bhattoo et al., 2022, Tiezzi et al., 2020). The constraint-based Lagrangian GNN formulates state evolution and convergence as a saddle-point problem, updating both node states and Lagrange multipliers in a unified optimization loop.

2. Architectural Principles and Variants

Hamiltonian/Lagrangian GNNs adhere to physics-enforced modularity. Common architectural features include:

• Graph representation: Nodes correspond to particles, articulated joints, or abstract graph vertices; edges represent pairwise interactions or physical constraints, with edge features (types, distances/angles).
• Hamiltonian/Lagrangian decomposition: Energies are decomposed into local node-wise and edge-wise contributions, each parameterized by dedicated MLPs or GNNs. For separable Hamiltonians, kinetic $T(p)$ and potential $V(q)$ are encoded separately (Varghese et al., 2024, Bishnoi et al., 2023).
• Message passing: Multi-layer message-passing aggregates neighbor information explicitly into node and edge embeddings, which feed into the energy networks. Activation functions (e.g., squareplus) and learned type embeddings are employed (Bishnoi et al., 2023, Bhattoo et al., 2022).
• Integrator design: Layer-wise updates correspond to explicit or symplectic integrators for the governing ODEs (e.g., velocity Verlet, explicit or symplectic Euler, high-order Runge–Kutta, learned splitting integrators) (Varghese et al., 2024, Kang et al., 2023).
• Symplectic structure learning: Some Hamiltonian GNNs (e.g., SAH-GNN) relax the canonical symplectic form, optimizing the symplectic structure $\Omega$ over the symplectic Stiefel manifold via Riemannian optimization, increasing expressivity and adaptability to graph geometry (Liu et al., 2023).

The following table summarizes prototypical architectures:

Model	Energy Parametrization	Integration	Message Passing	Symplectic Adaptation
HGNN (Bishnoi et al., 2023)	$T(p)$, $V(q)$ (MLPs)	Velocity Verlet	Yes	Fixed canonical $H(q,p)$0
SympGNN (Varghese et al., 2024)	$H(q,p)$1, $H(q,p)$2 (MLP/quad)	Learned splitting	Yes	Perm-equivariant modular
SAH-GNN (Liu et al., 2023)	$H(q,p)$3 (MLP)	RK4, symplectic	Yes	Learnable $H(q,p)$4 on Sp
LGNN (Bhattoo et al., 2022)	$H(q,p)$5, $H(q,p)$6 (GNNs)	EL Discrete	Yes	N/A (Lagrangian)

3. Training Methodology and Physics-Informed Objectives

Hamiltonian/Lagrangian GNNs are typically trained using standard trajectory-based loss functions. The principal approach is to:

• Observe state pairs (e.g., $H(q,p)$7 or $H(q,p)$8) from ground-truth simulations at consecutive time steps.
• Predict the next-step state using the learned model and minimize the mean squared error (MSE) between predicted and observed trajectories.
• No explicit loss is applied for energy or momentum conservation; the model structure and the physics-enforced flow automatically ensure these invariants are preserved up to integrator error (Bishnoi et al., 2023, Kang et al., 2023, Varghese et al., 2024).

Lagrangian GNNs (LGNN) minimize acceleration error, leveraging automatic differentiation of the Lagrangian to compute $H(q,p)$9 via the EL equations (Bhattoo et al., 2022).

When applied to node classification, the Hamiltonian layers act as depth-robust embedding generators, feeding final $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$0 (or $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$1) states into standard linear or MLP decoders with cross-entropy loss.

Regularization is generally unnecessary; the architectural constraint of energy function learning provides strong physical inductive bias, potentially reducing the need for problem-specific hyperparameter tuning (Kang et al., 2023, Bishnoi et al., 2023).

4. Empirical Performance and Generalization

Hamiltonian/Lagrangian GNNs demonstrate:

• Accurate long-horizon dynamics: HGNN achieves phase-space rollout matching for $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$2 integration steps, with energy/momentum conservation error remaining $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$3; fine-grained force and energy MSEs on the order $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$4–$$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$5 in standard test systems (Bishnoi et al., 2023).
• Zero-shot scalability: Models trained on small $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$6 generalize to $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$7 larger systems (e.g., pendulums, spring chains, Lennard–Jones fluids, articulated rigid-body chains) with comparable prediction error (Bishnoi et al., 2023, Bhattoo et al., 2022).
• Hybrid compositionality: Additivity of learned Hamiltonians allows simulating new hybrid systems by summing component Hamiltonians for previously unseen compositionality (Bishnoi et al., 2023).
• Robustness to adversarial perturbations and over-smoothing: Hamiltonian GNNs resist degradation in node classification under adversarial topology/feature attacks, outperforming traditional GCNs and continuous GNNs by up to 40 points in accuracy under white-box attacks (HANG, HANG-quad) (Zhao et al., 2023). Energy conservation and phase-space volume preservation explain empirical stability and resistance to over-smoothing at layer depths $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$8 (Kang et al., 2023, Varghese et al., 2024).
• System identification: SympGNN achieves state-of-the-art identification and multi-step prediction error in high-dimensional spring and 2000-particle LJ systems, maintaining energy drift $$\dot{q} = +\frac{\partial H}{\partial p}, \qquad \dot{p} = -\frac{\partial H}{\partial q},$$9 over $\mathcal{L}(q,\dot q)$0-step rollouts compared to $\mathcal{L}(q,\dot q)$1 in non-structure-preserving models (Varghese et al., 2024).
• Node classification and heterophily: LA-SympGNN and HamGNN match or surpass GCN/GAT and hyperbolic GNN benchmarks on a wide variety of graphs, including those with heterogeneous and mixed geometries, and resist accuracy collapse in high-depth or low-homophily regimes (Varghese et al., 2024, Kang et al., 2023).

5. Interpretability and Law Discovery

A distinctive feature of Hamiltonian/Lagrangian GNNs is their support for symbolic law recovery via post-hoc regression on the learned energy functions:

1. After training, sampled input features are passed through the model to evaluate node, edge, or overall energy terms.
2. Symbolic regression (e.g., via PySR) is applied to extract compact, closed-form expressions for kinetic and potential energies.
3. The learned laws recover canonical forms to high precision (e.g., $\mathcal{L}(q,\dot q)$2, $\mathcal{L}(q,\dot q)$3, Lennard-Jones $\mathcal{L}(q,\dot q)$4/$\mathcal{L}(q,\dot q)$5 potentials) even in complex multi-component systems (Bishnoi et al., 2023).

This approach enables interpretable, data-driven discovery of governing symbolic interaction laws directly from trajectories—unlike standard black-box GNNs.

6. Advancements in Geometric and Stability Structure

Recent work explores relaxation of the canonical symplectic structure to achieve greater adaptability:

• Learnable symplectic structure (SAH-GNN): Instead of fixing the symplectic $\mathcal{L}(q,\dot q)$6 matrix, a symplectic matrix $\mathcal{L}(q,\dot q)$7 is optimized via Riemannian gradient updates, ensuring that energy conservation and geometric structure persist across diverse graph types (Liu et al., 2023).
• Permutation equivariance and symmetry: SympGNN realizes symplectic integration that is also permutation equivariant—achieved by parametrizing $\mathcal{L}(q,\dot q)$8 and $\mathcal{L}(q,\dot q)$9 energies as sum or graph operator-based functionals—enabling direct application to many-body and generalized graph scenarios (Varghese et al., 2024).
• Lagrangian parameter manifold learning: The Lagrangian perspective suggests generalizing to learning Riemannian metrics or mass matrices as positive-definite matrix variables, extending geometric learning beyond energy structure (Liu et al., 2023).

These developments enable adaptation to latent curved geometries, hierarchical or composite topologies, as well as controlled stability and invariance properties for broader classes of physical and combinatorial systems.

7. Practical Applications and Extensions

Hamiltonian/Lagrangian GNNs have demonstrated effectiveness in:

• Physical system identification: Molecular dynamics, coupled oscillators, many-body atomic systems, articulated multibody simulation, and granular or soft-body media (Bishnoi et al., 2023, Bhattoo et al., 2022, Varghese et al., 2024).
• Symbolic law discovery: Automated inference of physical laws from trajectory data or atomistic simulation, subject only to graph connectivity choices (Bishnoi et al., 2023).
• Robust node embedding and classification: Node classification and link prediction on graphs with widely varying geometry, homophily, depth, and adversarial robustness constraints (Kang et al., 2023, Zhao et al., 2023, Kang et al., 2023, Varghese et al., 2024).
• Generalization and compositional hybrid modeling: Accurate transfer to systems with more components or new hybrid configurations unseen in training (Bishnoi et al., 2023, Bhattoo et al., 2022).

Potential extensions include equivariant Hamiltonian/Lagrangian layers for explicit $$\frac{d}{dt}\Bigl(\frac{\partial \mathcal{L}}{\partial \dot{q}}\Bigr) - \frac{\partial \mathcal{L}}{\partial q} = 0,$$0 invariance, incorporation of dissipation and stochasticity, data-driven constraint learning, and application to cellular, biological, or continuum systems with adaptive, latent topology (Bishnoi et al., 2023, Liu et al., 2023, Bhattoo et al., 2022).

---

In summary, Hamiltonian and Lagrangian GNNs collectively establish a framework for graph-based learning of complex dynamical systems and robust embeddings, distinguished by physical interpretability, exact conservation principles, inductive scalability, and extendability to broad downstream scientific and machine learning tasks (Bishnoi et al., 2023, Kang et al., 2023, Zhao et al., 2023, Liu et al., 2023, Tiezzi et al., 2020, Kang et al., 2023, Bhattoo et al., 2022, Varghese et al., 2024).