--- title: Hamiltonian ODE Graph Networks url: https://www.emergentmind.com/topics/hamiltonian-ode-graph-networks-hogn type: topic --- # Hamiltonian ODE Graph Networks Hamiltonian ODE Graph Networks (HOGN) are a class of models that embed the inductive bias of Hamiltonian dynamics into graph neural networks (GNNs), enabling principled modeling of complex interacting systems with strict conservation laws and adaptive geometry. They have been applied to both dynamics simulation and graph node embedding tasks, offering significant advantages in energy preservation, adversarial robustness, geometry adaptation, and zero-shot generalization. ## 1. Foundations and Mathematical Structure Hamiltonian ODE Graph Networks formulate the evolution of a system of interacting entities—represented as nodes on a graph—according to Hamilton's equations on a phase space. For a node with "position" $q^n \in \mathbb R^d$ and auxiliary "momentum" $p^n \in \mathbb R^d$, the evolution is prescribed by a node-level or global Hamiltonian function $H(q, p; \theta)$, where $\theta$ are learnable parameters: \[ \dot q^i = \frac{\partial H}{\partial p_i}, \qquad \dot p_i = -\frac{\partial H}{\partial q^i} \] The Hamiltonian function $H$ can be parameterized in multiple ways: - **Pseudo-Riemannian/Geodesic-inspired:** $H_{\mathrm{geo}}(q,p) = \frac{1}{2} g_{\mathrm{net}}^{ij}(q) p_i p_j$ with learnable metric $g_{\mathrm{net}}(q)$. - **Fully flexible MLP:** $H_{\mathrm{FC}}(q,p) = \mathrm{MLP}_\theta([q;p])$, with no explicit geometry constraints. - **Graph-structured Hamiltonians:** $H(q, p)$ can be a global output of a graph neural network receiving node and edge features, enabling the Hamiltonian to encode both physical symmetries and graph topology [2305.18965, 2303.01030, 1909.12790]. This structure ensures the flow in the $(q,p)$ phase space is symplectic and conservative, exactly or approximately preserving energy $H(q(t),p(t))$ over time, depending on the numerical integrator and Hamiltonian parameterization. ## 2. Graph Network Integration and Permutation Invariance Graph topology is integrated via message-passing architectures. Each node is updated using information from its local neighborhood via learned GNN layers. In dynamics settings, nodes correspond to physical particles, and edges model interactions such as springs or potentials. In node embedding tasks, the graph structure is injected at the aggregation stage after Hamiltonian evolution. Permutation invariance is achieved by summing or averaging over node and edge features, ensuring the Hamiltonian $H$ and the resulting dynamics are independent of node ordering. Variants such as G-SympGNN and LA-SympGNN further enhance permutation equivariance and symplecticity through explicit node- and edge-wise energy parameterizations or linear-algebraic message-passing [2408.16698]. ### Table: Graph message-passing vs. Hamiltonian parameterization | Parameterization | Graph Structure | Permutation Invariant | |--------------------|----------------------|----------------------| | Local Hamiltonian | No (default) | Yes (node-level sum) | | Graph Hamiltonian | Yes (GNN) | Yes | | Split-step (SympGNN) | Yes (explicit sum and message op) | Yes | ## 3. Numerical Integration and Training Framework The continuous-time Hamiltonian ODE is discretized using explicit symplectic integrators, such as: - **Euler method:** $q_{k+1} = q_k + h \nabla_p H(q_k, p_k),\quad p_{k+1} = p_k - h \nabla_q H(q_k, p_k)$ - **Störmer–Verlet (leapfrog):** Used especially for physical dynamics to minimize long-term energy drift [2307.05299, 2506.06558]. - **Velocity-Verlet:** Similarly preserves the symplectic structure and is efficiently compatible with autodiff frameworks. ODE solvers, including explicit Euler and higher-order Runge–Kutta schemes, are employed for node embedding architectures. During training, gradients are propagated through the ODE integration using the adjoint sensitivity method, supporting efficient end-to-end optimization [2303.01030, 2305.18965, 2310.06396, 1909.12790]. Recent advances introduce random feature-based parameterizations for the GNNs within HOGNs, enabling non-iterative training (e.g., Extreme Learning Machine or SWIM) and achieving 100–600× faster training compared to standard gradient-based optimizers [2506.06558]. ## 4. Adaptivity, Stability, and Over-smoothing Mitigation By learning a Hamiltonian energy function $H(q,p)$ over the cotangent bundle, HOGNs generalize beyond fixed-geometry (e.g., exponential map-based) manifolds. The architecture can automatically capture and adapt to local geometric structure (e.g., Euclidean, hyperbolic, or mixed curvature), a key advantage in handling datasets with heterogeneous subgraphs or varying hyperbolicity [2305.18965]. The conservation law inherent in the Hamiltonian flow anchors the node embedding or simulation trajectory, providing strong control over layer-wise feature evolution. This mitigates two systemic GNN pathologies: - **Over-smoothing:** Deep propagation does not drive embeddings to a constant subspace, as measured by stable accuracy with increasing layer depth (up to 20 or 64 layers without collapse) [2305.18965, 2303.01030]. - **Adversarial robustness:** Empirically, HOGNs exhibit minimal accuracy degradation (drop ≤ 5 points) under adversarial perturbations of edges or features, compared to significant drops in standard GCN/GAT/GRAND models. The flow is stable (BIBO and—when the Hamiltonian takes a natural form—Lyapunov stable) and resilient to chaotic drift under both evasion and poisoning attacks [2310.06396, 2303.01030]. ## 5. Task Domains and Empirical Results ### Node Embedding and Classification - HOGN (HamGNN, HDG) matches or improves upon Euclidean GNNs (GCN/GAT) on high-hyperbolicity (Euclidean) graphs (Cora, Citeseer, Pubmed) and outperforms all Euclidean and hyperbolic GNNs on low-hyperbolicity ("tree-like") graphs (Disease, Airport) and mixed-geometry unions (accuracy ∼95% vs. 75–90% for baselines) [2305.18965, 2303.01030]. - Link prediction (using inner product similarity of final $q(T)$): HOGN achieves higher ROC-AUC than standard baselines across multiple geometries. ### Physics-based Simulation - Hamiltonian ODE Graph Networks learn accurate large-scale dynamical models (mass-spring systems, pendulum chains, Lennard-Jones molecular dynamics) and preserve total energy over long rollouts (energy drift ≪ competing models). - Zero-shot generalization: Training on minimal systems (e.g., 8-node chains), HOGN transfers to 4096-node graphs or 2000-particle MD without retraining, with sub-$10^{-5}$ trajectory errors [2506.06558, 2307.05299, 2408.16698]. - Hybrid system composition: HOGN models trained independently on different systems can be composed for accurate simulation of hybrid systems, a property not matched by non-Hamiltonian baselines [2307.05299]. ### Adversarial Robustness - On standard attack benchmarks (PGD-GIA, TDGIA, MetaGIA, Nettack, Metattack), Hamiltonian neural flow GNNs (HANG, HANG-quad variants) substantially outperform all standard baselines, often doubling robust accuracy [2310.06396]. - HOGN achieves accuracy stability unaffected by the depth of message-passing layers, resisting feature collapse under both deep architectures and adversarial manipulations. ## 6. Symbolic Regression and Interpretability HOGNs support interpretable scientific discovery by enabling closed-form symbolic regression on their learned kinetic and potential energy functions. By regressing outputs of the Hamiltonian’s MLPs onto standard operator libraries, the model can recover underlying physical laws (e.g., $0.5m\|\dot x\|^2$, $\tfrac{1}{2}k(r-1)^2$, Lennard-Jones potentials) from trajectory data, including in highly nonlinear or hybrid settings [2307.05299]. This interpretability extends to hybrid systems: the symbolic regression can separately reconstruct the physical terms corresponding to each composed subsystem. ## 7. Limitations and Extensions - HOGNs are fundamentally restricted to conservative (Hamiltonian) dynamics; modeling dissipative or frictional forces requires extending the formalism, for instance via Rayleigh dissipation or Langevin terms [2307.05299]. - Scalability of dense GNNs can be a computational bottleneck for fully-connected physical graphs, but the inherent locality of message-passing supports efficient scaling on sparse/topological graphs [2506.06558, 2408.16698]. - Future research directions include symplecticity-preserving graph message passing, treating continuum systems or deformable bodies via mesh-based HOGN, and incorporating E(3) equivariance for enhanced physical fidelity [2307.05299, 2408.16698]. ## References - Node Embedding from Neural Hamiltonian Orbits in Graph Neural Networks [2305.18965] - Node Embedding from Hamiltonian Information Propagation in Graph Neural Networks [2303.01030] - Discovering Symbolic Laws Directly from Trajectories with Hamiltonian Graph Neural Networks [2307.05299] - Hamiltonian Graph Networks with ODE Integrators [1909.12790] - Adversarial Robustness in Graph Neural Networks: A Hamiltonian Approach [2310.06396] - SympGNNs: Symplectic Graph Neural Networks for identifiying high-dimensional Hamiltonian systems and node classification [2408.16698] - Rapid training of Hamiltonian graph networks without gradient descent [2506.06558] Source: https://www.emergentmind.com/topics/hamiltonian-ode-graph-networks-hogn