KAR-HNN: Structured Hamiltonian Neural Network

• KAR-HNN is a neural architecture that decomposes the Hamiltonian into a structured sum of univariate functions, enhancing local approximation and interpretability.
• It replaces monolithic MLP blocks with modular univariate function networks, leading to improved numerical stability, efficient learning, and robust energy conservation.
• Empirical evaluations reveal reduced energy drift and superior phase-space fidelity in complex dynamical systems compared to traditional HNNs.

A Kolmogorov-Arnold Representation-based Hamiltonian Neural Network (KAR-HNN) is a neural architecture for modeling dynamical systems governed by Hamiltonian mechanics, integrating the Kolmogorov-Arnold representation theorem as the network's function approximation core. Unlike conventional Hamiltonian Neural Networks (HNNs) based on generic multilayer perceptrons, KAR-HNN parameterizes the Hamiltonian function as a structured sum and composition of univariate (single-variable) functions, reflecting the decomposition guaranteed by Kolmogorov's theorem. This architectural principle imparts improved capacity for local function approximation, enhanced numerical stability, and more robust conservation of physical invariants over long prediction horizons, particularly in high-dimensional or multi-scale settings (Wu et al., 26 Aug 2025).

1. Mathematical and Theoretical Foundations

KAR-HNN builds on two classical results: the Hamiltonian formulation of dynamical systems and the Kolmogorov-Arnold representation theorem. Hamiltonian systems evolve according to the equations

$$\dot{q} = \frac{\partial H}{\partial p},\quad \dot{p} = -\frac{\partial H}{\partial q}$$

where $H(q,p)$ is the Hamiltonian encoding the system's total energy. Standard HNNs model $H$ with a general neural network, often a multilayer perceptron (MLP).

The Kolmogorov-Arnold theorem asserts that any continuous multivariate function $f : [0,1]^n \to \mathbb{R}$ can be represented as

$$f(x_1, \ldots, x_n) = \sum_{k=1}^{2n+1} \Phi^k \bigg( \sum_{j=1}^n f^{kj}(x_j) \bigg)$$

with continuous univariate functions $f^{kj}$ and $\Phi^k$. In KAR-HNN, the Hamiltonian $H$ is explicitly parameterized with this structure: $$H_\theta(z) = \sum_{r=1}^{2d+1} \Phi_r^\theta \left( \sum_{s=1}^d \phi_{r,s}^\theta(z_s) \right)$$ for $z = [q; p]$, the concatenated state vector in $2d$ dimensions (Wu et al., 26 Aug 2025, Poluektov et al., 2023, Polar et al., 2020).

This explicit functional decomposition is not only theoretically justified but is also well-suited for neural representation, as each function can be modeled with univariate basis expansions (splines, piecewise polynomials, or learned lookup tables), enhancing both interpretability and computational tractability.

2. Network Architecture and Implementation

The design replaces the monolithic MLP Hamiltonian block with a "sum-of-compositions-of-univariate" functional module. The architecture arranges learnable univariate functions $f^{kj}$ as "branch" modules that process individual coordinate inputs, summing these into internal auxiliaries $\phi_k$, which are then mapped to scalar outputs by corresponding outer univariate functions $\Phi^k$; the total Hamiltonian is the sum across these outputs.

Formally, for input $z \in \mathbb{R}^{2d}$,

• Each branch computes $f^{kj}(z_j)$ for its coordinate.
• For each outer function $\Phi^k$, compute $\Phi^k(\sum_j f^{kj}(z_j))$.
• Sum over $k$ to produce $H(z)$.

Learning occurs by adjusting the parameters of the univariate blocks, which may be implemented as:

• Piecewise linear interpolants or splines over fixed grids.
• Learnable lookup tables.
• Univariate shallow neural networks.

Gradient computation for Hamilton’s equations is systematically handled by applying the chain rule over the univariate components.

The modularity of this setup supports efficient projection methods for training individual basis weights, as in the Newton-Kaczmarz algorithm (Poluektov et al., 2023), and scalable parallel updates for each functional component (Polar et al., 2020).

3. Symplectic Structure and Energy Conservation

A defining feature of the KAR-HNN is preservation of the symplectic (canonical) structure inherent to Hamiltonian flows. Since the time evolution is generated directly from the partial derivatives of the composed Hamiltonian, any invariants or conservation laws encoded in the structure of $H$ are automatically respected, up to the representational capacity and optimization error of the model.

Empirical evaluations on dynamical systems (spring-mass, pendulum, two-body, three-body) reveal that KAR-HNN maintains lower energy drift and superior phase-space fidelity over long simulations compared to MLP-based HNNs—even for challenging chaotic systems. KAR-HNN achieves reduced sensitivity to hyperparameters, robust generalization to unseen initial conditions, and improved error scaling with problem dimensionality (Wu et al., 26 Aug 2025).

The localized approximation properties of univariate function modules make the model particularly adept at capturing multi-scale and high-frequency components in $H$, a known difficulty for global function approximators in high-dimensional settings.

4. Comparative Evaluation and Benchmarking

Experimental results on four canonical systems demonstrate the efficacy of the approach (Wu et al., 26 Aug 2025):

System	Test MSE (×10³)	Energy Drift	Remarks
Spring–Mass	28–30	≈1.63	Superior to both baseline & MLP-HNN
Simple Pendulum	34.3	Smallest	Robust oscillatory regime
Two-Body Problem	0.281 (×10⁶)	1.56	Best among tested models
Three-Body	17.9	5.8	Improved chaotic capture

The combination of accuracy in derivatives and tight energy conservation persists even as the system complexity increases, a domain where conventional architectures typically struggle with error accumulation and drift.

5. Applications and Scope

The structured, interpretable energy representation of KAR-HNN is attractive for scientific computing, engineering, and any context requiring long-term accurate predictions in systems governed (exactly or approximately) by Hamiltonian mechanics. Specific applications include:

• Planetary and celestial dynamics (two-/three-body, n-body problems).
• Molecular dynamics and lattice models with complex interaction potentials.
• Oscillatory engineering systems, e.g., robotics or electronics with rich multi-scale behavior.
• Partial differential equation modeling where physics-informed decomposition is beneficial (Poluektov et al., 2023).

For practical implementation, the fine-grained control of nodal density per univariate function supports a direct complexity–accuracy tradeoff, and explicit handling of quantized or categorical variables is native to the architecture (Polar et al., 2020).

6. Architectural Innovations and Future Research

KAR-HNN demonstrates that structured, modular representations—rooted in classical approximation theory—offer concrete improvements over traditional black-box architectures in data-driven physical modeling. Future research directions suggested by the literature include:

• Extension to higher-dimensional, highly multi-scale, or partially observed systems.
• Integration with advanced symplectic integrators or noise-robust estimation schemes.
• Hybridization with other inductive bias mechanisms (e.g., symmetry detection, almost Poisson structures for constrained dynamics).
• Investigation of efficient grid and parameterization schemes for univariate components, potentially leveraging advances in spline-based or edge-activation neural designs (KANs) (Moradi et al., 2 Oct 2024, Basina et al., 15 Nov 2024).

Ongoing work explores the interplay between modularity, interpretability, and computational efficiency, emphasizing architectures that allow for tractable error bounds and stable long-term simulation in high-dimensional, physically constrained environments.

7. Significance for Physics-Informed and Scientific Machine Learning

KAR-HNN provides a unifying perspective linking universal function approximation results (Kolmogorov-Arnold), physics-preserving neural architectures, and modern computational techniques for dynamical systems. Theoretical analyses demonstrate that, under sufficient expressive capacity, the error in the predicted energy gradient can be controlled, and classical stability/perturbation results (KAM theory) guarantee persistence of quasi-periodic orbits even under modeling error, provided the error is small (Chen et al., 2021). This ensures practical relevance and robust predictive capability when applied to the discovery and simulation of complex dynamical phenomena.

The framework setting is thus not only of practical computational importance but also clarifies the interplay between deep learning representations, approximation theory, and physical law.

---

References:

• (Wu et al., 26 Aug 2025) Kolmogorov-Arnold Representation for Symplectic Learning: Advancing Hamiltonian Neural Networks
• (Raj, 17 Jun 2025) Structured and Informed Probabilistic Modeling with the Thermodynamic Kolmogorov-Arnold Model
• (Basina et al., 15 Nov 2024) KAT to KANs: A Review of Kolmogorov-Arnold Networks and the Neural Leap Forward
• (Moradi et al., 2 Oct 2024) Kolmogorov-Arnold Network Autoencoders
• (Poluektov et al., 2023) Construction of the Kolmogorov-Arnold representation using the Newton-Kaczmarz method
• (Chen et al., 2021) KAM Theory Meets Statistical Learning Theory: Hamiltonian Neural Networks with Non-Zero Training Loss
• (Polar et al., 2020) A deep machine learning algorithm for construction of the Kolmogorov-Arnold representation
• (Schmidt-Hieber, 2020) The Kolmogorov-Arnold representation theorem revisited