Hamiltonian Pretraining: Approaches & Insights

• Hamiltonian pretraining is a technique that uses Hamiltonians as primary objects for staged optimization, enabling models to learn from partial Hamiltonians or dynamical processes.
• The method encompasses diverse paradigms such as sequential Hamiltonian assembly, self-supervised masked autoencoding, and SCF-dynamics learning, yielding improvements up to 43.89% in specific tasks.
• It reduces sample complexity and combats issues like vanishing gradients, offering robust warm starts and enhanced transferability in quantum and electronic structure modeling.

Hamiltonian pretraining denotes a family of training strategies in which Hamiltonians are used not merely as downstream labels or terminal objectives, but as the objects of pretraining, the curriculum over which optimization is staged, the state variable of a learned update operator, or the source of a structured initialization. Across current literature, the phrase is used in several related senses rather than as a single standardized algorithm. The common thread is that the Hamiltonian is treated as first-class structure: a model is first exposed to Hamiltonians themselves, to partial Hamiltonians, or to Hamiltonian-consistent dynamics, and only afterward adapted to the final task or full objective (Shihab et al., 6 May 2025, Roshani et al., 2024, Lou et al., 12 Jun 2026, Otterdijk et al., 27 Jan 2026).

1. Conceptual scope

The literature suggests four recurring usages of Hamiltonian pretraining.

Usage	Representative paper	Core mechanism
Hamiltonian as data	(Shihab et al., 6 May 2025)	Self-supervised pretraining on unlabeled Hamiltonians
Hamiltonian as staged objective	(Roshani et al., 2024)	Local-to-global optimization over partial Hamiltonians
Hamiltonian as fixed-point state	(Lou et al., 12 Jun 2026)	Learn the Hamiltonian update rule, then solve for equilibrium
Hamiltonian as structured initializer	(Otterdijk et al., 27 Jan 2026)	Initialize a nonlinear Hamiltonian model from a linear one

In the strongest form, Hamiltonian pretraining means that the pretraining corpus consists of Hamiltonians themselves. In a broader optimization sense, it means that parameters are warm-started by training first on easier or more local Hamiltonian surrogates. In electronic-structure settings, it can also mean learning the dynamical process by which a Hamiltonian converges, rather than regressing the converged Hamiltonian directly. A plausible implication is that the term is best understood as a methodological orientation toward Hamiltonian-native representation learning, rather than as a single architecture family.

A recurring distinction in this literature is between Hamiltonian-side staging and architecture-side staging. In the former, the circuit or model class may remain fixed while the Hamiltonian or loss is gradually assembled; in the latter, the Hamiltonian stays fixed while depth, parameters, or state dimension are grown. This distinction is explicit in the variational-quantum literature and remains useful across other settings (Stein et al., 2023).

2. Objective-side curricula: sequential assembly of partial Hamiltonians

The clearest optimization-centric instance of Hamiltonian pretraining is Sequential Hamiltonian Assembly (SHA), introduced for PQC training under global cost Hamiltonians. The starting point is a decomposable objective

$\hat H=\sum_{i=1}^{N}\hat H_i,$

with a partition

$$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$

At stage $k$, the active Hamiltonian is the cumulative partial sum

$$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$

and the parameters are optimized against

$$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$

with the stage-$k$ solution warm-starting stage $k+1$ (Roshani et al., 2024, Stein et al., 2023).

In this formulation, pretraining occurs on truncated or localized Hamiltonians rather than on auxiliary datasets. The method is therefore a curriculum over objectives. The central motivation is trainability: non-local loss functions are identified as one source of vanishing gradients and barren plateaus, whereas early optimization on more local partial Hamiltonians is intended to provide stronger gradient signal. The 2024 extension emphasizes that the quality of the partition matters: random partitions perform worst, while problem-informed partitions perform best; for graph-based problems, a node-wise partitioning strategy groups Hamiltonian terms by graph locality (Roshani et al., 2024).

The reported empirical gains are substantial within the studied regimes. On graph coloring, SHA8 improves mean accuracy over standard VQE by $29.99\%$, over layerwise learning by $17.58\%$, and over Layer-VQE by $5.12\%$ (Stein et al., 2023). In the extended study, combining SHA8 with Layer-VQE yields a $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$0 median accuracy improvement over standard VQE, while on Max-Cut the best node-wise SHA variant improves over standard VQE by $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$1 and over Layer-VQE by $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$2 (Roshani et al., 2024). At the same time, the same papers are explicit that this benefit is not free: SHA roughly increases optimization overhead, depends strongly on a meaningful decomposition, and does not improve every optimizer equally. In particular, the QAOA+SHA merger was reported not to enhance results in a meaningful way in the small graph-coloring experiments (Roshani et al., 2024).

A common misconception is to equate this with layerwise pretraining. The distinction is exact. Layerwise methods progressively grow the circuit while keeping the cost fixed; SHA keeps the circuit fixed while progressively assembling the Hamiltonian. Their successful combination indicates that Hamiltonian pretraining and ansatz pretraining are orthogonal axes of staged optimization rather than alternative names for the same procedure (Stein et al., 2023).

3. Self-supervised pretraining directly on Hamiltonians

A more literal form of Hamiltonian pretraining appears in Hamiltonian-Masked Autoencoding (HMAE) for quantum spin systems. HMAE pre-trains transformers on unlabeled quantum Hamiltonians and then fine-tunes in few-shot regimes. Unlike random masking approaches, it uses a physics-informed masking strategy based on quantum information theory to selectively mask Hamiltonian terms according to their physical significance (Shihab et al., 6 May 2025).

Here the Hamiltonian itself is the pretraining datum. That is the crucial conceptual move. The framework is motivated by the asymmetry between abundant or cheaply generated Hamiltonians and expensive downstream labels such as phases or energies. The abstract reports experiments on 12,500 quantum Hamiltonians, with $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$3 real-world and $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$4 synthetic data. With merely 10 labeled examples, HMAE achieves $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$5 accuracy in phase classification and $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$6 MAE in ground state energy prediction, outperforming classical graph neural networks at $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$7 and quantum neural networks at $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$8, with $$P=\{P_1,\ldots,P_M\}, \qquad P_j\subseteq \{1,\ldots,N\}, \qquad \bigcup_{j=1}^M P_j=\{1,\ldots,N\}.$$9 (Shihab et al., 6 May 2025).

The primary claimed advantage is sample efficiency. The method reduces required labeled examples by $k$0–$k$1 relative to the reported baselines. At the same time, the paper is explicit that this does not remove the need for expensive ground-truth generation: fine-tuning and evaluation labels still require exact diagonalization or tensor networks. The scope restriction is equally explicit. Training is limited to 12 qubits, with only limited extension to 16–20 qubits in testing, and the paper states that this size restriction prevents immediate application to larger systems of practical interest in materials science and quantum chemistry (Shihab et al., 6 May 2025).

This line of work places Hamiltonian pretraining closest to masked-language or masked-graph pretraining, but with the domain’s native object—the Hamiltonian—replacing text or molecular graphs. A plausible implication is that the effectiveness of the approach depends not only on scale but also on choosing a pretext task whose masked structure is physically meaningful rather than syntactically arbitrary.

4. Process-level pretraining for electronic Hamiltonians

In electronic structure, Hamiltonian pretraining has taken a different form: pretraining on the dynamics by which a Hamiltonian converges. HamEvo predicts the Kohn–Sham Hamiltonian in an LCAO atomic-orbital basis, where the underlying DFT problem is written as the Roothaan–Hall equation

$k$2

Rather than learning a direct map from geometry to converged Hamiltonian, HamEvo learns a one-step Hamiltonian Evolution Operator

$k$3

and defines the final prediction as the fixed point

$k$4

This makes pretraining an operator-learning problem over SCF trajectories rather than a one-shot regression problem (Lou et al., 12 Jun 2026).

The pretraining pipeline is two-stage. Stage 1 uses consecutive SCF pairs

$k$5

from DFT runs initialized with MINAO guesses. For the GDB17 subset used here, Stage 1 includes 8,446 molecules and 269,005 conformations, retaining SCF steps satisfying

$k$6

and always including the final converged step. The Stage 1 loss is

$k$7

Stage 2 calibrates the fixed point against converged Hamiltonians and density matrices using

$k$8

with density matrices computed as

$k$9

The equilibrium dataset combines a GDB17 subset of 16,000 molecules and 76,000 geometries with a QMugs subset of 91,200 conformations (Lou et al., 12 Jun 2026).

The empirical claim is that learning SCF update dynamics is more transferable than regressing final Hamiltonians directly. Across MD17, QH9, GDB17, and QMugs, HamEvo lowers Hamiltonian MAE by $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$0–$$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$1 relative to direct-regression and deep-equilibrium baselines. On QMugs, it reports HOMO and LUMO MAEs of $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$2 and $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$3, dipole MAE of $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$4, and occupied orbital similarity $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$5. Few-shot fine-tuning with only 20 reference conformations extends the method to molecules of up to 122 atoms, with Hamiltonian MAEs of $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$6, $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$7, and $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$8 on the reported OOD systems (Lou et al., 12 Jun 2026).

This formulation also clarifies a central difference between Hamiltonian pretraining in electronic structure and in quantum many-body few-shot learning. In HMAE, the Hamiltonian is the static object to be reconstructed. In HamEvo, the pretraining target is the update map whose fixed point is the Hamiltonian. The commonality is still Hamiltonian-native learning, but the object of pretraining shifts from representation to dynamics.

5. Structured warm starts and Hamiltonian-aware initialization

A narrower but practically important meaning of Hamiltonian pretraining is structured initialization from a simpler Hamiltonian model. In output-error port-Hamiltonian neural networks, one proposal is to estimate a linear port-Hamiltonian system first and then embed it as the initialization of the nonlinear PHNN. The nonlinear model is constructed as a linear PH backbone plus neural residuals: $$\hat H^{(k)} = \sum_{i\in \bigcup_{j=1}^{k} P_j}\hat H_i,$$9

$$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$0

$$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$1

A crucial implementation step is to initialize the final layers of the residual subnetworks to zero, so that the model starts exactly at the linear PH estimate rather than at a perturbed version of it. The encoder is then pretrained using only the linear PH model before full nonlinear training (Otterdijk et al., 27 Jan 2026).

In the reported chained mass-spring-damper experiments, this linear-PH initialization reduces spread across runs and lowers test NRMSE relative to both the purely linear model and the original nonlinear PHNN. Across 10, 20, 30, and 40 dB SNR, the initialized method reports mean NRMSEs of $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$2, $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$3, $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$4, and $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$5, with corresponding minimum NRMSEs of $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$6, $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$7, $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$8, and $$\min_{\theta}\; \langle \psi(\theta)|\hat H^{(k)}|\psi(\theta)\rangle,$$9 (Otterdijk et al., 27 Jan 2026).

An older and more general port-Hamiltonian view places the optimization process itself inside a Hamiltonian system. Parameters $k$0 and generalized momentum $k$1 define a state $k$2, and training proceeds by dissipative port-Hamiltonian dynamics on an augmented objective

$k$3

For the identity-mass case, the resulting second-order training law becomes

$k$4

This is presented as a full training framework rather than a pretraining method, but it naturally supports a finite-horizon warm-start interpretation: a PH flow can be run for a limited time to produce a refined initialization $k$5 before switching to another optimizer (Massaroli et al., 2019).

These initialization-based usages are narrower than self-supervised Hamiltonian representation learning, but they preserve the same structural principle: the starting point of the model is itself Hamiltonian-consistent.

6. Neighboring paradigms, misconceptions, and limitations

A common misconception is to treat every Hamiltonian-structured model as an instance of Hamiltonian pretraining. The literature draws sharper distinctions. Symplectic HNN training, for example, is primarily about choosing a loss induced by a symplectic integrator so that there exists an exact learnable modified Hamiltonian. The key loss has the form

$k$6

and its role is to replace an inconsistent forward-Euler objective with a realizable symplectic one (David et al., 2021). Likewise, later work on generalized Hamiltonians under noisy trajectories focuses on implicit midpoint rollouts and symplectic adjoint training, again as a structure-preserving training method rather than a pretraining phase (Choudhary et al., 25 Jun 2026).

A second misconception is to collapse pretraining into any fast Hamiltonian model fitting. Two random-feature lines make the opposite claim explicitly: their methods are not pretraining in the standard sense. In random-feature HNNs and Hamiltonian graph networks, hidden features are sampled once and frozen, and the output layer is solved by least squares. The central fitting rule is

$k$7

with gradients taken with respect to inputs, not parameters. These papers present backpropagation-free or gradient-descent-free training replacements, not learned initializations or pretrain-then-finetune pipelines (Rahma et al., 2024, Rahma et al., 6 Jun 2025).

A third neighboring perspective is theoretical rather than algorithmic. The Hamilton–Jacobi interpretation of deep learning argues that training can be understood as a search over Hamilton–Jacobi initial-value problems, with the weights encoding the initial data of a viscous HJ equation. In that view, pretraining can plausibly be interpreted as learning reusable initial data or an initial-data measure. This is a strong conceptual foundation for Hamiltonian pretraining, but it is not itself a named pretraining recipe (Miñoza et al., 27 May 2026). Similarly, Hamiltonian DNN frameworks derived from marginally stable continuous-time systems provide architectural reasons why Hamiltonian parameterizations may be attractive for staged training or transfer, but they do not introduce a pretraining procedure (Galimberti et al., 2021).

The main limitations reported across this literature are substantial. Objective-side curricula such as SHA require a decomposable Hamiltonian and incur extra optimization overhead; their benefits are partition-dependent and not uniformly positive across ansätze (Roshani et al., 2024). Self-supervised Hamiltonian pretraining for spin systems still depends on expensive exact labels during fine-tuning and is currently limited to small qubit counts (Shihab et al., 6 May 2025). SCF-trajectory pretraining requires rich DFT reference data, iterative fixed-point inference, and a fixed AO basis, even though it offers strong transfer gains (Lou et al., 12 Jun 2026). Large-scale electronic Hamiltonian learning shows that direct Hamiltonian supervision is feasible on systems with up to 3,000 atoms and $k$8 error in the eigenvalue spectra, but this is supervised local Hamiltonian regression rather than pretraining, and it remains basis- and workflow-dependent (Xia et al., 31 Jan 2025).

Taken together, these results suggest that Hamiltonian pretraining is best viewed as an emerging umbrella concept. Its most established forms are objective-side curricula in variational quantum optimization, self-supervised learning on Hamiltonians as data, SCF-dynamics pretraining for electronic Hamiltonians, and Hamiltonian-aware structured initialization. What unifies them is not a single loss or architecture, but the decision to make the Hamiltonian the native object of pretraining rather than merely the hidden generator of downstream labels.