Iterative Pretraining for Interatomic Potentials

• The paper demonstrates how iterative pretraining leverages teacher-student cycles to update ML interatomic potentials without repeated expensive DFT calculations.
• It employs cyclic data regeneration with a controlled forgetting mechanism to avoid bias from stale pseudo-labels and local minima.
• Empirical results on datasets like MD22, Chignolin, and Mo–S–O reveal significant improvements in prediction accuracy and simulation stability.

Searching arXiv for papers on IPIP and related iterative pretraining / transfer-learning approaches for interatomic potentials. {"query":"Iterative Pretraining Framework for Interatomic Potentials arXiv (Cui et al., 27 Jul 2025) GPIP (Cui et al., 2023) meta-learning interatomic potentials (Allen et al., 2023) universal interatomic potentials configuration-space generators (Hänseroth et al., 22 Jun 2026)", "max_results": 10} Iterative Pretraining for Interatomic Potentials (IPIP) denotes a class of staged or cyclic training strategies for machine-learning interatomic potentials (MLIPs) in which a model is not fit once to a fixed high-fidelity dataset, but is instead improved through repeated pretraining, relabeling, and refinement steps. In its explicit formulation, IPIP uses a strong foundation MLIP as a teacher, a lighter student architecture as the deployable model, iterative molecular-dynamics-driven data regeneration, and a forgetting mechanism that prevents the training process from collapsing onto early pseudo-labels or suboptimal local minima (Cui et al., 27 Jul 2025). More broadly, the term also covers closely related workflows in which unlabeled classical molecular dynamics, lower-fidelity force fields, meta-learning across quantum-chemical tasks, or model-guided exploration are used to construct successive pretraining stages before or alongside limited density-functional-theory (DFT) or higher-level fine-tuning (Cui et al., 2023).

1. Definition and formal setting

In the standard MLIP setting, an atomic configuration is represented by atomic numbers and positions,

$$Z = \{z_1,\dots,z_n\}, \qquad R = \{r_1,\dots,r_n\} \in \mathbb{R}^{n\times 3},$$

with a scalar potential energy $E(Z,R)$ and forces

$F_i = -\nabla_{r_i} E(Z,R).$

For graph-based models, atom features are initialized from atomic-number embeddings and propagated through message-passing layers; the total energy is then written as a sum of atomic contributions,

$\hat{E} = \sum_{i=1}^{n} \mathrm{MLP}(x_i^L).$

This conservative-energy formulation allows forces to be obtained by automatic differentiation and is the basis of the IPIP implementations built on ViSNet and PaiNN (Cui et al., 27 Jul 2025).

Within this formalism, IPIP differs from both training from scratch and one-shot pretrain–fine-tune transfer. Training from scratch fits a model directly to a labeled target dataset. One-shot transfer first pretrains on a source dataset and then fine-tunes once on the target. IPIP instead rebuilds the pretraining dataset across iterations, alternates between teacher-labeled and student-labeled supervision, and repeatedly re-optimizes the student on a changing data distribution that is shaped by the student’s own molecular dynamics trajectories (Cui et al., 27 Jul 2025).

The central motivation is that high-accuracy MLIPs typically require extensive DFT data, yet broad configuration-space coverage is expensive, especially for reactive or structurally heterogeneous systems. IPIP addresses this by shifting much of the structural exploration and representation formation into cheaper stages, while using a small fixed DFT set only to calibrate the final energy and force predictions for the target system (Cui et al., 27 Jul 2025).

2. Historical development and related paradigms

IPIP emerged within a broader methodological trajectory in which structural learning, cheap supervision, and iterative refinement were progressively decoupled from direct ab initio labeling. One important precursor is geometry-enhanced pretraining on interatomic potentials, which proposed a two-stage paradigm: first generate large unlabeled configuration pools with classical molecular dynamics, then pretrain invariant or equivariant graph neural networks with masking, denoising, and contrastive objectives before fine-tuning on limited ab initio labels. That approach improved data efficiency and transferability across MD17, ISO17, liquid water, and electrolyte systems, and it supplied much of the geometric pretraining logic later associated with IPIP-like workflows (Cui et al., 2023).

A second line of development treated “pretraining” as meta-learning over multiple potential-energy surfaces. Reptile-based meta-training across datasets and quantum levels was used to learn initializations that can be rapidly adapted to new levels of theory or new molecules with limited data. In that setting, tasks correspond to datasets or quantum-chemical levels, and the iterative inner-loop/outer-loop optimization itself functions as a principled form of iterative pretraining across heterogeneous PES definitions (Allen et al., 2023).

A third lineage came from cyclic data-generation systems in atomistic simulation. FLAME implemented an automated train–explore–retrain loop in which an initial neural-network interatomic potential is trained on DFT-relaxed bulk-derived structures, used to drive minima hopping for crystal-structure exploration, then retrained on newly selected DFT-labeled configurations. The entire workflow is iterative, diversity-filtered, and minimally supervised, making it an early explicit realization of cyclic MLIP refinement for inorganic materials (Mirhosseini et al., 2021).

Other adjacent formulations broadened the meaning of iterative pretraining. Ensemble-guided iterative refinement for reactive hydrogen dynamics on copper surfaces used uncertainty during scattering simulations to request new DFT labels until reaction probabilities converged, thereby tying iterative dataset expansion directly to target observables rather than only to static errors (Stark et al., 2023). Transfer learning for Gaussian Moment Neural Networks showed that DFT pretraining followed by high-level fine-tuning can yield chemically accurate forces even under energy-only fine-tuning, provided coverage is adequate, which supplied a strong argument for staged data-fidelity progression (Zaverkin et al., 2022). More recent variants include physics-informed pretraining from embedded-atom models to improve stability (Zheng et al., 23 Feb 2026), classical-to-quantum transfer learning with PaiNN for data-efficient GNN MLIPs (Pelletier et al., 25 Mar 2026), and universal-MLIP-guided DFT relabeling loops in which universal models act as configuration-space generators rather than final energy models (Hänseroth et al., 22 Jun 2026).

3. Core algorithmic structure of IPIP

In its explicit formulation, IPIP is a teacher–student loop. A universal or foundation MLIP serves as the teacher, while a lighter first-order equivariant student such as ViSNet or PaiNN is the deployable target model. The first stage constructs an initial pseudo-labeled pretraining dataset by running molecular dynamics with the teacher and recording teacher-predicted energies and forces for the sampled configurations. The student is pretrained on this dataset and then fine-tuned on a small, fixed DFT dataset for the target system (Cui et al., 27 Jul 2025).

After this initialization, the workflow becomes iterative. The current student drives new molecular dynamics simulations from representative initial configurations. Rather than merely collecting arbitrary snapshots, the method preferentially harvests “edge conformations,” particularly from trajectories that show instabilities or other indications that the current student is weak in those regions of configuration space. These newly generated configurations are then re-annotated by the teacher model, yielding fresh pseudo-labels for the next pretraining round (Cui et al., 27 Jul 2025).

The distinctive feature of IPIP is its forgetting mechanism. At iteration $k$, only 90% of the previous pretraining set is retained; the remaining 10% is randomly discarded. The retained 90% is not kept with stale labels: it is relabeled by the current student, while the new student-generated configurations are labeled by the teacher. In schematic form, if $\mathcal{D}_{k-1}$ is the previous pretraining set, the updated dataset is

$$\mathcal{D}_k = \tilde{\mathcal{D}}_{k-1}^{(S)} \cup \mathcal{D}_k^{\mathrm{new}},$$

where $\tilde{\mathcal{D}}_{k-1}^{(S)}$ denotes the retained subset relabeled by the student and $\mathcal{D}_k^{\mathrm{new}}$ denotes the new configurations labeled by the teacher (Cui et al., 27 Jul 2025).

This mechanism prevents iterative training from remaining anchored to early pseudo-labels or initial sampling bias. It also creates a moving target distribution: earlier data are gradually reinterpreted through the current student’s representation, while newly explored configurations inject harder cases from the student’s own deployment regime. No additional DFT is required inside this loop; the DFT dataset used for fine-tuning remains fixed. In this sense, IPIP is neither conventional active learning nor pure self-training, but a hybrid system that mixes teacher supervision, student-induced distribution shift, and controlled forgetting (Cui et al., 27 Jul 2025).

4. Objectives, architectures, and supervision levels

IPIP itself does not replace standard energy–force training objectives; it changes how the training data are assembled and refreshed. For ViSNet, the supervised objective is an energy–force mean-squared loss,

$$\mathcal{L}_{\mathrm{ViSNet}} = \frac{\alpha}{N}\sum_{i=1}^{N}\left(E^{(i)}-\hat{E}^{(i)}\right)^2 + \frac{\beta}{n}\sum_{j=1}^{n}\left\|\mathbf{F}^{(j)}-\hat{\mathbf{F}}^{(j)}\right\|^2,$$

where $E(Z,R)$0 is the number of configurations and $E(Z,R)$1 the number of atoms per configuration. For PaiNN, the paper uses an MAE-based energy–force objective,

$E(Z,R)$2

In both cases, energies and forces are learned jointly, and forces are derived from the energy gradient to preserve conservativity (Cui et al., 27 Jul 2025).

The architectural emphasis in the explicit IPIP work is on lightweight first-order equivariant models rather than large higher-order tensor architectures. ViSNet and PaiNN are chosen because they are substantially cheaper at inference than models such as MACE-OFF, yet remain accurate enough to benefit from iterative specialization. The teacher–student asymmetry is deliberate: the teacher can be broad and expensive, while the student is intended to become system-optimized and fast for production molecular dynamics (Cui et al., 27 Jul 2025).

Related iterative-pretraining schemes differ mainly in the source of the pretraining signal. Geometry-enhanced pretraining uses unlabeled classical-MD configurations and self-supervised objectives rather than pseudo-labeled teacher outputs; its total pretraining loss combines masking, denoising, and contrastive terms,

$E(Z,R)$3

before downstream energy–force fine-tuning (Cui et al., 2023). Classical-to-quantum transfer learning with T-PaiNN instead pretrains PaiNN on large datasets labeled by classical force fields such as UFF or TIP3P, then fine-tunes all model weights on smaller DFT datasets, using the coarse PES as a representation prior (Pelletier et al., 25 Mar 2026). Physics-informed pretraining with embedded-atom models follows a similar two-stage logic, but its emphasis is robustness and short-range repulsion rather than teacher–student cycling (Zheng et al., 23 Feb 2026).

These variants show that “pretraining” in MLIPs can mean at least four distinct things: self-supervised geometric representation learning, supervised transfer across fidelity levels, meta-learning across PES tasks, and iterative pseudo-label refresh under a teacher–student loop. IPIP, in the narrow sense, refers to the last of these, but it inherits methodological ideas from all four.

5. Benchmarks, applications, and empirical behavior

The explicit IPIP framework was evaluated on MD22, Chignolin, and a reactive Mo–S–O system. On MD22, ViSNet with IPIP improved over baseline training, geometry-focused pretraining, and prior distillation strategies. Representative energy/force MAEs include Ac-Ala3-NHMe, where the energy error decreased from 0.0896 for the baseline and 0.0744 for Frad to 0.0572 for IPIP, and the Buckyball catcher, where the force error decreased from 0.1843 for the baseline to 0.1054 under IPIP. Against distillation baselines on large systems, IPIP also yielded markedly lower errors, such as 0.2727/0.1054 energy/force MAE on Buckyball catcher compared with 0.4032/0.1548 for Hessian distillation (Cui et al., 27 Jul 2025).

On Chignolin, a 166-atom mini-protein with roughly 2,000,000 DFT-labeled configurations available, three iterations of IPIP improved accuracy across training subsets of 20,000, 100,000, and 600,000 structures. The key point was not only asymptotic accuracy but that iterative pseudo-augmentation remained beneficial even in lower-data regimes, indicating that the gains were not merely a consequence of large-scale supervision (Cui et al., 27 Jul 2025).

The Mo–S–O case is the canonical specialization benchmark. The abstract reports over 80% reduction in prediction error and up to 4× speedup relative to general-purpose force fields, reflecting the combination of lightweight students and system-specific iterative refinement (Cui et al., 27 Jul 2025). In reactive molecular dynamics of a $E(Z,R)$4 system, a baseline PaiNN trained directly on the DFT data exhibited numerical instability in 13% of 100 independent high-temperature simulations, whereas the IPIP-trained PaiNN completed all simulations without reported failures. Long trajectories further showed stable sulfiding pathways leading to $E(Z,R)$5-like products, illustrating that the iterative exposure to edge conformations altered not just static test errors but deployment-time stability (Cui et al., 27 Jul 2025).

Closely related staged-pretraining studies reinforce the same pattern. T-PaiNN showed that PaiNN pretrained on classical force-field data and fine-tuned on limited DFT data can reduce energy MAE on QM9 by up to 25.3× in the $E(Z,R)$6 low-data regime and cut force MAE for liquid water roughly in half relative to DFT-only training, while also improving density, diffusion, and RDF agreement in water simulations (Pelletier et al., 25 Mar 2026). Universal-MLIP-guided one-shot and iterative fine-tuning found that about 2,000 DFT-relabeled configurations were often sufficient for accurate material-specific models, while the challenging MoS$E(Z,R)$7 sulfur-vacancy problem required iterative self-training but could recover the DFT barrier with only 600 first-principles calculations in total (Hänseroth et al., 22 Jun 2026). These results do not instantiate the exact IPIP algorithm of (Cui et al., 27 Jul 2025), but they support the general thesis that staged, iterative, or multi-fidelity pretraining materially improves sample efficiency for interatomic potentials.

6. Limitations, controversies, and future directions

A central limitation of IPIP is that its improvements depend on the quality and relevance of the teacher. Foundation models can provide broad coverage, but they also embody a trade-off between generality and specialization. The explicit IPIP work is motivated by the observation that large universal models are often accurate yet computationally heavy and may remain under-specialized for a particular target system; the student can surpass them on-domain, but only insofar as the teacher provides useful pseudo-supervision and broad enough exploration (Cui et al., 27 Jul 2025).

Teacher bias is not the only concern. Related work on universal MLIPs as configuration-space generators showed that zero-shot models can soften barriers or miss reactive observables even when force and energy RMSEs look acceptable, especially for transport and high-barrier processes such as sulfur-vacancy migration in MoS$E(Z,R)$8 (Hänseroth et al., 22 Jun 2026). This implies that iterative pretraining can inherit a biased sampling prior if early trajectories do not visit the relevant transition regions. Similarly, meta-learning across multiple quantum levels found that direct transfer from the closest lower-level dataset can outperform broader multi-dataset pretraining when the target is tightly localized, so “more pretraining” is not automatically better if the pretraining distribution is off-target (Allen et al., 2023).

The forgetting mechanism in IPIP is heuristic. The reported framework discards 10% of the previous pretraining set at each iteration, but no theoretical account is given for why 10% is optimal, nor for how the forgetting rate should scale with teacher error, dataset diversity, or student capacity (Cui et al., 27 Jul 2025). This invites comparison with continual-learning questions such as catastrophic forgetting and replay-buffer design, but the explicit analysis remains open. Likewise, current IPIP does not integrate formal uncertainty quantification into configuration selection; edge conformations are identified through instabilities and heuristic trajectory filtering rather than ensemble variance or Bayesian criteria (Cui et al., 27 Jul 2025).

A broader methodological tension concerns objective alignment. Geometry-enhanced self-supervision improves data efficiency, but its pretraining targets are only indirectly related to downstream energies and forces (Cui et al., 2023). End-to-end backpropagation through structural relaxations can optimize final structures directly and reduce structural-relaxation error by nearly 50%, yet it may worsen local force fidelity, showing that optimizing deployment-time trajectories and optimizing local PES derivatives are not identical problems (Dramko et al., 30 Nov 2025). IPIP sits between these extremes: it keeps standard energy–force supervision, but its iterative dataset construction is meant to better align training data with the model’s actual deployment distribution.

Future directions therefore include integrating uncertainty-aware selection into the IPIP loop, replacing or augmenting teacher labeling with selective DFT calls, extending the framework to long-range interactions and more complex periodic materials, and combining iterative pseudo-label refresh with relaxation-level or observable-level objectives. Another likely direction is hierarchical multi-fidelity IPIP, in which classical force fields, universal MLIPs, DFT, and post-DFT data appear as successive stages rather than as a single teacher–student pair. The cumulative evidence across teacher–student iteration (Cui et al., 27 Jul 2025), geometric self-supervision (Cui et al., 2023), meta-learning (Allen et al., 2023), classical-to-quantum transfer (Pelletier et al., 25 Mar 2026), and universal-model-guided relabeling (Hänseroth et al., 22 Jun 2026) suggests that IPIP is best understood not as one isolated algorithm, but as a general organizing principle for building fast, specialized, and data-efficient interatomic potentials through repeated refinement of both model parameters and training distributions.