Force Field Pre-Training

• Force field pre-training is the initialization of MLFFs using large-scale upstream data—like energies, forces, and denoising objectives—to enhance model robustness and transferability.
• It employs a variety of methodologies including supervised learning, self-supervised denoising, and multi-fidelity schemes to capture detailed interatomic interactions.
• The approach improves molecular dynamics stability and accuracy by broadening configurational coverage and harnessing both equilibrium and off-equilibrium data.

Searching arXiv for papers on force field pre-training and closely related MLFF pre-training methods. Searching for the core paper and related denoising, unified pre-training, universal force-field, and multi-fidelity work. Force field pre-training denotes the initialization of a machine-learning force field (MLFF) or machine-learned interatomic potential on a large upstream corpus, auxiliary physical objective, or cheaper label source before downstream fine-tuning on a target domain. In contemporary atomistic modeling, the upstream signal may be direct supervision on energies, forces, and stresses; denoising objectives that are explicitly connected to score matching and force learning; force-centric objectives spanning equilibrium and off-equilibrium conformations; or universal-model workflows that are later specialized by fine-tuning and distillation. Across molecular and materials settings, the central motivation is not only lower prediction error, but also broader configurational coverage, improved data efficiency, and greater robustness under molecular dynamics (MD), including regimes where low force mean absolute error (MAE) alone is insufficient to guarantee stable simulation (Maheshwari et al., 17 Jun 2025).

1. Scope of the paradigm

Force field pre-training has emerged at the intersection of molecular representation learning, interatomic potentials, and transfer learning. In the molecular setting, early formulations treated pre-training as supervised learning on large energy-labeled datasets with subsequent transfer to downstream force-field or property tasks (Gao et al., 2022). In parallel, denoising-based pre-training was given a physical interpretation: when equilibrium structures are corrupted with Gaussian noise and the model is trained to predict the perturbation, the objective is equivalent to learning the score of a smoothed molecular distribution, and therefore an approximate force field (Zaidi et al., 2022).

Subsequent work diversified the upstream signal. Some methods remained fully supervised and optimized forces directly on large off-equilibrium datasets, while regularizing equilibrium data with zero-force and denoising terms (Feng et al., 2023). Others emphasized unlabeled geometry collections from classical molecular dynamics, using masking, denoising, and contrastive objectives to improve downstream MLIPs with limited ab initio labels (Cui et al., 2023). Materials-oriented research extended the same logic to chemically broad universal models trained on millions of density-functional-theory (DFT) structures or, alternatively, on cheaply computed structural fingerprints rather than quantum-mechanical labels (Xie et al., 2024, Jia et al., 3 Mar 2025).

Accordingly, “pre-training” in this literature is not a single recipe. It includes at least four recurrent forms: supervised pre-training on energies and forces; self-supervised denoising or score matching; universal model training followed by task- or material-specific fine-tuning; and multi-fidelity schemes in which cheap labels, classical force fields, or lower-level quantum methods are used upstream before refinement on expensive targets (Gardner et al., 17 Jun 2025).

2. Supervised and force-centric formulations

A direct supervised formulation was presented in “Supervised Pretraining for Molecular Force Fields and Properties Prediction” (Gao et al., 2022). There, neural networks are pre-trained on a dataset of 86 millions of molecules with atom charges and 3D geometries as inputs and molecular energies as labels. For force-field transfer, the pre-training loss combines an energy term with a force-regularization term that penalizes the norm of the predicted energy gradient at optimized geometries,

$$\mathcal{L}_\mathrm{pretrain, FF} = (1 - \alpha) \mathcal{L}_E + \alpha \mathcal{L}_{\partial \hat{E} / \partial R}.$$

The reported outcome is improved performance on seven molecular property prediction tasks and two force field tasks, together with linear-probe evidence that the learned representations encode atom types, interatomic distances, molecular scaffolds, and molecular fragments (Gao et al., 2022).

A more explicitly force-oriented construction was introduced by ET-OREO, a unified force-centric pre-training model for 3D molecular conformations (Feng et al., 2023). It is pre-trained on over 15 million conformations from MD17, ANI1-x, PCQM4Mv2, and poly24. Off-equilibrium conformations are learned from direct force supervision, while equilibrium conformations are handled through zero-force regularization and a force-based denoising term:

$$\Vert -\nabla\Phi_\theta(\mathbf{r}_x)\Vert_2^2, \qquad \mathbb{E}_{\varepsilon\sim \mathcal{N}(0, \sigma^2)} \left[ \left\Vert \nabla_{\mathbf{r}_x}\Phi(\mathbf{r}_x-\varepsilon) - \varepsilon \right\Vert_2^2 \right].$$

Empirically, this pre-training objective increases force accuracy by around 3 times compared to the un-pre-trained Equivariant Transformer model, solves unstable MD behavior in vanilla Equivariant Transformers, achieves state-of-the-art simulation performance with stable trajectories up to 300 ps, and attains 2.45 times faster inference time than NequIP (Feng et al., 2023).

A related supervised transfer result was obtained with GemNet-T in “Beyond Force Metrics: Pre-Training MLFFs for Stable MD Simulations” (Maheshwari et al., 17 Jun 2025). Two strategies were compared on Aspirin from MD17: direct training on 9,500 configurations and pre-training on 2 million OC20 S2EF structures followed by fine-tuning on the same MD17 subset. Both models reached force MAE of approximately 5 meV/Å per atom, but the pre-trained model yielded markedly longer stable trajectories. This case became a focal example because the principal gain of pre-training appeared in simulation robustness rather than in held-out force error (Maheshwari et al., 17 Jun 2025).

3. Denoising, score matching, and physics-informed self-supervision

The denoising line of work supplied a statistical-mechanical interpretation for pre-training on equilibrium geometries. “Pre-training via Denoising for Molecular Property Prediction” models the observed molecular distribution as a mixture of Gaussians centered at equilibrium structures,

$$q_\sigma(\mathbf{x}) = \frac{1}{n} \sum_{i=1}^n \mathcal{N}(\mathbf{x}; x_i, \sigma^2 I),$$

and shows that training a GNN to predict Gaussian perturbations is equivalent to score matching on $q_\sigma(\mathbf{x})$, hence to learning an approximate force field (Zaidi et al., 2022). The downstream effect was substantial: the pre-trained model achieved a new state-of-the-art on the majority of targets in QM9, and the detailed results report state-of-the-art on 10 out of 12 targets (Zaidi et al., 2022).

Later work argued that isotropic coordinate noise is too restrictive because real molecular flexibility is anisotropic. Frad addressed this by combining dihedral angle noise with coordinate noise, while denoising only the coordinate component:

$$\mathcal{L}_{Frad} = \mathbb{E}_{p(\tilde{x}|x_a)p(x_a|x_i)p(x_i)} \left\| GNN_{\theta}(\tilde{x}) - (\tilde{x} - x_a) \right\|^2.$$

This preserves force-learning equivalence while expanding coverage of low-energy conformations along chemically realistic directions. The original Frad paper reported new state-of-the-art on 9 out of 12 tasks of QM9 and on 7 out of 8 targets of MD17 (Feng et al., 2023), and the later framework paper described customizable chemical-aware noise that established state-of-the-art results across force prediction, quantum chemical properties, and binding affinity tasks (Ni et al., 2024).

SliDe pushed the same theme toward explicit intramolecular mechanics. It perturbs bond lengths, angles, and torsions according to a classical mechanical intramolecular potential and then uses a random slicing trick to avoid materializing the Jacobian needed to map internal-coordinate gradients into Cartesian force targets (Ni et al., 2023). On 1,000 molecules from PCQM4Mv2, its estimated force field reached a correlation coefficient of 0.895 compared with 0.631 for Frad and 0.616 for Coord, described as a 42\% improvement over previous denoising methods. The same study reports state-of-the-art mean MAE on 10 out of 12 QM9 properties and best-in-class MD17 performance in the low-data regime (Ni et al., 2023).

A more geometry-centric but still self-supervised variant is GPIP, which first generates large numbers of unlabeled 3D configurations using classical MD and then applies masking, denoising, and contrastive learning (Cui et al., 2023). The method was evaluated on MD17, ISO17, liquid water, and electrolyte solutions, and the reported numbers include reductions such as SchNet on Aspirin from 1.35 to 0.85 kcal/mol/Å and SchNet on ISO17 with 10,000 labeled configurations from 18.08 to 9.95 (Cui et al., 2023). This family of results suggests that self-supervised pre-training can be effective even when the upstream data are not labeled by ab initio calculations, provided that the objective preserves informative geometric structure.

4. Data scale, universality, and materials-oriented pre-training

Materials MLFFs generalized the pre-training idea from molecular benchmarks to wide chemical spaces. ALIGNN-FF trained a unified atomistic line graph neural network force field on a subset of 307,113 structures drawn from the JARVIS-DFT dataset, which contains around 75,000 materials and 4 million energy-force entries (Choudhary et al., 2022). The model covers any combination of 89 elements from the periodic table and reports test MAE of 0.086 eV/atom for energies and 0.047 eV/Å for forces. The paper emphasizes transferability across chemically and structurally diverse materials and demonstrates structure optimization and genetic-algorithm crystal search (Choudhary et al., 2022).

GPTFF scaled this universal strategy much further. It is trained on 2,234,767 unique compounds with 37.8 million single-point energies, 11.7 billion force pairs, and 340.2 million stresses, covering both 349,043 equilibrium and 37.4M off-equilibrium states (Xie et al., 2024). The reported MAEs are 32 meV/atom for energy, 71 meV/Å for force, and 0.365 GPa for stress, and the model is presented as a graph-based pre-trained transformer force field for arbitrary inorganic materials (Xie et al., 2024). Its case studies include crystal structure optimization, phase transition simulations, and ion transport.

Not all materials pre-training requires quantum-mechanical labels. “Pre-training Graph Neural Networks with Structural Fingerprints for Materials Discovery” explicitly contrasts supervised force-field fitting and denoising on DFT-optimized structures with a fingerprint-target pre-training objective (Jia et al., 3 Mar 2025). Weighted atom-centered symmetry functions, Gaussian multipoles, and embedded atomic density are used as cheap structural targets. Across 11 downstream datasets, descriptor-based pre-training yields average MAE reductions of 15–17\%, with the best-performing descriptor, EAD, reaching an average 16.9\% reduction and a best-case improvement of 35\% (Jia et al., 3 Mar 2025). The paper describes this approach as slightly lower on average than pre-training with forces, but competitive, while reducing computational expense by several orders of magnitude and supporting large scale foundational models for atomistic data (Jia et al., 3 Mar 2025).

These materials results define an important extension of force field pre-training: the upstream task need not be identical to downstream energy-force prediction, provided that it induces representations aligned with local geometry, chemistry, and configurational variability.

5. Simulation stability and the limits of force MAE

A recurring misconception in the MLFF literature is that low test-set force MAE is a sufficient proxy for MD reliability. The clearest counterexample is GemNet-T on OC20→MD17 (Maheshwari et al., 17 Jun 2025). After 300 epochs, the RMS deviation between the test-force curves of the pre-trained and non-pre-trained models is approximately 0.42 meV/Å, yet the instability onset times differ sharply: 274.2 ps with standard deviation 44.6 ps for the pre-trained model, versus 42.6 ps with standard deviation 25.9 ps for the model trained from scratch. The former is described as very close to the theoretical maximum of 300 ps, whereas the latter fails about 6× sooner. Moreover, in the non-pre-trained case the force MAE continued improving at later epochs while the instability onset time decreased, which the authors interpret as overfitting to on-manifold train/test configurations and poor off-manifold generalization (Maheshwari et al., 17 Jun 2025).

The same pattern appears in ET-OREO. The paper reports that zero-force and denoising regularizations on equilibrium data are essential for stable and realistic MD trajectories and explicitly connects the failure mode of vanilla Equivariant Transformers to the inadequacy of force accuracy alone as a criterion (Feng et al., 2023). GPIP likewise reports that pre-training improves not only force MAE but also simulation stability, radial distribution function MAE, and diffusion-coefficient prediction in water and electrolytes (Cui et al., 2023).

A related extension uses temporal ordering in MD itself. FRAMES introduces an auxiliary loss on pairs of consecutive MD frames and finds that two-frame temporal information is often sufficient, while longer windows can degrade performance (Mollahosseini et al., 14 Apr 2026). On MD17, the best performing model is Equiformer + 2 Frames, and on ISO17 the reported in-distribution energy MAE is 0.00569 versus 0.13228 for the baseline Equiformer, while the outside-distribution energy MAE is 0.0248 versus 0.1460 (Mollahosseini et al., 14 Apr 2026). This suggests that robust force-field learning can benefit from trajectory structure without requiring temporal context at inference.

The common implication is that pre-training should be evaluated against explicit MD criteria, such as instability onset, structural distributions such as $h(r)$, or long-time trajectory behavior, rather than against force MAE in isolation.

6. Multi-fidelity, cheap teachers, and workflow extensions

As pre-training matured, a central question became how to exploit cheap but imperfect labels. “Understanding multi-fidelity training of machine-learned force-fields” compares sequential pre-training/fine-tuning with multi-headed training across multiple quantum-chemical methods (Gardner et al., 17 Jun 2025). The study finds that downstream performance correlates near-linearly with pre-training task performance, that pre-training on both energies and forces is much more effective than on energies alone, and that the fidelity gap matters: the mean absolute DFT-CC offset is approximately 18 meV/atom, whereas xTB-CC is approximately 47 meV/atom. A key mechanistic conclusion is that internal representations learned during pre-training are method-specific, so full-model fine-tuning at low learning rate is necessary; by contrast, multi-headed models learn method-agnostic representations and provide an extensible route toward universal MLFFs, albeit with a slight accuracy compromise compared to sequential fine-tuning (Gardner et al., 17 Jun 2025).

A different inexpensive-teacher strategy uses classical force fields. In “Teachers that teach the irrelevant,” pre-training is performed on low-quality, practically free, single-molecule non-reactive force field data, while all intermolecular interactions and reactive properties are learned only during fine-tuning (Yuan et al., 17 Sep 2025). The pre-training data are generated by rattling coordinates with Gaussian noise of 0.2 Å, typically producing 100,000+ uncorrelated geometries per molecule, and the same weighted energy-force loss is used in both stages with $w_E = 1$ and $w_F = 20$. The reported outcome is stable and accurate molecular dynamics and metadynamics simulations of gas phase molecules, liquid water, and hydrogen combustion reactions compared to models trained from scratch (Yuan et al., 17 Sep 2025).

Workflow-level specialization appears in PFD, which treats a universal model as a starting point for automatic generation of material-specific force fields through fine-tuning and distillation (Wang et al., 28 Feb 2025). The paper reports one to two orders of magnitude less training data than traditional methods and up to 100x faster inference after distillation. For crystalline Si, the detailed example uses 33 DFT data for fine-tuning and reports 1.5 h for fine-tuning and 1.5 h for distillation (Wang et al., 28 Feb 2025). In this setting, pre-training becomes part of a production pipeline rather than merely an initialization heuristic.

Finally, differentiable simulation broadens the downstream target beyond energies and forces. “Force field optimization by end-to-end differentiable atomistic simulation” uses JAX-MD, an inner simulation loop, and outer-loop automatic differentiation to optimize Stillinger-Weber, EDIP, and message-passing GNN potentials against elastic constants, phonon dispersion, vibrational density of states, and radial distribution functions (Gangan et al., 2024). The paper reports that machine-learned potentials fine-tuned on target properties such as RDFs exhibit improved accuracy and generalizability to unseen temperatures relative to models fine-tuned on energies and forces alone (Gangan et al., 2024). This suggests a broader interpretation of force field pre-training and fine-tuning: upstream objectives may establish a physically reasonable potential-energy surface, while downstream optimization can be directed at simulation observables that matter for deployment.

Across these strands, force field pre-training has shifted from a narrow transfer-learning convenience to a design principle for robust atomistic modeling. The field now spans supervised and self-supervised objectives, molecular and materials scales, universal and specialized models, and both single- and multi-fidelity data regimes. The strongest shared conclusion is not merely that pre-training improves sample efficiency, but that it changes what is learned: configurational priors, physically constrained local geometry, and out-of-distribution behavior that become visible only when the model is embedded in actual simulation (Maheshwari et al., 17 Jun 2025).