Synthetic pre-training of graph-network models for predicting solid-state NMR parameters

Abstract: Nuclear magnetic resonance (NMR) is a powerful probe of atomic structure, but accurate quantum-mechanical predictions of tensorial NMR parameters are computationally demanding. This creates a bottleneck both for direct quantum-mechanical studies and for collecting high-quality training data for machine-learning (ML) models. Here, we introduce a synthetic pre-training and fine-tuning protocol for graph-based ML models of solid-state NMR parameters. We first pre-train models on synthetic tensorial data, as obtained using an existing ML model, and subsequently fine-tune those models on new ground-truth data. We observe a pronounced improvement in data efficiency when pre-training and fine-tuning span the same compositional and configurational space, and we carry out initial experiments regarding chemical transferability. Our work outlines a route toward future data-efficient training workflows for tensorial ML models for solid-state NMR, combining inexpensive synthetic supervision with targeted first-principles refinement.

• The paper proposes a synthetic pre-training approach using the TensorMACE architecture to enhance predictions of NMR tensor components with 3–10× data efficiency gains.
• It utilizes synthetic NMR labels generated via a NequIP-based model for pre-training, followed by fine-tuning on limited high-quality QM data to significantly reduce RMSE errors.
• The study demonstrates robust transferability across chemical domains and highlights the key role of model capacity in achieving accurate, orientation-resolved tensorial predictions.

Synthetic Pre-training of Graph Neural Networks for Solid-State NMR Parameter Prediction

Introduction

The quantum-mechanical prediction of tensorial NMR (nuclear magnetic resonance) parameters, particularly magnetic shielding (MS) and electric field gradient (EFG) tensors, plays a critical role in the interpretation of solid-state NMR spectra and the validation of structural models. The computational expense associated with producing high-quality quantum mechanical (QM) reference data acts as a bottleneck in generating sufficient training labels for ML models targeting these tensorial properties. This paper "Synthetic pre-training of graph-network models for predicting solid-state NMR parameters" (2606.11038) addresses this challenge by proposing and systematically evaluating a synthetic pre-training and fine-tuning strategy for equivariant graph neural network (GNN) models, focusing on the prediction of solid-state NMR tensorial parameters in inorganic solids.

Methodology

TensorMACE Model Architecture

The proposed approach is built upon an E(3)-equivariant message passing neural network architecture, specifically an extension of the MACE framework termed TensorMACE. TensorMACE represents atom-centered environments as graphs with physically motivated equivariant features, expanded via angular and radial basis functions, and processed through iterative message passing. The model predicts the full spherical tensor decomposition of the MS tensor: isotropic ($o^{(0)}$), antisymmetric ($o^{(1)}$), and traceless symmetric ($o^{(2)}$) components, thereby guaranteeing rotational covariance of the output. The architecture utilizes explicit tensor couplings to achieve a general and symmetry-consistent representation of tensorial targets, overcoming parity-related feature constraints of standard implementations.

Datasets and Training Protocols

The study uses three core datasets: amorphous SiO$_2$ (a-SiO$_2$) with both synthetic and QM-computed labels, and a chemically diverse Si-O-Al-H zeolitic dataset. Synthetic NMR labels are generated using a NequIP-based model trained on limited QM calculations, providing an approximate—yet computationally inexpensive—source of tensorial supervision. Models are first pre-trained on large pools of structures with synthetic labels, and subsequently fine-tuned on small sets of high-quality QM-labeled data, allowing for isolation of the effects attributable to pre-training.

Multiple architectural scales (large and small TensorMACE) are tested, differing in tensor feature rank. Loss minimization is performed on spherical tensor components, including per-element learnable offsets. Model evaluation is reported using %RMSE, which facilitates direct comparison across isotropic and anisotropic quantities for different nuclear sites.

Numerical Results

Data Efficiency and Pre-training Impact

Synthetic pre-training consistently reduces prediction errors for both isotropic and anisotropic MS tensor components compared to direct training on QM data alone. Data efficiency gains are most pronounced in the strong data-scarce regime: models pre-trained on 5,000 synthetically labeled a-SiO$_2$ structures and fine-tuned with as few as 10 QM structures match or exceed the performance of directly trained models using 250 QM structures. Specifically, fine-tuned TensorMACE achieves RMSE values of 0.6 ppm (Si) and 1.58 ppm (O) for $o^{(0)}$, outperforming direct models (1.1 ppm and 2.8 ppm, respectively), and achieves a comparable reduction for $o^{(2)}$.

Influence of Synthetic Label Quality and Model Capacity

Pre-training model performance is robust with respect to the synthetic label generator’s absolute accuracy. Even lower-quality synthetic labels produced via a model trained on only 50 QM configurations significantly enhance downstream data efficiency: models fine-tuned on 50 QM structures rival the performance of direct training on 250 QM structures, representing a five-fold reduction. Diminishing returns are observed beyond ~1,000 pre-training structures, indicating rapid saturation of useful geometric-correlation knowledge.

Model capacity impacts both baseline prediction accuracy and the magnitude of the pre-training benefit. Larger models with higher equivariant feature rank yield lower errors, especially for anisotropic and orientation-dependent tensorial properties. Synthetic pre-training partially compensates for reduced architectural capacity by guiding parameter optimization toward physically meaningful basins, but is ultimately limited by model expressivity.

Fidelity of Tensorial Predictions

Pre-training notably improves orientation-resolved fidelity of $o^{(2)}$ predictions. Spherical error maps reconstructed for test atoms demonstrate that pre-trained models exhibit uniformly reduced directional errors compared to direct models, particularly along MS tensor principal axes—especially valuable for accurate spectral simulation of powder and anisotropic systems.

Transferability and Chemical Domain Generalization

Transfer experiments from a-SiO$_2$ to Si-O-Al-H zeolite systems exhibit that synthetic pre-training encodes transferable geometric-response correlations. For common elements (Si, O), pre-trained models fine-tuned with limited QM data outperform direct models throughout the data-scarce regime and achieve stronger gains for anisotropic properties ($o^{(1)}$0, derived Haeberlen parameters). For elements absent in the pre-training dataset (Al, H), improvements are smaller but persist when local environments are geometrically similar. RMSE reductions across all elements underscore partial transferability of geometry-driven features, while element- and chemistry-specific effects are refined during fine-tuning.

Implications and Future Directions

This work establishes synthetic pre-training as an effective strategy to alleviate QM data scarcity in tensorial property prediction, providing 3–10× data efficiency gains in practical NMR applications. The approach accelerates model convergence and enhances robustness, particularly for higher-rank tensor components and in low-capacity models. The transferability studies further advocate for the utility of geometry-driven pre-training in facilitating rapid adaptation of ML models to new chemical and structural domains.

The results motivate the broader development of data-efficient, hierarchical training workflows combining synthetic supervision (potentially from MLIPs or less-accurate ML NMR models) with targeted high-accuracy refinement, paralleling foundational model paradigms emerging in ML-related interatomic potentials. The primary limitation for NMR-targeted foundational models remains the patchy availability of QM NMR parameters, especially for less-explored inorganic and metallic systems. Automated, scalable generation of synthetic labels and expansion of model chemistry/structure coverage represent key avenues for future research.

Conclusion

Synthetic pre-training of equivariant GNNs provides a substantial advance in data-efficient learning of tensorial NMR properties, particularly for solid-state materials where direct QM data are limited or costly. This strategy enables accurate, robust, and partially transferable models for both simple and chemically diverse environments, narrowing the performance gap imposed by architectural limitations and supporting the development of broadly applicable, ML-driven NMR simulation tools. Future progress will depend on extending workflows for generating synthetic tensorial labels and further enhancing cross-domain transferability in atomistic machine learning.