Parameter-Efficient Fine-Tuning of Machine-Learning Interatomic Potentials for Phonon and Thermal Properties

Abstract: Machine-learning interatomic potentials are widely used as computationally efficient surrogates for density functional theory in atomistic simulations, enabling large-scale, long-time modeling of materials systems. We investigate how different fine-tuning strategies influence the prediction of harmonic phonon band structures, thermal properties, and the potential energy surface along imaginary phonon modes. We achieve substantial accuracy improvements with minimal additional data, with as few as 10 additional training structures already yielding significant gains. In addition to existing approaches, we introduce Equitrain, a finetuning framework that implements LoRA-based adaptation. Across 53 materials systems, we show that fine-tuned models consistently outperform both the underlying pretrained model and models trained from scratch. Equitrain achieves the best overall performance, and our results demonstrate that fine-tuning enables accurate phonon predictions.

• The paper introduces Equitrain, a LoRA-based fine-tuning strategy that achieves DFT-level accuracy with as few as 10 material-specific configurations.
• The paper shows that utilizing large supercells during fine-tuning significantly reduces force MAEs and improves phonon frequency predictions by up to 5-fold.
• The paper finds that the parameter-efficient approach not only enhances vibrational and elastic property predictions but also drastically lowers computational costs for high-throughput studies.

Parameter-Efficient Fine-Tuning of MLIPs for Phonon and Thermal Prediction

Overview of Methodological Framework

The study addresses the parameter-efficient fine-tuning of machine learning interatomic potentials (MLIPs) for the accurate prediction of phonon and thermal properties, emphasizing minimal data regimes. Pre-trained foundation models such as MACE-MP-0b3, trained on large, diverse datasets, offer highly generalizable representations for atomistic modeling. However, phonon property prediction, particularly harmonic and anharmonic features pertinent to instabilities and thermal transport, remains highly sensitive to force errors at the meV/Å scale, often exceeding the capabilities of generic foundation models.

This work systematically evaluates multiple fine-tuning strategies—standard transfer learning, multi-head adaptation, and a LoRA-based framework termed "Equitrain"—across a comprehensive benchmark of 53 compounds with diverse structure types and chemical motifs. The aim is to determine how efficiently these methods can upgrade foundation models to achieve density functional theory (DFT)-level accuracy for vibrational and related thermodynamic quantities using only small material-specific datasets. The workflow encompasses structured data generation (rattled structures, relaxation trajectories), model adaptation, and multiproperty validation spanning force/energy/stress metrics, phonon dispersions, thermal, and elastic properties.

Figure 1: Schematic of the fine-tuning strategies—transfer learning, multihead, and Equitrain. Blue: training data; red: updated modules; grey: frozen modules.

Data-Efficient Fine-Tuning Regimes

A central claim is that as few as 10 material-specific training configurations suffice to induce substantial accuracy gains when fine-tuning a pre-trained foundation MLIP. The curated test set contains compounds with unit cells spanning 2 to 40 atoms, with an emphasis on phase-change and thermoelectric chalcogenides. Performance is systematically measured across data size, structure sizes (primitive, small, large supercells), and adaptation strategy.

Fine-tuning using large supercells, encompassing more diverse atomic environments and long-range force interactions, delivers the lowest mean absolute errors (MAE) in force predictions. Notably, all fine-tuning methods drastically surpass the foundation model and from-scratch training; among them, Equitrain is statistically superior, exhibiting the most rapid convergence and lowest MAEs.

Figure 2: Dataset summary (left) and force MAE convergence with increasing training set size for the evaluated strategies.

Phonon and Thermodynamic Property Prediction

The fine-tuned models achieve a median phonon frequency MAE as low as 0.05 THz with large supercells in Equitrain, contrasting sharply with 0.27 THz for the base foundation model. The use of larger supercells enhances the representation of long-range interactions critical for accurate phonon band structures.

Noteworthy is Equitrain's ~5-fold reduction in phonon MAE relative to MP-0b3, outperforming other strategies across maximum/mean frequency errors and density of states deviations. For mean phonon frequencies and properties averaging over all modes (e.g., heat capacity), foundation models already provide reasonable values, but fine-tuning yields critical corrections to maximum frequencies and DOS features necessary for stability and transport phenomena.

Transfer of Fine-Tuned Models to Thermodynamic and Elastic Properties

Examination extends to phonon-derived thermodynamic properties (heat capacity, entropy, free energy at 300 K), the bulk and shear modulus, as well as the phenomenological Slack thermal conductivity. Systematic evaluation reveals:

• All fine-tuned models achieve median errors within ±5% for phonon-derived properties, with Equitrain leading.
• From-scratch models, while improving some averaged thermodynamic values, underperform for elastic constants, exposing their inability to capture complex mechanical response from limited data.
• The foundation model systematically underestimates thermal conductivity and elastic constants by ≈50%, with wide error dispersions mitigated by fine-tuning.

  Figure 3: Distribution of deviations of thermal and elastic property predictions from DFT benchmarks across strategies.

Capturing Dynamic Instabilities and Modeling Transitions

A stringent test is the fidelity of fine-tuned MLIPs in predicting dynamical instabilities and displacive phase transitions—requiring accurate reproduction of both imaginary phonon modes and the associated anharmonic energy surfaces.

• Equitrain achieves perfect precision (no false positive unstable predictions) and high recall in detecting instabilities.
• Explicit tracking of phase transition pathways via collective mode displacements reveals that only Equitrain consistently matches DFT in both qualitative (double-well energy landscape) and quantitative (final relaxed phase, energy difference, space group) agreement.

Figure 4: (a) Section of the phonon band structure of K$_3$Sb for various models vs. DFT; (b) Anharmonic double-well potential along the K-mode; (c) Relaxed phases via displacive transition pathways.

As a further test, the high-$T$ $Cmcm$ phase of SnSe (a canonical anharmonic system) is investigated; Equitrain captures the relevant instabilities and phase connections with best quantitative accuracy, whereas other approaches (especially from-scratch) fail to mirror DFT energy profiles or predict correct ground-state phases.

Figure 5: Phonon band structure and anharmonic pathway analysis for Cmcm SnSe, revealing the correspondence between Equitrain and DFT.

Quantitative tabulation across all unstable compounds shows Equitrain maximizes true positive phase reproduction (precision 68%, F1 = 0.66), even outperforming the foundation model, while other fine-tuning methods exhibit catastrophic forgetting or decreased generalization.

Computational Cost and Practical Implications

Fine-tuning with a minimal dataset leads to substantial computational savings:

• DFT supercell-based phonon calculations average 27 h per material.
• Data generation for fine-tuning requires only 18.8 h per material (32% reduction), with the benefit increasing for systems requiring more phonon supercells.
• Model training time is negligible by comparison (<10 min per material for the most complex approach).

  Figure 6: Comparative CPU time for DFT phonon calculations and MLIP fine-tuning data generation as a function of system size.

Thus, not only does parameter-efficient fine-tuning (especially Equitrain) realize substantial improvements in accuracy, but it also delivers pragmatic savings for high-throughput workflows targeting vibrational and thermal properties in materials science.

Theoretical Implications and Future Directions

The results delineate the optimal balance between foundation pretraining for broad coverage and parameter-efficient, system-specific fine-tuning to reach DFT-level fidelity for phonon and thermal physics. Regularized adaptation (i.e., full-rank LoRA with weight decay) enables robust correction of systematic foundation model errors while preserving useful prior knowledge—minimizing catastrophic forgetting and enabling generalization to unseen instabilities and phase landscapes.

This work underlines the future promise of hybrid approaches that integrate scalable foundation model training with targeted, efficient adaptation. Fine-tuning protocols such as Equitrain are immediately extensible to newer, larger foundation models and multi-property datasets.

Open directions include extending these strategies to finite-temperature anharmonicity (phonon-phonon interactions), explicit electron-phonon coupling, and out-of-distribution behaviors, as well as developing rigorous uncertainty quantification and active-learning-driven fine-tuning for robust deployment.

Conclusion

The investigation demonstrates that parameter-efficient fine-tuning strategies, especially Equitrain's regularized additive approach, enable highly data-efficient and robust adaptation of foundation MLIPs for precise phonon, thermal, and elastic property prediction. With only a handful of system-specific configurations, it provides substantial accuracy gains over both foundation and from-scratch models. This paradigm unlocks rapid, cost-effective, and reliable energy landscapes for a broad spectrum of materials, supporting high-throughput discovery and detailed mechanistic studies of dynamical phenomena.