KappaFormer: Physics-aware Transformer for lattice thermal conductivity via cross-domain transfer learning

Abstract: Machine learning has been widely used for predicting material properties. However, efficient prediction of lattice thermal conductivity ($κ\mathrm{L}$) remains a long-standing challenge, primarily due to the scarcity of high-quality training data. Here we introduce KappaFormer, a physics-aware Transformer architecture that embeds the harmonic-anharmonic decomposition of $κ\mathrm{L}$ within the network. KappaFormer comprises a harmonic branch pre-trained on large-scale elastic property data and an anharmonic branch fine-tuned on limited experimental $κ\mathrm{L}$ data, enabling effective knowledge transfer and enhanced generalization. High-throughput screening with KappaFormer identifies multiple candidates with ultralow $κ\mathrm{L}$, which are further confirmed by first-principles calculations. Physics interpretability further elucidates the vibrational mechanisms governing thermal transport suppression, linking structural motifs to strong anharmonicity. This study provides a generalizable framework for physics-guided machine learning to accelerate the discovery of new materials.

• The paper introduces KappaFormer, a physics-aware Transformer model that decomposes harmonic and anharmonic effects to predict lattice thermal conductivity.
• It employs a dual-stage cross-domain transfer learning approach, achieving a mean absolute error of 0.17 for log10 conductivity and over 79% of predictions under 30% error.
• The model provides interpretability by linking learned features to physical contributions, enabling high-throughput screening and discovery of ultralow κₗ materials.

Physics-aware Transformer Architectures for Lattice Thermal Conductivity Prediction

Introduction

Accurate prediction of lattice thermal conductivity ($\kappa_\mathrm{L}$) is critical for the rational design of functional materials for thermoelectric, thermal barrier, and electronic heat management applications. Owing to the prohibitive cost of experiments and high-fidelity first-principles calculations, there is a pressing need for data-efficient, physically interpretable surrogate models. The paper "KappaFormer: Physics-aware Transformer for lattice thermal conductivity via cross-domain transfer learning" (2604.03547) introduces an architecture that leverages domain-specific physical decomposition, multi-stage transfer learning, and graph-based equivariant attention to robustly predict $\kappa_\mathrm{L}$ across diverse material spaces, addressing challenges of data scarcity and interpretability in materials informatics.

KappaFormer Architecture: Physics-aware Representation of Harmonic and Anharmonic Effects

The KappaFormer architecture explicitly integrates the underlying physical decomposition of $\kappa_\mathrm{L}$ into harmonic and anharmonic contributions within a Transformer-based GNN. The model first encodes periodic structures as multi-edge graphs and captures geometric and chemical information using atom-pair-aware equivariant graph attention. Scalar recurrent blocks process structure embeddings to separately model harmonicity (governed by elastic moduli) and anharmonicity (related to Grüneisen parameters), with adaptive feature fusion via a Multi-gate Mixture-of-Experts (MMoE) framework. The final prediction composes these branches in accordance with the Slack model decomposition—enabling supervision and knowledge transfer at the level of physically meaningful quantities.

Figure 2: Schematic of the KappaFormer architecture, showing explicit harmonic–anharmonic decomposition embedded in a graph-attention Transformer backbone.

Dual-stage Cross-domain Transfer Learning Strategy

The authors propose a dual-stage transfer learning regime which first pre-trains the harmonic branch on extensive DFT-derived elastic property datasets (bulk modulus $B$, shear modulus $G$) using Huber loss, followed by fine-tuning on the scarce experimental $\kappa_\mathrm{L}$ data by optimizing both branches with $L_1$ loss on thermal conductivity and continued loss on elastic moduli. The harmonic branch parameters are frozen during fine-tuning, preserving knowledge and mitigating overfitting in the low-data regime for $\kappa_\mathrm{L}$.

Figure 1: a) The dual-stage transfer learning workflow bridging data-abundant and data-scarce domains; b–d) comparison of predicted and ground-truth values for elastic moduli and $\kappa_\mathrm{L}$. Insets visualize test errors.

Strong numerical results are reported: on the test set, KappaFormer achieves mean absolute error (MAE) of 0.17 and $R^2$ of 0.92 for $\kappa_\mathrm{L}$0, and MAEs of 5.23 (bulk modulus) and 6.08 (shear modulus), surpassing benchmarks such as SchNet, CGCNN, MEGNet, ALIGNN, DeeperGATGNN, and Matformer. Notably, 79% of predictions for $\kappa_\mathrm{L}$1 have relative error below 30%.

High-throughput Screening and Discovery of Ultralow $\kappa_\mathrm{L}$2 Materials

The pretrained KappaFormer is deployed for high-throughput predictions on over 24,000 candidate semiconductors from the Materials Project, with constraints on formation energy, bandgap, and composition. Statistical analysis reveals strong elemental trends: materials with Cs, halides (Br, Cl, I), and heavy cations are preferentially associated with low $\kappa_\mathrm{L}$3. The model discovers thousands of new compounds with predicted $\kappa_\mathrm{L}$4 W/mK, substantially expanding the chemical space beyond the original training distribution.

Figure 3: a) Periodic table visualization of chemical trends for low $\kappa_\mathrm{L}$5; b) Scatter of elastic moduli and $\kappa_\mathrm{L}$6 with highlighted new candidates; c-d) DFT validation and temperature-dependent anisotropy for selected materials.

Three representative candidates—CsNb$\kappa_\mathrm{L}$7Br$\kappa_\mathrm{L}$8, Cs$\kappa_\mathrm{L}$9AgI$\kappa_\mathrm{L}$0, Cs$\kappa_\mathrm{L}$1CdSe$\kappa_\mathrm{L}$2—are validated via explicit solution of the phonon Boltzmann transport equation. The KappaFormer’s physics-aware predictions closely match DFT results, in contrast to an ablated end-to-end model without physical decomposition that exhibits notable overestimation, confirming the criticality of physical priors in model robustness and OOD generalization.

Physics Interpretability: Atom-resolved and Modal Insights

A central contribution is the interpretability framework connecting the learned KappaFormer embeddings and attention weights to atom-resolved contributions to harmonicity and anharmonicity. The approach aggregates attention-derived importances over chemical species, quantifying the relative roles of constituent elements in both branches.

Figure 4: Schematic of the interpretability pipeline, linking model outputs to physical contributions and DFT-derived modal analysis.

This data is cross-examined with DFT-derived quantities: PDOS for harmonicity, COHP for bond strength, and the phonon participation ratio (PPR) for localization and anharmonic effects. For all three validated compounds, the dominant harmonic contribution is traced to halogen or chalcogen anions, correlating with soft ionic frameworks and weak bonding, while the Cs cation contributes more significantly to anharmonic modes, especially via rattling, as visualized by localized phonon eigenmodes with low PPR. These mechanistic insights elucidate the physical origin of ultralow $\kappa_\mathrm{L}$3 as arising from both softened lattice force constants and localized, strongly anharmonic structural units.

Figure 7: Atom-resolved model decompositions and DFT modal analyses of harmonic and anharmonic contributions for the three candidate materials.

Implications and Future Directions

The explicit embedding of domain-specific physical decomposition within a scalable, equivariant Transformer permits both data efficiency and interpretability. KappaFormer’s architecture provides a blueprint for surmounting data scarcity in other materials prediction tasks governed by multiple, intertwining physical mechanisms. The physically-informed backbone, dual-domain transfer handling, and model interpretability will facilitate rapid acceleration of inverse design and discovery protocols for functional solids.

In practical terms, the identification of previously unreported thermodynamically stable materials with ultralow $\kappa_\mathrm{L}$4 demonstrates the utility of KappaFormer as a front-end for experimental screening efforts. The framework also raises the prospect of integrating more sophisticated many-body physics or extending to time-domain transport phenomena, as well as modular adoption in other property-prediction tasks suited to decomposable theoretical models.

Conclusion

The KappaFormer model demonstrates that integrating harmonic–anharmonic physical decomposition in a Transformer–graph attention paradigm, coupled with cross-domain transfer learning, achieves robust, generalizable, and interpretable prediction of lattice thermal conductivity, even in regimes of limited data. Both the numerical benchmarks and interpretability case studies affirm the centrality of physics-guided architectures as the foundation for new materials informatics pipelines and the next-generation of scientific machine learning models.