Sci-ML: Physics-Integrated Machine Learning

• Scientific Machine Learning is an interdisciplinary field that integrates physics-based models with machine learning techniques to simulate complex scientific and engineering systems.
• It employs methods like physics-informed neural networks and neural operators to embed physical laws, thereby improving accuracy and reducing the need for vast data.
• Advanced Sci-ML frameworks enable uncertainty quantification, hybrid model development, and efficient surrogate-based optimization across diverse scientific domains.

Scientific Machine Learning (Sci-ML) denotes the interdisciplinary research field that combines physics-based models and data-driven ML approaches for the quantitative analysis and simulation of complex scientific and engineering systems. By embedding physical knowledge into statistical and deep learning techniques, Sci-ML aims to achieve accurate, physically consistent, and computationally efficient surrogates, surrogate-based optimization, and inference frameworks that address the limitations of both traditional mechanistic modeling and purely data-driven approaches. Sci-ML is characterized by mathematical formalism that supports calibration, uncertainty quantification, and the systematic handling of partial or noisy data; by architectures and workflows that encode physical constraints; and by best practices for verification and validation to ensure scientific trustworthiness and generalizability across natural and engineered domains.

1. Foundations and Historical Context

Sci-ML emerged as a response to the expanding gap between the predictive power of traditional physics-based digital models (built on first principles and numerical solvers) and the scalability and flexibility of data-driven ML algorithms. Classic mechanistic models—based on PDEs or ODEs representing conservation laws, constitutive relations, and boundary conditions—offer theoretical interpretability but are limited by incomplete knowledge, empirical closures, and high computational cost. In contrast, ML models, especially deep neural networks, have demonstrated success in extracting nonlinear relations from high-dimensional data but typically lack physical interpretability, extrapolative robustness, and consistency with governing laws (Willard et al., 2020, Quarteroni et al., 30 Jan 2025).

Scientific machine learning defines a rigorous framework on the continuum between these extremes by integrating physics into the training objective, architecture, data generation (e.g., synthetic data from simulators), and model initialization (Willard et al., 2020, Okazaki, 27 Sep 2024, Quarteroni et al., 30 Jan 2025). The field crystallized in the late 2010s, driven by advances such as physics-informed neural networks (PINNs), operator learning, and the need for interpretable, data-efficient surrogates in the context of growing experimental and simulation datasets (Hey et al., 2019, Okazaki, 27 Sep 2024).

2. Taxonomy and Methodological Classes

Sci-ML methodologies are classified along several axes: the manner in which scientific knowledge is incorporated, the nature of the ML architecture, and the coupling between mechanistic and data-driven components (Willard et al., 2020). Willard et al. (Willard et al., 2020) provide a canonical taxonomy:

1. Physics-Guided Loss Functions: Augmentation of the standard training loss with additional penalties for violation of physical laws or empirical constraints:

$$\text{Loss}(\theta) = \text{Loss}_{\rm TRN} + \lambda \cdot R(\theta) + \gamma \cdot \text{Loss}_{\rm PHY},$$

where $\text{Loss}_{\rm PHY}$ enforces, e.g., PDE residuals, conservation constraints, or monotonicity. Physics-informed neural networks (PINNs) are a prototypical example (Okazaki, 27 Sep 2024, Quarteroni et al., 30 Jan 2025).

1. Physics-Guided Initialization: ML models are pre-trained on synthetic data from mechanistic simulators, enabling rapid adaptation and reducing dependence on expensive experimental data (Willard et al., 2020).
2. Physics-Guided Architectures: Model structure encodes invariances/symmetries (e.g., rotation/group equivariance), physical sub-blocks, or conservation at the layer level (e.g., Hamiltonian Neural Networks, tensor-basis networks) (Willard et al., 2020, Miret et al., 2022).
3. Hybrid Physics-ML Models: ML and mechanistic models run in tandem or are blended via residual correction, ensemble mixtures, input augmentation, or component replacement. For instance, ML surrogates may replace sub-modules (e.g., microphysics) in PDE solvers, or correction nets may refine inversion results (Willard et al., 2020).

Each class enables unique trade-offs between interpretability, generalization, data efficiency, and computational cost.

3. Core Mathematical and Algorithmic Elements

3.1 Physics-Informed Neural Networks (PINNs)

A solution $u$ to a PDE (or a system thereof), such as

$L[u](x) = f(x),\quad x \in \Omega,$

is represented by a neural network $u_\theta(x)$. The loss incorporates both data fidelity and physics residuals:

$$\mathcal{L}_{\rm total} = \mathcal{L}_{\rm data} + \lambda_{\rm phys}\,\mathcal{L}_{\rm phys}$$

where

$$\mathcal{L}_{\rm data} = \frac1{N_{\rm data}} \sum_i |u_\theta(x_i) - u^{\rm obs}_i|^2,\ \mathcal{L}_{\rm phys} = \mathcal{L}_{\rm PDE} + \mathcal{L}_{\rm BC} + \mathcal{L}_{\rm IC}.$$

$\mathcal{L}_{\rm PDE}$ penalizes violations of the governing equation at collocation points via automatic differentiation (Okazaki, 27 Sep 2024).

3.2 Neural Operators

Operator learning seeks a mapping

$\mathcal{G} : a(\cdot) \mapsto u(\cdot)$

that maps functions $a(x)$ (e.g., spatially heterogeneous coefficients, initial conditions) to $u(x)$. Architectures such as DeepONet and the Fourier Neural Operator (FNO) build discretization-invariant, grid-transferable networks (Okazaki, 27 Sep 2024):

• DeepONet:

$$\mathcal{G}[a](y) = \mathbf{b}[a] \cdot \mathbf{t}(y),$$

with a branch network for $a$ and a trunk network for $y$.

• FNO:

$$v_{t+1}(x) = \sigma\left(Wv_t(x) + \mathcal{F}^{-1}\left(R_\theta(k) \mathcal{F}(v_t)(k)\right)(x)\right).$$

Physics-informed versions add PDE residuals to the training objective (PINOs) (Okazaki, 27 Sep 2024).

3.3 Multi-Fidelity Surrogates

Scientific contexts often present expensive high-fidelity ($f^{(1)}$) and cheaper low-fidelity ($f^{(2)}$) models. Multi-fidelity regression uses control variates to form unbiased, variance-reduced surrogates:

$\hat\beta_{MF} = C_{XX}^{-1} \hat C_{XY},$

with

$$\hat C_{XY}^\alpha = \frac{1}{m_1} \sum_{i=1}^{m_1} x_i y^{(1)}_i + \alpha \left[ \frac{1}{m_2} \sum_{i=1}^{m_2} x_i y^{(2)}_i - \frac{1}{m_1} \sum_{i=1}^{m_1} x_i y^{(2)}_i \right]$$

for optimal $\alpha^*$, yielding order-of-magnitude data savings (Qian et al., 13 Mar 2024).

4. Representative Applications and Benchmarks

Domain	Task/Objective	Example Techniques
Environmental	Lake temperature, climate downscaling	PGRNN, DeepSD, PINNs, GANs
Materials Sci	Energy prediction, inverse problems	Graph NNs, ML surrogates, multi-fidelity models
Fluid Mechanics	Airfoil optimization, turbulent flows	PINNs, DeepONet, RL/PINN hybrid surrogates
Seismology	Wave propagation, inverse tomography	PINNs (forward/inverse), Neural Operators (FNO)
Engineering	Micromixer design optimization	DRL+parametric PINN surrogate

Benchmarking frameworks, such as MLCommons Science Benchmarks Ontology, provide standardized evaluation across scientific domains, organizing benchmarks by domain, AI/ML motif (classification, regression, surrogate modeling), and application or system scope. Endorsement is based on a normalized rubric evaluating software, problem definition, data FAIRness, metrics, reference solution, and documentation (Hawks et al., 6 Nov 2025, Hey et al., 2019).

5. Verification, Validation, and Trustworthiness

Scientific accuracy and reliability are established through dedicated verification and validation (V&V) protocols (Jakeman et al., 21 Feb 2025):

• Verification (code and solution):
  • Method of Manufactured Solutions (MMS): Design test problems with known analytic solutions to test code correctness and convergence under mesh, data, or model refinement.
• Validation (realistic benchmarks):
  • Compare model predictions to reference data (experiments, high-fidelity simulation), measuring RMSE, $R^2$, and physical constraint adherence.
  • Probabilistic calibration (Bayesian inference) and partitioning of data for robust performance assessment.
• Best practices:
  • Data documentation (fidelity, processing, bias).
  • Sensitivity and uncertainty quantification for predictions via MC propagation or Bayesian posterior sampling.
  • Comparative assessment with alternative ML and classical approaches.
  • Interpretability tools and continuous documentation (Jakeman et al., 21 Feb 2025).

Sixteen consensus recommendations address problem definition, code verification, validation, UQ, reproducibility, and comparative benchmarking to establish trustworthy Sci-ML modeling.

6. Emerging Architectures, Automated Discovery, and Future Directions

Advances in Sci-ML encompass novel architectures—adaptive mixtures-of-experts, decomposition-based PINNs for domains with singularities, physics-constrained operator learning—and AutoML paradigms driven by multi-agent reasoning systems (Jiang et al., 10 Nov 2025). These frameworks orchestrate specialized agents (proposers, critics, engineers) to autonomously invent, critique, and evolve new Sci-ML strategies, yielding up to four orders of magnitude improvement over human-designed baselines in complex PDE, operator, and inference benchmarks.

Ongoing challenges and future research avenues include:

• Incorporation of uncertainty quantification at scale (Bayesian PINNs, variational methods, ParVI).
• Scaling to high-dimensional, multi-physics, and multiparameter domains.
• Hybrid schemes leveraging neural operators inside PINNs or data-assisted ensemble surrogates to address spectral bias and data scarcity.
• Extending modular Sci-ML toolkits (e.g., Open MatSci ML, MyCrunchGPT) to new scientific domains with standardized pipelines and interpretability/FAIR benchmarks (Miret et al., 2022, Kumar et al., 2023).
• Integration of reinforcement learning for physical design, high-fidelity surrogate modeling, and real-time adaptive control (Hassanzadeh et al., 10 Nov 2025).

7. Scientific Impact, Limitations, and Outlook

Scientific Machine Learning has demonstrated robust gains in predictive accuracy, data efficiency, and computational speed across a range of domains, particularly where classical physics-based or black-box ML approaches alone are suboptimal. Key merits include:

• Accelerated surrogate modeling and optimization (e.g., DRL+PINN in microfluidic design) achieving $>30\%$ improvement in mixing efficiency with instantaneous inference (Hassanzadeh et al., 10 Nov 2025).
• Operator networks and PINNs matching or exceeding classical solvers with orders-of-magnitude fewer data and lower memory (Okazaki, 27 Sep 2024).
• Unification of deep learning and rigorous statistical/Bayesian inference for uncertainty quantification and robust extrapolation (Okazaki, 27 Sep 2024, Jakeman et al., 21 Feb 2025).

Challenges persist in large-scale, multi-modal data integration; theoretical generalization guarantees; and automated, interpretable architecture discovery. The standardization of benchmarks and V&V workflows is critical for the sustained impact of Sci-ML in scientific computing, engineering, and data-intensive discovery (Hawks et al., 6 Nov 2025). The field is poised for continued expansion as architectures, computational infrastructure, and automated design paradigms mature, enabling cross-domain transfer, robustness assessment, and interpretable surrogate science at unprecedented scale.