Scientific Machine Learning (SciML)

• Scientific Machine Learning is an interdisciplinary paradigm that integrates physical models with data-driven methods to solve complex scientific problems effectively.
• SciML methodologies, such as physics-informed neural networks and neural operators, leverage known physics constraints to boost simulation accuracy and computational efficiency.
• Hybrid approaches in SciML merge mechanistic laws with machine learning to enable enhanced uncertainty quantification, scalability, and real-world applicability.

Scientific Machine Learning (SciML) is an interdisciplinary paradigm that systematically integrates physical modeling—often in the form of partial differential equations (PDEs) or other mechanistic laws—with ML and data-driven inference. SciML aims to combine the interpretability, generalization, and inductive biases of physics-based models with the flexibility and data-adaptivity of modern ML. This synthesis enables efficient, robust, and physically consistent solutions to complex scientific and engineering problems, particularly in regimes characterized by sparse, noisy, or incomplete data, high dimensionality, or computational bottlenecks (Quarteroni et al., 30 Jan 2025, Okazaki, 27 Sep 2024, Adombi, 24 May 2025).

1. Foundational Principles and Motivation

Core to SciML is the fusion of two distinct modeling paradigms:

• Physics-based models: Rely on established phenomenological, constitutive, or conservation laws to formalize system behavior. Numerical approximations (e.g., finite difference, finite element, spectral methods) are typically required for solution, often resulting in high computational demands.
• Data-driven models: Employ statistical or machine learning frameworks to approximate complex relationships in data without explicit assumptions about underlying causal mechanisms. These include neural networks, Gaussian processes, and symbolic regression methods.

SciML leverages physical insight to constrain data-driven learning—injecting prior knowledge, ensuring physical plausibility, and reducing sample complexity—while conversely using ML to uncover hidden laws, automate model reduction, or accelerate large-scale simulations (Quarteroni et al., 30 Jan 2025, Okazaki, 27 Sep 2024, Adombi, 24 May 2025). Applications span climate modeling, cardiac electrophysiology, seismology, hydrology, materials, and more.

2. Methodological Taxonomy: Unified SciML Framework

A comprehensive taxonomy of SciML organizes methods into four main families, each distinguished by the mode of physics–ML interaction (Adombi, 24 May 2025):

Family	Integration Mechanism	Canonical Mathematical Formulation
Physics-Informed	Physics as loss constraints	$$L_{\rm total} = \alpha_1\|y-y_{\rm obs}\|^2 + \alpha_2\text{PDE residual} + \ldots$$
Physics-Guided	Physics as input/output features	$$y = F_{\rm out}(\mathrm{concat}(X, Z_{\rm phys}); \theta)$$
Hybrid Phy-ML	Coupled separate modules	(a) $y = y_{\rm phys} + T_{\rm ML}(x,y_{\rm phys})$, (b) physics-embedded ML, (c) module replacement
Physics Discovery	Data-driven law discovery	(i) $u = F_{\rm disc}(x;\theta)$ (symbolic), (ii) $\partial_t u = L_{\rm det}[u;\theta]+\sigma dW$ (SUPDE)

2.1 Physics-Informed Machine Learning (PIML)

Physics-Informed Neural Networks (PINNs) and variants penalize deviations from underlying physical laws by incorporating PDE residuals, boundary/initial condition violations, and interface conditions as soft or hard constraints in a composite loss (Okazaki, 27 Sep 2024, Adombi, 24 May 2025). General UPIML architectures feature multiple "state modules" approximating physical fields, "parameterization modules" for latent variables, and adaptive loss weighting for multi-component constraints. Physics-informed neural operators extend this concept to learn infinite-dimensional solution operators via architectures such as DeepONet and Fourier Neural Operator (FNO) (Okazaki, 27 Sep 2024, Subramanian et al., 2023).

2.2 Physics-Guided Machine Learning (PGML)

Here, outputs from physical simulators or reduced-order models are used as features or surrogates in downstream data-driven learning, but physical constraints are not imposed directly in the objective (Adombi, 24 May 2025). This modularity enables hybridization of legacy codes with ML, at the expense of direct physical fidelity in outputs.

2.3 Hybrid Physics–Machine Learning

Hybrid strategies combine physics and ML components additively, embed differentiable physics within neural architectures, or replace selected mechanistic modules with neural surrogates. Jointly optimizing both physics and ML parameters may offer significant flexibility and acceleration while maintaining interpretability (Adombi, 24 May 2025). Submodule replacement and additive-residual learning are common in large-scale environmental and engineering applications.

2.4 Data-Driven Physics Discovery

Symbolic regression, stochastic universal PDEs, and conceptual model discovery enable identification of governing laws directly from data, bridging scientific inference with data-driven hypothesis generation (Adombi, 24 May 2025). Approaches include sparse identification (SINDy), deep symbolic regression, and stochastic regime inference.

3. Mathematical and Computational Formulations

3.1 Universal Differential Equations

The Universal Differential Equation (UDE) formalism generalizes classical ODE/PDE models by embedding trainable neural or symbolic approximators into the right-hand side:

$\dot{x}(t) = f(x(t),t;\phi) + U_\theta(x(t),t)$

where $f$ encodes known dynamics and $U_\theta$ learns unknown components (Rackauckas et al., 2020, Kashyap et al., 9 Oct 2024). This enables seamless blending of mechanistic and data-driven knowledge for forward simulation, parameter identification, and inverse modeling, supporting both deterministic and stochastic, delay, or hybrid systems.

3.2 Operator-Learning Architectures

Neural operators—including DeepONet and FNO—learn mappings between infinite-dimensional function spaces, enabling mesh-agnostic surrogates for solution operators to families of PDEs (Subramanian et al., 2023, Okazaki, 27 Sep 2024). Architectures combine branch (input function encoding) and trunk (evaluation point encoding) networks, yielding scalable and discretization-independent solvers.

3.3 Physics-Constrained Surrogate Modeling

Polynomial chaos expansions (PCE) and their physics-constrained variants (PC²) provide interpretable surrogates by regressing onto orthonormal polynomial bases, enforcing PDE residuals and physical constraints via collocation and regularization (Sharma et al., 23 Feb 2024). This structure supports built-in uncertainty quantification, analytic moment extraction, and strict enforcement of physical consistency.

3.4 Optimization in Function Spaces

SciML tasks involve optimization in infinite-dimensional Hilbert/Banach spaces—formalizing first- and second-order flows in natural metrics (e.g., $L^2$ or energy norms) leads to mesh-independent, well-conditioned algorithms, e.g., natural gradient descent or function-space Newton methods (Müller et al., 11 Feb 2024). This paradigm, "optimize-then-discretize," outperforms parameter-space optimizers and exposes the underlying geometry of PDE-constrained ML.

4. Applications and Benchmarks

SciML methodologies have demonstrated impact across multiple domains and workflows:

• Cardiac simulation: Data-driven methods uncover constitutive relationships for cardiac tissue and enable reduced-order surrogates, improving efficiency and interpretability in large-scale cardiac electrophysiology (Quarteroni et al., 30 Jan 2025).
• Seismology: PINNs and neural operators address forward/inverse wave propagation, parameter estimation, and real-time geophysical inversion, overcoming data scarcity and providing uncertainty-aware predictions (Okazaki, 27 Sep 2024).
• Hydrology: Unified SciML frameworks enable modular modeling of unsaturated flow, multi-physics transport, residual correction of process-based simulators, and data-driven law discovery, systematically categorizing methods and elucidating their mathematical structures (Adombi, 24 May 2025).
• Climate and turbulence: SciML-based super-resolution techniques enhance simulation and reanalysis data, though pixel-level fidelity does not guarantee physical consistency unless constraints (e.g., divergence-free conditions) are explicitly enforced (Ren et al., 2023).
• Optimization and uncertainty quantification: Rigorous workflows for uncertainty-aware surrogate modeling, robustness certification in optimization, and scalable Bayesian calibration have been demonstrated, providing predictive intervals and reproducibility guarantees (Almeida et al., 2022, Queiroz et al., 2022, Zou et al., 12 Apr 2024).

5. Scalability, Efficiency, and Trustworthiness

5.1 Pretraining, Transfer, and Foundation Models

Scaling laws and transferability play critical roles: foundation neural operators pretrained on large, heterogeneous PDE datasets enable rapid fine-tuning on novel or out-of-distribution tasks, yielding $10^2$–$10^3\times$ data savings and robust performance under moderate physics shifts (Subramanian et al., 2023). A single model can encode multiple operators, facilitating multi-task and data-efficient learning.

5.2 Computational Acceleration

Mixed-precision training with master–slave weight schemes in PINNs and DeepONets provides up to 2$\times$ speedups and halved memory usage while preserving accuracy, overcoming the limitations of pure half-precision arithmetic (gradient vanishing, update underflow) (Hayford et al., 30 Jan 2024).

5.3 Verification and Validation (V&V)

Trustworthy SciML requires adapted code and solution verification (using synthetic/manufactured solutions, error scaling), Bayesian or UQ-driven calibration, purpose-specific validation metrics, and rigorous documentation of data provenance, processing, and hyperparameter selection (Jakeman et al., 21 Feb 2025). Best practices involve benchmarking against state-of-the-art CSE solvers, uncertainty propagation, and interpretability analyses.

6. Open Challenges and Research Frontiers

Active research directions include:

• Theoretical convergence guarantees and error bounds for operator learners and deep nets in SciML contexts (Jakeman et al., 21 Feb 2025).
• Automated, multi-agent scientific discovery using collaborative evolutionary search and best-of-breed ensemble strategies for model architecture and loss formulation innovation (Jiang et al., 10 Nov 2025).
• Uncertainty quantification and robustness: Holistic frameworks integrating Bayesian inference, Monte Carlo propagation, and ensemble training to quantify epistemic and aleatory uncertainty for deployment in safety-critical domains (Almeida et al., 2022, Zou et al., 12 Apr 2024, Sharma et al., 23 Feb 2024).
• Physics discovery: Compositional, noise-robust symbolic regression and process-oriented conceptual model discovery for inferring new physics directly from sparse/uncertain data (Adombi, 24 May 2025).
• Computational scaling: Distributed/parallel solvers, domain-adaptive bases, and memory-efficient operator representations for extending SciML to high-dimensional, multi-physics, real-time, and streaming settings (Sharma et al., 23 Feb 2024, Sorokin, 26 Nov 2025).

7. Synthesis and Outlook

Scientific Machine Learning unifies a diverse repertoire of physics-based, data-driven, and hybrid methods under a structured paradigm for physically-informed, data-constrained scientific inference and prediction. The field's evolution is defined by its modular methodological families, rigorous mathematical frameworks, and a growing corpus of scalable, trustworthy strategies for tackling the next generation of frontier problems in science and engineering (Adombi, 24 May 2025, Jakeman et al., 21 Feb 2025, Subramanian et al., 2023, Okazaki, 27 Sep 2024).