Physics-Based Loss Function in Scientific ML

• Physics-based loss functions are training objectives derived from governing physical laws—using methods like PDE residuals, variational energies, or conservation principles—to validate model predictions.
• They encompass diverse formulations, from strong-form residual minimization in PINNs to energy-based and probabilistic equilibria approaches, enhancing accuracy in scientific machine learning.
• These loss functions reshape optimization geometry and emphasize measurement consistency, enabling robust, domain-aware models that better capture physical phenomena.

Physics-based loss function denotes a training objective whose semantics are supplied by the scientific system itself rather than by generic prediction error alone. Across the literature, such losses are built from governing partial differential equations, boundary and initial conditions, variational energies, conservation laws, acquisition operators, or task-specific scientific figures of merit; accordingly, they define correctness through physical admissibility, measurement consistency, or domain utility instead of only through mean squared error or cross-entropy against labels (Rowan et al., 5 Feb 2026, Shi et al., 2019, Bardhan et al., 2024, Kaba et al., 3 Nov 2025). In contemporary scientific machine learning, the term therefore covers a broad family of objectives, from the squared residual strong-form loss and the Deep Ritz energy in PINNs to line-integral projection losses for PET attenuation correction, significance-aware surrogates in collider event classification, and local energy losses for molecular and spin systems (Rowan et al., 5 Feb 2026, Shi et al., 2019, Bardhan et al., 2024, Kaba et al., 3 Nov 2025).

1. Conceptual scope and distinction

In the PDE-centered formulation used in physics-informed neural networks, a physics-based loss is “a training objective built directly from the governing physics—typically a PDE and its associated boundary conditions or variational structure—rather than from labeled input-output data alone” (Rowan et al., 5 Feb 2026). That definition captures the canonical case, but the broader literature extends the same principle to other scientific settings. In medical imaging, the loss may be defined in the projection domain because attenuation correction depends on line integrals rather than voxelwise errors (Shi et al., 2019). In high-energy physics, it may be derived from the approximate median significance $Z \approx N_s/\sqrt{N_b}$, because discovery sensitivity rather than average classification accuracy is the relevant terminal objective (Bardhan et al., 2024). In atomistic learning, it may arise from a local Boltzmann model centered on each sample, which turns supervised training into minimization of an approximate physical energy difference (Kaba et al., 3 Nov 2025).

This breadth matters because “physics-based” does not identify a single mathematical template. Some objectives enforce laws directly through residuals or balance relations; others encode invariants, energy conservation, or measurement operators; still others reweight errors toward physically decisive regions, such as the recirculation bubble in rarefied micro-step flow (Roohi et al., 21 Sep 2025). A plausible implication is that the term is best understood functionally: a loss is physics-based when its penalty structure is induced by quantities that already have scientific meaning in the target domain.

The literature also draws an important negative distinction. In the Deep Physics Corrector for stochastic differential equations, physics enters primarily through the gray-box model architecture and rollout dynamics, while the training objective is conditional maximum mean discrepancy; the method is therefore a physics-enhanced model with a statistical loss rather than a residual-based physics-loss framework (Tushar et al., 2022). This distinction separates physics in the model from physics in the objective.

2. Principal mathematical formulations

The most common formulation is strong-form residual minimization. If a neural field $u(\cdot;\theta)$ approximates the solution of a PDE with differential operator $\mathcal D$ and source term $s$, the loss can be written as

$$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$

For Poisson-type problems, this becomes the familiar squared $L^2$-norm of the PDE residual; in PINNs this residual is typically combined with initial- and boundary-condition penalties or enforced through architectural constraints (Rowan et al., 5 Feb 2026).

A second canonical formulation is variational or energy minimization. In the Deep Ritz method, training minimizes an energy functional rather than a squared strong-form residual: $$\Pi(\theta)=\int_\Omega \text{energy density} - \text{loading term}.$$ For the one-dimensional elliptic benchmark studied in the loss-landscape literature,

$$\Pi(\theta)=\int_0^1 \left[\frac{1}{2}\left(u_x(x;\theta)\right)^2-f(x)\,u(x;\theta)\right]\,dx,$$

whereas the corresponding strong-form PINN objective is

$$\mathcal L(\theta)=\frac{1}{2}\int_0^1 \left(u_{xx}(x;\theta)+f(x)\right)^2\,dx.$$

The two formulations differ structurally because Deep Ritz lowers the order of differentiation, while the strong-form loss enforces the PDE pointwise in the $L^2$ sense (Rowan et al., 5 Feb 2026).

A third formulation derives the loss from a probabilistic equilibrium model. In “Energy Loss Functions for Physical Systems,” the conditional uncertainty around a target configuration $u(\cdot;\theta)$0 is modeled as

$u(\cdot;\theta)$1

which yields the reverse-KL objective

$u(\cdot;\theta)$2

Since the partition function does not depend on $u(\cdot;\theta)$3, optimization reduces to minimizing the physically chosen energy $u(\cdot;\theta)$4. In the atomistic case, the principal instantiation is a pair-distance energy,

$u(\cdot;\theta)$5

which is invariant to Euclidean motions and to appropriate automorphisms of the distance graph (Kaba et al., 3 Nov 2025).

Other formulations remain simpler but are still physically motivated. In mechanics prediction, one can augment mean-squared error by “a measure of the difference in the energy between the input and output states,” thereby enforcing conservation of energy without constructing PDE residuals (Raymond et al., 2021). In “Molecular Learning Dynamics,” the learned object is itself a type-specific scalar loss $u(\cdot;\theta)$6, whose gradient plays the role of an effective force, so that $u(\cdot;\theta)$7 parallels the usual $u(\cdot;\theta)$8 correspondence between mechanics and potential energy (Gusev et al., 14 Apr 2025).

3. PINNs, mixed formulations, and loss construction

In standard PINNs, the data and physics terms are usually written as

$u(\cdot;\theta)$9

with scalarized objective

$\mathcal D$0

The multi-objective interpretation is central: the apparent trade-off between data fitting and physics enforcement is not fixed, because system parameters, domain size, and nondimensionalization rescale the residuals through the chain rule. For diffusion, the paper identifies the scale ratio $\mathcal D$1; for the steady Navier–Stokes example, $\mathcal D$2. The resulting “apparent Pareto front” is defined as the set of loss values achievable with gradient-based optimization (Rohrhofer et al., 2021).

Alternative residual constructions modify how the physics term is computed rather than what it represents. In FDM-PINN, the PDE loss is evaluated by finite-difference stencils on a regular grid instead of by automatic differentiation. The total loss remains

$\mathcal D$3

but $\mathcal D$4 is formed from discrete residuals such as the five-point Laplacian for Laplace’s equation or finite-difference approximations of $\mathcal D$5, $\mathcal D$6, and $\mathcal D$7 for Burgers’ equation (Lim et al., 25 Feb 2026).

A different reformulation removes the explicit initial- and boundary-condition penalties. In “About optimal loss function for training physics-informed neural networks under respecting causality,” the original problem is extended with Heaviside functions so that initial and boundary conditions appear as distributional contributions inside a single generalized residual term. This produces a one-term residual loss in which Dirac contributions encode the initial and boundary data, and it motivates causal weighting schemes based on running averages, Heaviside factors, or Dirac-like weights over time slices (Es'kin et al., 2023).

The choice of residual norm is itself nontrivial. For high-dimensional nonlinear Hamilton–Jacobi–Bellman equations, the paper “Is $\mathcal D$8 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?” proves that small $\mathcal D$9 residual does not generally imply a small solution error. For the HJB class studied there, stability with $s$0 residuals requires $s$1 to be sufficiently large, and the authors argue that $s$2 is a better choice than the de-facto standard $s$3 loss; they then propose an adversarial max-residual training algorithm to approximate $s$4-type minimization (Wang et al., 2022).

Loss scaling can also be derived from the PDE coefficients themselves. For mixed-formulation PINNs applied to neutron diffusion, Physics-Based Loss Scaling replaces the standard mixed residual norm by

$s$5

where $s$6 is the diagonal part of the removal operator and $s$7 is the diffusion matrix. The paper states that this weighted norm is equivalent to the classical MF-PINN loss in the continuous sense but improves convergence and accuracy in one-group and multigroup, 2D and 3D neutron diffusion benchmarks (Do et al., 27 Apr 2026).

A further variation moves the innovation to the data-side term. In meta-learning for parametric PDEs, the PDE residual loss is retained, but the standard initial/boundary-condition MSE is replaced during support-stage adaptation by a task-specific Generalized Additive Model fitted to the residuals. The resulting support loss becomes $s$8, while query performance is still evaluated with the conventional PINN objective (Koumpanakis et al., 2024).

4. Measurement-, decision-, and region-aligned objectives

Physics-based loss functions need not be equation residuals. In collider event classification, the training target can be derived directly from the search metric. “Loss function to optimise signal significance in particle physics” constructs a surrogate for the approximate median significance

$s$9

where selected signal and background yields are weighted by process cross sections and luminosity. Because the resulting objective is a set function over false negatives and false positives, the authors use the Lovász extension of a submodular loss to obtain a differentiable surrogate that is explicitly aware of process rates; the paper’s central motivation is that “not all event rates are equal,” whereas ordinary binary cross-entropy treats events symmetrically unless manually reweighted (Bardhan et al., 2024).

In PET attenuation-map synthesis, the relevant physics lies in the acquisition operator. The proposed line-integral projection loss is

$$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$0

with projection angles uniformly sampled over $$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$1. The motivation is that attenuation correction depends on the line integral of $$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$2 along a line of response rather than on voxelwise attenuation-map similarity. The resulting combined objective,

$$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$3

produced only slight changes in image-domain attenuation metrics relative to a pure image loss, but it reduced projection-domain errors and improved downstream PET attenuation-corrected image accuracy (Shi et al., 2019).

Electromagnetic inverse scattering motivates losses built from near-field quantities inside the domain of interest. The paper compares the baseline contrast loss

$$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$4

with two physics-guided alternatives: $$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$5 and

$$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$6

These losses inject multiple-scattering-based near-field priors into training rather than supervising only the final contrast image (Liu et al., 2021).

Region-aware weighting is another recurring design. For a DeepONet surrogate of rarefied micro-step flow, the vortex region is defined by $$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$7, and the total loss is

$$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$8

where each regional loss is normalized by the number of points in that zone. In the reported ablation, the full-domain relative $$\mathcal L(\theta)=\frac12\int_\Omega \|\mathcal D[u(\cdot;\theta)]-s\|^2\,d\Omega.$$9 error changed only marginally, whereas the vortex-region relative $L^2$0 error improved substantially, which the authors interpret as better engineering fidelity of the recirculation structure (Roohi et al., 21 Sep 2025).

5. Energetic, invariant, and interaction-based constructions

Not all physically meaningful objectives are formulated at the level of field equations. In mechanics, one can train a one-step predictor of pendulum states with mean-squared error plus an additional term measuring the energy difference between the input and predicted output state. The paper reports that the MSE-only model drifts toward lower-energy trajectories under rollout, whereas the energy-augmented loss preserves the phase-space orbit much better over four periods (Raymond et al., 2021).

In power electronics, the supervisory signal may be latent. “Data-driven Power Loss Identification through Physics-Based Thermal Model Backpropagation” does not train against direct power-loss labels; instead, it embeds a reduced-order thermal model into the computation graph and optimizes

$L^2$1

where the first term compares measured and predicted temperatures, $L^2$2 penalizes negative corrected power losses, and $L^2$3 regularizes deviation from the nominal power-loss model. The reported reduction in temperature and loss errors illustrates a form of indirect physical supervision through a differentiable thermal model (Scarpa et al., 31 Mar 2025).

“Molecular Learning Dynamics” takes an even stronger stance: the loss is not an auxiliary penalty but the scalar quantity whose gradient defines the particle dynamics. Each atom type uses a learned quadratic loss

$L^2$4

built from invariant descriptors of the surrounding environment. The induced equation of motion is $L^2$5, which generalizes the role of a global interaction energy to type-specific effective losses over invariant features (Gusev et al., 14 Apr 2025).

The energy-loss framework for equilibrium systems makes symmetry an intrinsic property of the objective. Because the atomistic loss depends only on pairwise distances, its minimizers are symmetry-equivalent configurations rather than a single coordinate embedding, and the paper argues that this yields gradients better aligned with valid physical configurations than coordinatewise MSE. The same paper extends the reverse-KL/Boltzmann derivation to discrete systems through a free-energy objective and instantiates it for spin systems with a local-field energy (Kaba et al., 3 Nov 2025).

Segmentation offers a different interaction-based construction. In “A Physics-Informed Loss Function for Boundary-Consistent and Robust Artery Segmentation in DSA Sequences,” the predicted and true boundaries are treated as interacting elastic curves through the nonlocal energy

$L^2$6

Here the physics is not hemodynamics but an elastic analogy inspired by dislocation theory in materials physics; the goal is smooth contour evolution and structural consistency of thin vessel boundaries (Irfan et al., 25 Nov 2025).

Wave forecasting introduces yet another pattern. The MI2A framework decomposes physical-space MSE into dissipation and dispersion components: $L^2$7

$L^2$8

and trains with

$L^2$9

The decomposition is motivated by the classical dissipation-versus-dispersion taxonomy of numerical wave propagation, so the loss is physics-based without being a PDE residual (Deo et al., 15 Apr 2025).

6. Optimization geometry, conditioning, and open questions

Loss design affects not only what is being penalized but also the geometry seen by optimization. In visualized loss landscapes of physics-informed neural networks, both Deep Ritz and squared strong-form losses exhibited monotonic linear interpolation from initialization in the one-dimensional elliptic case, smooth convex basins near trained solutions, mode connectivity between solutions, highly singular Hessians with many near-zero eigenvalues, and low-dimensional optimizer trajectories. Around trained states, both formulations appeared “smooth, well-conditioned, and convex in the vicinity of the solution,” and in the two-dimensional hyperelastic example their gradient directions agreed “to a large extent.” The same study also found that Monte Carlo integration can differentiate the two substantially: in 1D the Deep Ritz integrand had variance five orders of magnitude larger than the strong-form integrand at the solution, so stochastic estimates of the Deep Ritz objective became much noisier even though deterministic quadrature landscapes were similar (Rowan et al., 5 Feb 2026).

These observations complicate several common intuitions. One line of work shows that residual scaling induced by characteristic lengths, times, and coefficients can distort multi-objective optimization and shift the apparent Pareto front, making nominally identical loss weights behave very differently across parameterizations (Rohrhofer et al., 2021). Another shows that the default $$\Pi(\theta)=\int_\Omega \text{energy density} - \text{loading term}.$$0 residual can be theoretically unsuitable for high-dimensional nonlinear HJB equations, whereas $$\Pi(\theta)=\int_\Omega \text{energy density} - \text{loading term}.$$1-type control is more stable (Wang et al., 2022). A plausible implication is that there is no universally “correct” physics-based loss independent of the PDE class, coefficient scales, sampling rule, or approximation regime.

The literature also cautions against overgeneralization. The benign PINN landscapes reported for Deep Ritz and strong-form losses were obtained with boundary conditions enforced exactly through multiplicative trial functions and with objectives containing only interior physics terms, so those findings were explicitly not generalized to all PINN losses with boundary penalties (Rowan et al., 5 Feb 2026). Likewise, some methods are physics-guided rather than physics-constrained: DeepONet zonal loss prioritizes a physically important region, but it does not enforce conservation laws; DPC embeds approximate SDE physics in the model while retaining a statistical CMMD objective (Roohi et al., 21 Sep 2025, Tushar et al., 2022).

Current directions therefore span both hand-designed and learned objectives. Meta-learning for parametric PDEs replaces the standard data loss by a task-specific GAM-based loss while retaining the PDE residual, suggesting that even the loss itself can become an adaptable object rather than a fixed functional form (Koumpanakis et al., 2024). Taken together, these works suggest that physics-based loss functions are best viewed not as a single recipe but as a design space in which governing laws, invariants, acquisition operators, utility metrics, and optimization geometry are coupled.