Physics-Informed Residual Neural Network

• Physics-Informed Residual Neural Network is a framework that enforces physical laws by minimizing the discrepancy (residual) between neural model outputs and governing equations.
• It integrates multiple residual types—continuous, discrete, and architectural—to stabilize training, improve gradient propagation, and adaptively weight sample contributions.
• Applications span PDE solving, inverse problems, and digital twins, achieving improved error metrics and uncertainty estimates despite challenges in hyperparameter sensitivity.

A Physics-Informed Residual Neural Network is a neural model whose training is organized around a residual that measures violation of governing physics, typically a differential, integral, or operator equation, and whose architecture may, in some formulations, also contain residual or skip connections. In the most common PINN formulation, a neural approximation $u_\theta$ is trained so that the PDE residual, boundary conditions, and initial conditions are jointly minimized; in a broader literature, the same label also covers residual-network backbones, residual-driven sampling and weighting, residual-based uncertainty quantification, and solver-coupled residual learning (Han et al., 2022, Dashtbayaz et al., 2024, Lee et al., 10 Mar 2026).

1. Terminology and scope

The term “residual” is used in at least three technically distinct senses in this literature. First, it denotes the physics residual, namely the discrepancy between the learned field and the governing equation. Second, it denotes an architectural residual, as in ResNet-style updates $y_{l+1}=y_l+F_l(y_l)$. Third, in solver-coupled settings it denotes the algebraic or discrete residual of a reduced or discretized system, such as $b-Ax$ or a time-discrete reduced-order residual (Han et al., 2022, Meng, 9 Mar 2025, Shan et al., 2021).

Residual meaning	Representative expression	Representative use
Physics residual	$r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$	Classical PINN training
Architectural residual	$y_{l+1}=y_l+F_l(y_l;\theta_l)$	ResNet, Neural ODE, residual PINN backbones
Discrete or solver residual	$\mathbb{R}_k=b-Ax_k$	Fixed-point, reduced-order, and operator-learning formulations

A recurrent clarification in the literature is that these senses should not be conflated. In computed tomography perfusion, for example, “residual” explicitly refers to the physics discrepancy $r_{\mathrm{phys}}$, not to ResNet-style skip connections (Lee et al., 10 Mar 2026). The same distinction is emphasized in theoretical work on PINN residual loss, where “residual” refers strictly to $\mathcal{L}[u_\theta]-f$, not to architectural identity mappings (Dashtbayaz et al., 2024).

At the same time, many works deliberately combine both meanings. Residual blocks are used in tropical-cyclone Neural ODE forecasting, integral-equation solvers, hybrid neural–RBF PINNs, residual-attention PINNs, residual CNN PINNs for neutron diffusion, and deep residual SIREN PINNs for acoustic inverse problems, while the training objective remains physics-informed through PDE, ODE, or integral residuals (Meng, 9 Mar 2025, Moghaddam et al., 22 Jan 2025, Cooley et al., 2024, Zhang et al., 2024, Kurata et al., 28 Dec 2025).

2. Mathematical structure of physics residual learning

For a PDE written as

$N[u](x,t)=f(x,t),$

a standard PINN represents the solution by $u_\theta(x)$ or $y_{l+1}=y_l+F_l(y_l)$0, and defines the residual

$y_{l+1}=y_l+F_l(y_l)$1

A common loss is

$y_{l+1}=y_l+F_l(y_l)$2

where $y_{l+1}=y_l+F_l(y_l)$3 are optional sample weights on residual points (Han et al., 2022).

This formulation generalizes beyond standard PDE collocation. In evidential perfusion PINNs, the governing physics is a tracer-kinetic convolution model, and the residual is written in differential form as

$y_{l+1}=y_l+F_l(y_l)$4

with $y_{l+1}=y_l+F_l(y_l)$5 under exact physics consistency (Lee et al., 10 Mar 2026). In broader PINN language, the same paper states the generic ODE residual

$y_{l+1}=y_l+F_l(y_l)$6

and PDE residual

$y_{l+1}=y_l+F_l(y_l)$7

which make explicit that the residual is the primary interface between neural approximation and governing law (Lee et al., 10 Mar 2026).

Residual learning also extends naturally to integral and integro-differential equations. RISN defines, for example, the Fredholm residual

$y_{l+1}=y_l+F_l(y_l)$8

the Volterra residual

$y_{l+1}=y_l+F_l(y_l)$9

and corresponding residuals for integro-differential, fractional, and Helmholtz-type integral equations (Moghaddam et al., 22 Jan 2025). This suggests that a physics-informed residual neural network is best understood as a residual-centered learning paradigm rather than a PDE-specific architecture.

3. Residual architectures and physics-encoded blocks

One major line of work uses residual architectures to stabilize deep optimization while preserving the physics-informed objective. In tropical-cyclone forecasting, the Physics-Informed Residual Neural Ordinary Differential Equation model uses the discrete residual map

$b-Ax$0

inside a Neural ODE

$b-Ax$1

with the residual structure used to preserve initial intensity information and improve gradient flow (Meng, 9 Mar 2025). Here, “physics-informed” enters mainly through feature engineering and model design rather than through a symbolic PDE penalty.

Other works make the residual block itself the core architectural primitive. RISN employs a residual multilayer perceptron with

$b-Ax$2

so that nonlocal integral operators and fractional operators can be handled by a deeper network without the same vanishing-gradient difficulties as standard PINNs (Moghaddam et al., 22 Jan 2025). HyResPINNs replace the residual branch by an adaptive hybrid of a neural subnetwork and a compactly supported RBF subnetwork,

$b-Ax$3

followed by a gated residual update

$b-Ax$4

thereby combining global smooth approximation with localized basis functions (Cooley et al., 2024).

Residual-attention variants push this further by modulating the residual channel itself. In one RA-PINN formulation for steady electrothermal multiphysics, the block update is

$b-Ax$5

where $b-Ax$6 is an attention gate and $b-Ax$7 a transformed feature branch (Zhou et al., 24 Mar 2026). In another residual-attention PINN for irregular interfaces and multi-peak transport fields, the normalized update is

$b-Ax$8

again using channel-wise gating to emphasize interface and hotspot features (Zhou et al., 24 Mar 2026).

A related but distinct construction appears in physics-encoded residual networks for digital twins, where a differentiable physics block $b-Ax$9 is embedded directly in the forward map, and a residual correction $r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$0 is added at the output: $r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$1 This architecture is not a PINN in the usual collocation sense, but it is a residual neural network whose forward pass explicitly contains physics operators and whose residual path stabilizes learning when static physical operators are interleaved with trainable blocks (Zia et al., 2024).

4. Residual weighting, sampling, risk, and uncertainty

Once the residual is the central training object, its distribution becomes an explicit design variable. Residual-Quantile Adjustment begins with residual-based weights such as

$r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$2

normalizes them, and then caps all weights above the $r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$3-quantile by the median: $r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$4 followed by renormalization (Han et al., 2022). The stated motivation is that the bottleneck in adaptive PINN training is the tail behavior of the residual distribution, not merely its maximum. This yields a residual-weighting scheme that suppresses extreme tail dominance while retaining adaptivity (Han et al., 2022).

RRaPINNs make the same point in risk-theoretic form. Instead of minimizing only the mean residual, they treat

$r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$5

as a random variable and impose chance-constraint style control via

$r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$6

To make this trainable, they optimize the empirical $r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$7 or a Mean-Excess surrogate

$r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$8

so that the worst $r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|$9 slice of the residual distribution is explicitly targeted (Akazan et al., 23 Nov 2025). A plausible implication is that PINN training can be interpreted not just as average residual minimization, but as residual-shape control.

Residuals also drive adaptive point placement. R2-PINN introduces Residual Adaptive Resampling, in which the domain is partitioned, the average residual in each subinterval is computed, and new collocation points are added in the subinterval with the highest average residual (Zhang et al., 2024). AM-PIRN instead keeps the total number of collocation points fixed and redistributes a movable subset according to a residual-based density

$y_{l+1}=y_l+F_l(y_l;\theta_l)$0

implemented through discrete resampling on a dense candidate set (Gao et al., 4 Apr 2025). Residual-attention multiphysics PINNs likewise use adaptive residual-based collocation to focus samples on interfaces, steep gradients, and coefficient-sensitive regions (Zhou et al., 24 Mar 2026).

In uncertainty-aware formulations, the residual itself becomes a probabilistic target. EPPINN places a Normal–Inverse–Gamma distribution over the physics residual, fixes the residual mean at zero, and optimizes an evidential negative log-likelihood plus a residual-dependent regularizer

$y_{l+1}=y_l+F_l(y_l;\theta_l)$1

thereby producing voxel-wise aleatoric and epistemic uncertainty maps for physics consistency without Bayesian sampling or ensemble inference (Lee et al., 10 Mar 2026).

5. Discretized, operator-theoretic, and theoretical formulations

A physics-informed residual neural network need not rely on continuous collocation with automatic differentiation of the governing operator. PINN-DG interpolates the network output into a finite-element space and minimizes a discontinuous-Galerkin residual energy,

$y_{l+1}=y_l+F_l(y_l;\theta_l)$2

thereby avoiding pointwise Hessian evaluation of the neural network for elliptic problems (Grekas et al., 4 Jul 2025). The same work proves equicoercivity and $y_{l+1}=y_l+F_l(y_l;\theta_l)$3-convergence of the discrete energies to the continuous least-squares energy, making residual minimization a variationally controlled procedure rather than a purely empirical one (Grekas et al., 4 Jul 2025).

Discretized residuals also appear in reduced-order and solver-coupled formulations. DisPINN constructs the physics loss from time-discretized governing equations, often after POD–Galerkin projection and optional DEIM hyper-reduction, so that the network is trained against residuals of a discretized reduced-order system rather than a continuous PDE operator (Halder et al., 2023). PhiSRL and related Pi-ResNets go further by identifying the residual block with a learned fixed-point correction driven by the algebraic residual

$y_{l+1}=y_l+F_l(y_l;\theta_l)$4

leading to stationary and non-stationary iterative updates that mimic classical linear solvers while using CNNs to learn the preconditioned correction (Shan et al., 2021).

Theoretical work has also treated residuals as the primary object of analysis. One result states that, for a $y_{l+1}=y_l+F_l(y_l;\theta_l)$5-th order PDE, effective residual minimization requires an activation whose $y_{l+1}=y_l+F_l(y_l;\theta_l)$6-th derivative is bijective; under suitable width and non-degeneracy assumptions, wide networks can globally minimize the residual loss (Dashtbayaz et al., 2024). A separate line derives certified, training-independent error bounds from residual information alone for most linear ODEs, certain nonlinear ODEs, and first-order linear PDEs, using integral-operator or characteristic-based constructions rather than assumptions about optimizer success (Liu et al., 2023). These results suggest that “physics-informed residual neural network” names not only a modeling strategy, but also a theoretical regime in which approximation, stability, and certification are all expressed through residual structure.

6. Applications, empirical behavior, and limitations

The application range of physics-informed residual neural networks is unusually broad. In forward and inverse PDE solving, the framework has been used for linear parabolic equations, Allen–Cahn, elliptic equations, Poisson, Burgers, Korteweg–de Vries, heat flow, neutron diffusion, integral and integro-differential equations, and discontinuous interface problems (Han et al., 2022, Akazan et al., 23 Nov 2025, Zhang et al., 2024, Moghaddam et al., 22 Jan 2025, Grekas et al., 4 Jul 2025). In scientific inverse problems and imaging, residual-based formulations appear in CT perfusion analysis, room-impulse-response interpolation, magnetic-wake vessel-length estimation, and electromagnetic volume-integral-equation modeling (Lee et al., 10 Mar 2026, Kurata et al., 28 Dec 2025, Fallah et al., 27 Apr 2025, Shan et al., 2021). Related residual-physics architectures have also been used in tropical-cyclone forecasting and digital-twin models for robotics and vehicle steering (Meng, 9 Mar 2025, Zia et al., 2024).

Representative empirical reports are correspondingly heterogeneous. RQA is reported to achieve the smallest $y_{l+1}=y_l+F_l(y_l;\theta_l)$7 error and smallest absolute maximum error throughout training on 5D linear parabolic and 5D Allen–Cahn problems, and to outperform SelectNet on 20D versions of both PDEs (Han et al., 2022). PIR-NODE reports a $y_{l+1}=y_l+F_l(y_l;\theta_l)$8 reduction in RMSE and a $y_{l+1}=y_l+F_l(y_l;\theta_l)$9 increase in $\mathbb{R}_k=b-Ax_k$0 relative to a neural-network baseline for 24-hour tropical-cyclone intensity prediction (Meng, 9 Mar 2025). PINN-DG reports speedups of $\mathbb{R}_k=b-Ax_k$1–$\mathbb{R}_k=b-Ax_k$2 over quadrature-based collocation PINNs and substantially lower memory usage on elliptic problems (Grekas et al., 4 Jul 2025). EPPINN is reported to attain the highest voxel-level and case-level infarct-core detection sensitivity on a clinical cohort while providing conservative uncertainty estimates with high empirical coverage (Lee et al., 10 Mar 2026).

Across these domains, several recurring advantages are repeatedly documented. Residual connections improve gradient flow and permit deeper backbones; residual weighting and sampling focus training on hard regions; residual-based probabilistic models quantify physics inconsistency; and discrete or solver-coupled residuals reduce mismatch between the numerical model used to generate data and the physics term used in training (Cooley et al., 2024, Zhang et al., 2024, Lee et al., 10 Mar 2026, Halder et al., 2023). This suggests that residual-centric design functions simultaneously as an optimization device, a numerical-analysis device, and a representational prior.

The limitations are equally recurrent. Many methods depend sensitively on hyperparameters governing residual weights, quantiles, tail levels, or resampling schedules; noisy or biased residual evaluation can mislead adaptive weighting; high-dimensional, stiff, or highly localized dynamics can make residual statistics unstable; and architectural improvements such as attention, per-case optimization, or hybrid kernels often increase runtime and tuning burden (Han et al., 2022, Lee et al., 10 Mar 2026, Moghaddam et al., 22 Jan 2025, Zhou et al., 24 Mar 2026, Halder et al., 2023). Forward-model simplifications are another persistent issue: box-residue perfusion models, simplified tracer kinetics, finite-depth wake approximations, or low-order mechanical priors can improve tractability while limiting generality (Lee et al., 10 Mar 2026, Fallah et al., 27 Apr 2025, Zia et al., 2024).

Taken together, the literature does not define a single canonical physics-informed residual neural network. Rather, it defines a family of methods unified by one principle: the residual—whether continuous, discrete, probabilistic, or architectural—is treated as the central carrier of physical structure. In some works that residual is the discrepancy in the governing equation; in others it is the update mechanism of the network; in the most developed formulations it is both at once.