---
title: Physics-Informed Residual Neural Network
url: https://www.emergentmind.com/topics/physics-informed-residual-neural-network
type: topic
---

# Physics-Informed Residual Neural Network

A Physics-Informed Residual Neural Network is a neural model whose training is organized around a residual that measures violation of governing physics, typically a differential, integral, or operator equation, and whose architecture may, in some formulations, also contain residual or skip connections. In the most common PINN formulation, a neural approximation \(u_\theta\) is trained so that the PDE residual, boundary conditions, and initial conditions are jointly minimized; in a broader literature, the same label also covers residual-network backbones, residual-driven sampling and weighting, residual-based uncertainty quantification, and solver-coupled residual learning [2209.05315], [2405.01680], [2603.09359].

## 1. Terminology and scope

The term “residual” is used in at least three technically distinct senses in this literature. First, it denotes the **physics residual**, namely the discrepancy between the learned field and the governing equation. Second, it denotes an **architectural residual**, as in ResNet-style updates \(y_{l+1}=y_l+F_l(y_l)\). Third, in solver-coupled settings it denotes the **algebraic or discrete residual** of a reduced or discretized system, such as \(b-Ax\) or a time-discrete reduced-order residual [2209.05315], [2503.06436], [2104.13231].

| Residual meaning | Representative expression | Representative use |
|---|---|---|
| Physics residual | \(r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|\) | Classical PINN training |
| Architectural residual | \(y_{l+1}=y_l+F_l(y_l;\theta_l)\) | ResNet, Neural ODE, residual PINN backbones |
| Discrete or solver residual | \(\mathbb{R}_k=b-Ax_k\) | Fixed-point, reduced-order, and operator-learning formulations |

A recurrent clarification in the literature is that these senses should not be conflated. In computed tomography perfusion, for example, “residual” explicitly refers to the physics discrepancy \(r_{\mathrm{phys}}\), not to ResNet-style skip connections [2603.09359]. The same distinction is emphasized in theoretical work on PINN residual loss, where “residual” refers strictly to \(\mathcal{L}[u_\theta]-f\), not to architectural identity mappings [2405.01680].

At the same time, many works deliberately combine both meanings. Residual blocks are used in tropical-cyclone Neural ODE forecasting, integral-equation solvers, hybrid neural–RBF PINNs, residual-attention PINNs, residual CNN PINNs for neutron diffusion, and deep residual SIREN PINNs for acoustic inverse problems, while the training objective remains physics-informed through PDE, ODE, or integral residuals [2503.06436], [2501.16370], [2410.03573], [2407.10988], [2512.22915].

## 2. Mathematical structure of physics residual learning

For a PDE written as
\[
N[u](x,t)=f(x,t),
\]
a standard PINN represents the solution by \(u_\theta(x)\) or \(u_\theta(x,t)\), and defines the residual
\[
r(x,t;\theta)=|N[u_\theta](x,t)-f(x,t)|.
\]
A common loss is
\[
L(\theta)=\frac{1}{N_r}\sum_{i=1}^{N_r} w_i |r(x_i,t_i;\theta)|^2
+\frac{\lambda_B}{N_b}\sum_{j=1}^{N_b}|B(u_\theta(x_j^{(B)},t_j^{(B)}))-g(x_j^{(B)},t_j^{(B)})|^2
+\frac{\lambda_I}{N_i}\sum_{k=1}^{N_i}|I(u_\theta(x_k^{(I)},0))-h(x_k^{(I)})|^2,
\]
where \(w_i\ge 0\) are optional sample weights on residual points [2209.05315].

This formulation generalizes beyond standard PDE collocation. In evidential perfusion PINNs, the governing physics is a tracer-kinetic convolution model, and the residual is written in differential form as
\[
r_{\mathrm{phys}}(t,x)=\frac{\partial C(t,x)}{\partial t}-\mathrm{CBF}(x)\big[A(t-\Delta t(x))-A(t-\Delta t(x)-\mathrm{MTT}(x))\big],
\]
with \(r_{\mathrm{phys}}\equiv 0\) under exact physics consistency [2603.09359]. In broader PINN language, the same paper states the generic ODE residual
\[
r_{\mathrm{phys}}(t)=y'_{\mathrm{pred}}(t)-f(y_{\mathrm{pred}}(t),t)
\]
and PDE residual
\[
r_{\mathrm{phys}}(t,x)=u_{t,\mathrm{pred}}(t,x)+N[u_{\mathrm{pred}}](t,x),
\]
which make explicit that the residual is the primary interface between neural approximation and governing law [2603.09359].

Residual learning also extends naturally to integral and integro-differential equations. RISN defines, for example, the Fredholm residual
\[
r_F(x;\theta)=\hat u(x;\theta)-\lambda\int_\Omega K(x,y)\hat u(y;\theta)\,dy-f(x),
\]
the Volterra residual
\[
r_V(x;\theta)=\hat u(x;\theta)-f(x)-\int_a^x K(x,y)\hat u(y;\theta)\,dy,
\]
and corresponding residuals for integro-differential, fractional, and Helmholtz-type integral equations [2501.16370]. This suggests that a physics-informed residual neural network is best understood as a residual-centered learning paradigm rather than a PDE-specific architecture.

## 3. Residual architectures and physics-encoded blocks

One major line of work uses residual architectures to stabilize deep optimization while preserving the physics-informed objective. In tropical-cyclone forecasting, the Physics-Informed Residual Neural Ordinary Differential Equation model uses the discrete residual map
\[
y_{l+1}=y_l+F_l(y_l,\theta_l)
\]
inside a Neural ODE
\[
\frac{dy(t)}{dt}=f_\theta(y(t),t),
\]
with the residual structure used to preserve initial intensity information and improve gradient flow [2503.06436]. Here, “physics-informed” enters mainly through feature engineering and model design rather than through a symbolic PDE penalty.

Other works make the residual block itself the core architectural primitive. RISN employs a residual multilayer perceptron with
\[
A_i=\sigma_i(A_{i-1}\Theta^{(i)}+b^{(i)})+A_{i-1},
\]
so that nonlocal integral operators and fractional operators can be handled by a deeper network without the same vanishing-gradient difficulties as standard PINNs [2501.16370]. HyResPINNs replace the residual branch by an adaptive hybrid of a neural subnetwork and a compactly supported RBF subnetwork,
\[
H^{(l)}(z^{(l)})=\sigma\!\Big(\phi(\alpha^{(l)})F_R^{(l)}(z^{(l)})+\big(1-\phi(\alpha^{(l)})\big)F_N^{(l)}(z^{(l)})\Big),
\]
followed by a gated residual update
\[
z^{(l+1)}=z^{(l)}+\phi(\beta^{(l)})H^{(l)}(z^{(l)}),
\]
thereby combining global smooth approximation with localized basis functions [2410.03573].

Residual-attention variants push this further by modulating the residual channel itself. In one RA-PINN formulation for steady electrothermal multiphysics, the block update is
\[
\mathbf{z}^{(l+1)}=\mathbf{z}^{(l)}+\big(\mathbf{1}+\mathbf{m}^{(l)}\big)\odot \mathbf{t}^{(l)},
\]
where \(\mathbf{m}^{(l)}\) is an attention gate and \(\mathbf{t}^{(l)}\) a transformed feature branch [2603.23578]. In another residual-attention PINN for irregular interfaces and multi-peak transport fields, the normalized update is
\[
\mathbf{h}^{(l+1)}=\mathrm{LayerNorm}\!\left(\mathbf{h}^{(l)}+\mathbf{g}^{(l)}\odot \mathbf{f}^{(l)}\right),
\]
again using channel-wise gating to emphasize interface and hotspot features [2603.22803].

A related but distinct construction appears in physics-encoded residual networks for digital twins, where a differentiable physics block \(\mathcal P\) is embedded directly in the forward map, and a residual correction \(\mathcal R\) is added at the output:
\[
\hat a=\mathcal P(x,l)+r,\qquad r=\mathcal R(V;\phi_r).
\]
This architecture is not a PINN in the usual collocation sense, but it is a residual neural network whose forward pass explicitly contains physics operators and whose residual path stabilizes learning when static physical operators are interleaved with trainable blocks [2411.11497].

## 4. Residual weighting, sampling, risk, and uncertainty

Once the residual is the central training object, its **distribution** becomes an explicit design variable. Residual-Quantile Adjustment begins with residual-based weights such as
\[
w_i^{(0)}\propto |r(x_i,t_i;\theta)|^p
\quad\text{or}\quad
w_i^{(0)}\propto |r(x_i,t_i;\tilde\theta)|^{p-2},
\]
normalizes them, and then caps all weights above the \(q\)-quantile by the median:
\[
w_i'=
\begin{cases}
m,& w_i>\tau,\\
w_i,& \text{otherwise},
\end{cases}
\]
followed by renormalization [2209.05315]. The stated motivation is that the bottleneck in adaptive PINN training is the tail behavior of the residual distribution, not merely its maximum. This yields a residual-weighting scheme that suppresses extreme tail dominance while retaining adaptivity [2209.05315].

RRaPINNs make the same point in risk-theoretic form. Instead of minimizing only the mean residual, they treat
\[
R_\theta:=|r(X;\theta)|,\qquad X\sim\mu,
\]
as a random variable and impose chance-constraint style control via
\[
\mathbb{P}(R_\theta\le \varepsilon)\ge \alpha.
\]
To make this trainable, they optimize the empirical \(\mathrm{CVaR}_\alpha\) or a Mean-Excess surrogate
\[
\mathcal{P}_{\mathrm{ms}}(R;\varepsilon,w)=\frac{1}{N}\sum_{i=1}^N w_i(R_i-\varepsilon)_+^2,
\]
so that the worst \((1-\alpha)\) slice of the residual distribution is explicitly targeted [2511.18515]. A plausible implication is that PINN training can be interpreted not just as average residual minimization, but as residual-shape control.

Residuals also drive adaptive point placement. R2-PINN introduces Residual Adaptive Resampling, in which the domain is partitioned, the average residual in each subinterval is computed, and new collocation points are added in the subinterval with the highest average residual [2407.10988]. AM-PIRN instead keeps the total number of collocation points fixed and redistributes a movable subset according to a residual-based density
\[
p(x;\Theta)\propto |r(x;\Theta)|^k/E[|r(x;\Theta)|],
\]
implemented through discrete resampling on a dense candidate set [2504.03244]. Residual-attention multiphysics PINNs likewise use adaptive residual-based collocation to focus samples on interfaces, steep gradients, and coefficient-sensitive regions [2603.23578].

In uncertainty-aware formulations, the residual itself becomes a probabilistic target. EPPINN places a Normal–Inverse–Gamma distribution over the physics residual, fixes the residual mean at zero, and optimizes an evidential negative log-likelihood plus a residual-dependent regularizer
\[
L_{\mathrm{reg}}=|r|(2\lambda+\alpha),
\]
thereby producing voxel-wise aleatoric and epistemic uncertainty maps for physics consistency without Bayesian sampling or ensemble inference [2603.09359].

## 5. Discretized, operator-theoretic, and theoretical formulations

A physics-informed residual neural network need not rely on continuous collocation with automatic differentiation of the governing operator. PINN-DG interpolates the network output into a finite-element space and minimizes a discontinuous-Galerkin residual energy,
\[
\tilde{\mathcal E}_h(u_\ell)
=
\sum_{K\in T_h}\int_K |\Delta I_h^q u_\ell+I_h^q f|^2\,dx
-2\sum_{e\in E_h^i}\int_e \{\!\{\Delta I_h^q u_\ell+I_h^q f\}\!\}\,[\nabla I_h^q u_\ell\cdot n]\,ds
+\alpha\,\mathrm{pen}(u_\ell),
\]
thereby avoiding pointwise Hessian evaluation of the neural network for elliptic problems [2507.03521]. The same work proves equicoercivity and \(\Gamma\)-convergence of the discrete energies to the continuous least-squares energy, making residual minimization a variationally controlled procedure rather than a purely empirical one [2507.03521].

Discretized residuals also appear in reduced-order and solver-coupled formulations. DisPINN constructs the physics loss from time-discretized governing equations, often after POD–Galerkin projection and optional DEIM hyper-reduction, so that the network is trained against residuals of a discretized reduced-order system rather than a continuous PDE operator [2311.14045]. PhiSRL and related Pi-ResNets go further by identifying the residual block with a learned fixed-point correction driven by the algebraic residual
\[
\mathbb{R}_k=b-\mathbb{A}x_k,
\]
leading to stationary and non-stationary iterative updates that mimic classical linear solvers while using CNNs to learn the preconditioned correction [2104.13231].

Theoretical work has also treated residuals as the primary object of analysis. One result states that, for a \(k\)-th order PDE, effective residual minimization requires an activation whose \(k\)-th derivative is bijective; under suitable width and non-degeneracy assumptions, wide networks can globally minimize the residual loss [2405.01680]. A separate line derives certified, training-independent error bounds from residual information alone for most linear ODEs, certain nonlinear ODEs, and first-order linear PDEs, using integral-operator or characteristic-based constructions rather than assumptions about optimizer success [2306.03786]. These results suggest that “physics-informed residual neural network” names not only a modeling strategy, but also a theoretical regime in which approximation, stability, and certification are all expressed through residual structure.

## 6. Applications, empirical behavior, and limitations

The application range of physics-informed residual neural networks is unusually broad. In forward and inverse PDE solving, the framework has been used for linear parabolic equations, Allen–Cahn, elliptic equations, Poisson, Burgers, Korteweg–de Vries, heat flow, neutron diffusion, integral and integro-differential equations, and discontinuous interface problems [2209.05315], [2511.18515], [2407.10988], [2501.16370], [2507.03521]. In scientific inverse problems and imaging, residual-based formulations appear in CT perfusion analysis, room-impulse-response interpolation, magnetic-wake vessel-length estimation, and electromagnetic volume-integral-equation modeling [2603.09359], [2512.22915], [2504.19112], [2104.13231]. Related residual-physics architectures have also been used in tropical-cyclone forecasting and digital-twin models for robotics and vehicle steering [2503.06436], [2411.11497].

Representative empirical reports are correspondingly heterogeneous. RQA is reported to achieve the smallest \(L^2\) error and smallest absolute maximum error throughout training on 5D linear parabolic and 5D Allen–Cahn problems, and to outperform SelectNet on 20D versions of both PDEs [2209.05315]. PIR-NODE reports a \(25.2\%\) reduction in RMSE and a \(19.5\%\) increase in \(R^2\) relative to a neural-network baseline for 24-hour tropical-cyclone intensity prediction [2503.06436]. PINN-DG reports speedups of \(3.1\times\)–\(11.1\times\) over quadrature-based collocation PINNs and substantially lower memory usage on elliptic problems [2507.03521]. EPPINN is reported to attain the highest voxel-level and case-level infarct-core detection sensitivity on a clinical cohort while providing conservative uncertainty estimates with high empirical coverage [2603.09359].

Across these domains, several recurring advantages are repeatedly documented. Residual connections improve gradient flow and permit deeper backbones; residual weighting and sampling focus training on hard regions; residual-based probabilistic models quantify physics inconsistency; and discrete or solver-coupled residuals reduce mismatch between the numerical model used to generate data and the physics term used in training [2410.03573], [2407.10988], [2603.09359], [2311.14045]. This suggests that residual-centric design functions simultaneously as an optimization device, a numerical-analysis device, and a representational prior.

The limitations are equally recurrent. Many methods depend sensitively on hyperparameters governing residual weights, quantiles, tail levels, or resampling schedules; noisy or biased residual evaluation can mislead adaptive weighting; high-dimensional, stiff, or highly localized dynamics can make residual statistics unstable; and architectural improvements such as attention, per-case optimization, or hybrid kernels often increase runtime and tuning burden [2209.05315], [2603.09359], [2501.16370], [2603.23578], [2311.14045]. Forward-model simplifications are another persistent issue: box-residue perfusion models, simplified tracer kinetics, finite-depth wake approximations, or low-order mechanical priors can improve tractability while limiting generality [2603.09359], [2504.19112], [2411.11497].

Taken together, the literature does not define a single canonical physics-informed residual neural network. Rather, it defines a family of methods unified by one principle: the residual—whether continuous, discrete, probabilistic, or architectural—is treated as the central carrier of physical structure. In some works that residual is the discrepancy in the governing equation; in others it is the update mechanism of the network; in the most developed formulations it is both at once.

Source: https://www.emergentmind.com/topics/physics-informed-residual-neural-network