Residual-Based Adaptive Refinement (RAR)

• Residual-based Adaptive Refinement (RAR) is a strategy that adapts discretizations based on local PDE residuals to target regions where approximations are most inaccurate.
• Variants like greedy RAR-G, RAD, and RAR-D differ in point selection and distribution, balancing between rapid error reduction and maintaining global domain coverage.
• Empirical results indicate that distribution-based methods such as RAD can significantly improve accuracy in problems with steep gradients and multi-scale features compared to uniform sampling.

Residual-based Adaptive Refinement (RAR) denotes a class of strategies that adapt a discretization, a mesh, or a set of training points according to the current residual of the governing partial differential equation. In the physics-informed neural network (PINN) setting, RAR means dynamically adjusting residual or collocation points during training using the instantaneous PDE residual $$\varepsilon(\mathbf{x}) = |f(\mathbf{x};\hat u(\mathbf{x}))|$$; in finite element, finite-difference, and isogeometric settings, it appears as a solve–estimate–mark–refine procedure driven by local residual indicators (Wu et al., 2022, Divi et al., 2022, Kasumba et al., 1 Jun 2026). The common principle is that large residuals indicate regions where the approximation is inaccurate or insufficiently constrained, so computational effort should be concentrated there.

1. Residuals, collocation sets, and the basic abstraction

In PINNs, the PDE is written abstractly as

$$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$

with boundary or initial conditions

$$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$

A neural network $\hat u(\mathbf{x};\theta)$ represents the solution, and automatic differentiation is used to evaluate the pointwise PDE residual

$$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$

For forward problems with boundary conditions enforced exactly through the architecture, the residual loss is

$$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$

where $\mathcal{T}_f\subset\Omega$ is the set of residual points, also called collocation points. For inverse problems, a data misfit term

$$\mathcal{L}_i(\theta,\boldsymbol\lambda;\mathcal{T}_i)=\frac{1}{|\mathcal{T}_i|}\sum_{\mathbf{x}\in\mathcal{T}_i} |\hat u(\mathbf{x})-u(\mathbf{x})|^2$$

is added, and the total loss becomes $\mathcal{L}=w_f\mathcal{L}_f+w_i\mathcal{L}_i$. In this formulation, RAR changes how $\mathcal{T}_f$ is placed and updated, thereby changing where the network is forced to satisfy the PDE (Wu et al., 2022).

In classical discretizations, the same logic is expressed through local error indicators built from strong-form residuals, flux jumps, and sometimes boundary or stabilization terms. For example, immersed isogeometric analysis with THB-splines uses elementwise residuals, jump terms, Nitsche boundary terms, and cut-cell stabilization terms, all scaled locally and fed into a Dörfler-type marking procedure; the same structural pattern reappears in stabilized conforming FEM, where a residual representative in a dual DG norm yields local indicators for a standard SOLVE–ESTIMATE–MARK–REFINE loop (Divi et al., 2022, Calo et al., 2019). This suggests that RAR is not a single algorithm but a residual-driven adaptation principle instantiated in different approximation spaces.

2. Classical greedy RAR in PINNs

The first adaptive PINN sampling scheme discussed in this literature is the classical Residual-based Adaptive Refinement of Lu et al. (DeepXDE, 2019), denoted in the comparative study as RAR-G, with “G” indicating greed. The algorithm begins with an initial residual set sampled uniformly, trains the PINN for a fixed number of iterations, samples a dense candidate set over the domain, evaluates $$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$0 on that set, selects the $$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$1 points with largest residual, augments the training set, and retrains. The process repeats until a stopping condition such as iteration count or maximum number of points is reached. Its defining feature is deterministic top-$$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$2 selection, so the collocation set grows monotonically and increasingly concentrates near maximum-residual regions (Wu et al., 2022).

This greedy mechanism is effective when the approximation error is strongly localized. It can capture steep gradients, fronts, and localized structures very efficiently because it explicitly targets the worst residual locations. At the same time, its limitations are equally direct: it may oversample a small region, under-sample the rest of the domain, increase training cost through monotone set growth, and preserve global coverage only through the initial sampling. These are not incidental side effects; they are consequences of the top-$$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$3 rule itself (Wu et al., 2022).

Early application papers used essentially this greedy interpretation. In the CGNLS vector-soliton study, the RAR-PINN algorithm computes a mean residual

$$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$4

selects the $$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$5 points with the largest residual, adds them to the training data, and repeats until the residual criterion is met; the reported effect is faster convergence and better approximation ability, especially for shape-changing vector-soliton interactions, and successful parameter discovery in coupled nonlinear equations (Qin et al., 2022). A two-phase porous-media study reaches a more cautionary conclusion: adaptivity is essential to capture moving flow fronts, but greedy max-residual RAR tends to produce excessively clustered points when the residual is large in very narrow regions, particularly for moving sharp fronts or discontinuities (Hanna et al., 2021).

3. Distribution-based variants: RAD and RAR-D

A major generalization replaces greedy selection by residual-weighted sampling from a probability density. In Residual-based Adaptive Distribution (RAD), the residual-based density is

$$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$6

with hyperparameters $$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$7 and $$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$8. The parameter $$f(\mathbf{x};u(\mathbf{x})) = 0,\quad \mathbf{x}\in\Omega,$$9 controls how sharply sampling concentrates in high-residual regions, while $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$0 provides a baseline probability that prevents complete neglect of low-residual regions. RAD keeps the number of residual points fixed and completely resamples them from this density after each training phase. In the same framework, Random-R is recovered as a special case: $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$1 makes the residual term constant, and large $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$2 makes the density approximately uniform (Wu et al., 2022).

Residual-based Adaptive Refinement with Distribution, also abbreviated RAR-D, uses the same density but adds rather than replaces points. After pretraining on an initial residual set, it repeatedly samples $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$3 new points from

$$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$4

augments the collocation set, and retrains. RAR-D therefore interpolates between two behaviors: accumulation of points as in greedy refinement, and distribution-based sampling as in RAD. In the limit of very large $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$5, only the largest residual locations retain significant weight, so RAR-D approaches greedy RAR-G; the study reports that moderate $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$6 performs better than extremely large $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$7 across the tested PDEs (Wu et al., 2022).

The PINN literature contains a minor nomenclature variation around this acronym. One source expands RAR-D as “Residual-based Adaptive Refinement with Distribution,” while an elliptic interface paper refers to “Residual-based Adaptive Refinement with Diversity.” In both cases, the defining mechanism is the same: residual-weighted probabilistic sampling with explicit preservation of coverage outside the highest-residual region (Wu et al., 2022, Xin et al., 2 Feb 2026).

For the benchmark suite in the comparative PINN study, the reported default choices are $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$8 for RAD and $$\mathcal{B}(u,\mathbf{x}) = 0,\quad \mathbf{x}\in\partial\Omega.$$9 for RAR-D; for the multi-scale wave equation, adding 10 points per refinement often worked best. In elliptic interface problems, dual-network PINN and KAN solvers adopt RAR-D on interior collocation points in each subdomain $\hat u(\mathbf{x};\theta)$0, while interface and boundary points are handled separately rather than through the adaptive density (Wu et al., 2022, Xin et al., 2 Feb 2026).

4. Empirical behavior across forward and inverse PDEs

The most systematic evidence comes from a comparison of 10 sampling methods across four forward problems and two inverse problems, summarized from more than 6000 PINN simulations. The methods comprise six fixed uniform samplers, Random-R, RAR-G, RAD, and RAR-D. The central empirical pattern is that RAD almost always achieves the best accuracy, RAR-D is often second, and both typically outperform non-adaptive methods by large margins on difficult problems; RAR-G can be highly effective on some forward problems but can underperform badly on inverse problems (Wu et al., 2022).

Problem	Representative errors	Interpretation
Burgers’ equation, 2000 residual points	Fixed uniform $\hat u(\mathbf{x};\theta)$1; Random-R $\hat u(\mathbf{x};\theta)$2; RAR-G $\hat u(\mathbf{x};\theta)$3; RAR-D $\hat u(\mathbf{x};\theta)$4; RAD $\hat u(\mathbf{x};\theta)$5	Strong localized feature; all adaptive methods help, RAD best
Wave equation, multi-scale, 2000 residual points	Uniform fixed $\hat u(\mathbf{x};\theta)$6; Random-R $\hat u(\mathbf{x};\theta)$7; RAR-G $\hat u(\mathbf{x};\theta)$8; RAR-D $\hat u(\mathbf{x};\theta)$9; RAD $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$0	Distribution-based refinement clearly exceeds greedy refinement
KdV inverse problem, 600 residual points	Best fixed uniform: solution $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$1, parameters $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$2; Random-R: solution $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$3, parameters $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$4; RAR-G: solution $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$5, parameters $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$6; RAR-D: solution $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$7, parameters $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$8; RAD: solution $$\varepsilon(\mathbf{x}) = \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|.$$9, parameters $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$0	Greedy RAR can be poor on inverse identification

The smooth diffusion–reaction inverse problem behaves differently. There, all reasonable methods perform well; RAD still gives the lowest errors, but fixed low-discrepancy sequences and Random-R are close, and RAR-G or RAR-D may not help. The practical conclusion drawn from these cases is sharply problem dependent: for sharp gradients, fronts, and multi-scale structure, residual-based adaptivity can reduce errors by orders of magnitude relative to non-adaptive uniform sampling, whereas for very smooth problems the gain is smaller and simple low-discrepancy or uniform-resampling strategies may suffice (Wu et al., 2022).

5. Generalizations across discretization paradigms

RAR is not confined to PINN collocation. In a finite-difference setting, a recent hybrid method uses a PINN only as an off-grid residual probe for adaptive mesh refinement, not as the final solver. The PINN residual $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$1 is sampled inside each cell $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$2, aggregated into a cellwise indicator

$$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$3

and then fed to either threshold marking or Dörfler marking before the PDE is re-solved by a classical finite-difference method. On the viscous Burgers benchmark, PINN-threshold AMR attained final relative $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$4 error $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$5 with $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$6 degrees of freedom, compared with $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$7 for uniform refinement with $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$8 degrees of freedom; PINN-Dörfler gave $$\mathcal{L}_f(\theta;\mathcal{T}_f) = \frac{1}{|\mathcal{T}_f|}\sum_{\mathbf{x}\in\mathcal{T}_f} \bigl|f(\mathbf{x};\hat u(\mathbf{x}))\bigr|^2,$$9 with $\mathcal{T}_f\subset\Omega$0 degrees of freedom. The same study also reports that a gradient indicator remains slightly more accurate, so the evidence supports usefulness rather than universal superiority (Kasumba et al., 1 Jun 2026).

In immersed isogeometric analysis, residual-based adaptivity retains the classical structure of element residuals and jumps but must additionally incorporate cut-cell stabilization and immersed-boundary terms. For Laplace and Stokes problems with THB-splines, local indicators combine volume residuals, flux or stress jumps, Nitsche boundary residuals, and stabilization contributions, and Dörfler marking is used to trigger THB-spline refinement in cut-cell regions and near singularities (Divi et al., 2022). In stabilized conforming FEM for advection–reaction, residual minimization in the dual of a DG norm produces a residual representative $\mathcal{T}_f\subset\Omega$1; local indicators $\mathcal{T}_f\subset\Omega$2 are then built from its cell, boundary, jump, and gradient contributions, again feeding a standard SOLVE–ESTIMATE–MARK–REFINE cycle (Calo et al., 2019).

Goal-oriented and test-space variants further broaden the concept. Dual Weighted Residual (DWR) methods keep the residual basis of RAR but multiply local residual and jump terms by dual weights associated with a quantity of interest, thereby targeting functional error rather than a global norm; this has been developed for biomechanics, hyperelasticity, and fluid–structure interaction (Becker et al., 12 Nov 2025). In the Deep Fourier Residual method, adaptivity occurs in the test space rather than in the trial network: local weak-residual contributions on overlapping subdomains are used as indicators, and a Dörfler-type marking algorithm enriches the local spectral test basis near singularities and sharp features (Taylor et al., 2024). Taken together, these formulations show that RAR spans collocation adaptation, mesh adaptation, goal-oriented mesh control, and test-space refinement.

6. Practical considerations, misconceptions, and scope

A recurrent misconception is that RAR is synonymous with greedily inserting the highest-residual points. The literature does not support that identification. Greedy top-$\mathcal{T}_f\subset\Omega$3 insertion is only one member of a broader family that includes full residual-based resampling (RAD), residual-weighted refinement with accumulating points (RAR-D), cellwise AMR driven by residual indicators, DWR for quantities of interest, and adaptive test-space enrichment (Wu et al., 2022, Becker et al., 12 Nov 2025, Taylor et al., 2024). Another misconception is that residual-based refinement is uniformly superior to simpler indicators. The available evidence is more qualified: on complex PDEs with sharp or multi-scale structure it is often much better than uniform sampling, but in smooth problems low-discrepancy sequences or Random-R can be competitive, and in PINN-guided FD-AMR a gradient indicator remained slightly more accurate in absolute error (Wu et al., 2022, Kasumba et al., 1 Jun 2026).

The main practical guidance in the PINN setting is comparatively specific. For difficult forward and inverse problems, RAD is presented as the default adaptive choice with $\mathcal{T}_f\subset\Omega$4. When retaining old collocation points is preferred, RAR-D is the recommended compromise with $\mathcal{T}_f\subset\Omega$5, small batches of added points, and modest retraining between refinement stages; the study gives 1000–2000 iterations and approximately 10 added points per refinement as effective simple choices. If adaptive sampling is not available, Random-R or a fixed low-discrepancy sequence such as Hammersley or Sobol is recommended as a fallback (Wu et al., 2022).

The main limitations are also consistent across papers. Residual evaluation on dense candidate sets, PDF construction, and repeated resampling or mesh refinement add computational overhead. Early in training, residuals can be noisy or poorly converged, so some pretraining on uniform points is helpful. In high-dimensional settings, brute-force candidate-set sampling becomes expensive; inverse transform sampling, MCMC, or GAN-based samplers are explicitly mentioned as possible alternatives for higher-dimensional RAD-style sampling (Wu et al., 2022). For greedy RAR in moving-front problems, overconcentration is a concrete failure mode rather than a theoretical possibility: the two-phase-flow study reports that narrow high-residual bands can attract too many collocation points and reduce global effectiveness (Hanna et al., 2021).

Finally, the choice of marking strategy matters once residuals have been converted into cellwise indicators. In a benchmark study driven by a residual-based Kelly estimator for steady solid and fluid mechanics, quantile and z-score markings were the most robust, Dörfler was effective for large bulk parameters, maximum marking was sensitive to irregular fields, and Isolation Forest could rival top classical methods only with a generous contamination level (Wege et al., 30 Apr 2026). A plausible implication is that, in many modern uses of RAR, the residual is only one part of the design problem: the aggregation norm, marking rule, refinement geometry, and solver architecture all materially affect the final efficiency.