Goal oriented error estimation for adaptive sampling of PINNS

Abstract: Physics-Informed Neural Networks (PINNs) are mesh-free approaches for the numerical approximation of partial differential equations, where a neural network is trained by minimizing a loss function derived from the governing equations and boundary conditions. The Deep Ritz method can be interpreted as a particular variational form of a PINN, where the loss corresponds to the minimization of an energy functional associated with a symmetric positive definite problem. In this work, we study the approximation of the Laplace equation using both the classical PINN formulation and its variational counterpart, the Deep Ritz method, with the objective of accurately estimating prescribed goal functionals. When standard sampling strategies, such as uniform or loss-based sampling, are employed during training, the convergence of the functional error and the attained minimal functional value can be slow. To address this issue, we introduce a functional-oriented importance sampling strategy that can be applied to both PINNs and the Deep Ritz method. The key ingredient is the construction of a reliable and accurate estimator for the error in a given quantity of interest. This estimator is derived using concepts from the Dual Weighted Residual (DWR) framework and is implemented entirely within the neural network setting. It is then used to adaptively guide the sampling of training points in the computational domain, focusing computational effort on regions that have the strongest influence on the functional value. Numerical experiments demonstrate that the proposed adaptive sampling strategy significantly accelerates the convergence of the functional error and improves the minimization of the target functional during training for both PINN and Deep Ritz formulations.

• The paper develops a PINN-compatible DWR error estimator that drives adaptive sampling by localizing goal functional errors.
• The approach leverages neural network approximations for both primal and adjoint problems, eliminating the need for mesh-based methods.
• Extensive numerical tests show faster convergence and improved accuracy for high-dimensional and non-smooth PDEs compared to uniform sampling.

Goal-Oriented Error Estimation for Adaptive Sampling in PINNs: An Authoritative Technical Overview

The paper "Goal oriented error estimation for adaptive sampling of PINNS" (2604.01835) develops a fully neural-network-based framework for a posteriori goal-oriented error estimation in the context of Physics-Informed Neural Networks (PINNs) and the Deep Ritz Method (DRM). By leveraging principles from finite element methods—specifically, the Dual Weighted Residual (DWR) technique—the authors propose an adaptive training regime tailored to minimize the error in prescribed goal functionals, thereby focusing computational resources on the most impactful areas. This essay details the methodology, the key contributions, extensive numerical validation, and the broader implications and open questions prompted by this work.

Background and Motivation

PINNs have emerged as mesh-free alternatives for solving partial differential equations (PDEs), representing the solution via trainable neural networks constrained by the underlying physics. Traditional sampling strategies—such as uniform or residual-based distributions—can result in slow convergence or inefficient minimization of functional outputs, especially where the quantity of interest (QoI) is highly localized or non-smooth.

Goal-oriented adaptivity, a mainstay in the numerical analysis of PDEs (notably via DWR methods), offers a principled avenue to focus training on the attainment of sharp error control for specific outputs. While importance sampling and residual-driven adaptivity (RAD, RAR-D) exist in the PINN literature, prior methods either depend on residual distributions or require latent mesh structures, which limit scalability or mesh-free advantages.

Methodological Advances

Neural-Network-Based DWR Estimation

The central technical contribution is a PINN-compatible implementation of the DWR error estimator. The estimator quantifies the error in the QoI $J(u)$ (a functional of the solution $u$), relying on solutions to both the primal (original) and adjoint (dual) variational problems, both approximated by neural networks. Unlike mesh-based DWR approaches, the method constructs all approximations in the neural function space, circumventing the need for any mesh infrastructure.

The approach consists of:

• Training a network for the adjoint problem, yielding an adjoint PINN, $z_{\lambda, \mathcal{N}}$.
• Training the primal network, $u_{\lambda, \theta}$, on standard loss formulations (either PINN or DRM).
• Localizing the global QoI error estimator using adjoint and primal derivatives, enabling spatial adaptivity for collocation point sampling.

This estimator can be localized to guide adaptive sampling—preferentially selecting collocation points from regions with the largest contributions to goal functional error.

Adaptive Sampling Strategy

The DWR-derived error indicator, $\mu(x)$, is employed to define a sampling distribution:

$$p_\mu : x \mapsto \frac{|\mu(x)|}{\int_\Omega |\mu(x)|}$$

This distribution directs sampling towards regions of highest goal functional sensitivity. The method encompasses both completely resampling collocation points and refinement by iteratively adding points in error-dominated regions. The adaptive regime can be integrated seamlessly into the PINN or Deep Ritz training paradigm without altering core objective functions.

Numerical Results and Effects

The methodology is validated on a range of test cases: smooth and non-smooth 2D Poisson problems, high-dimensional PDEs in $\mathbb{R}^5$, and both PINN and Deep Ritz formulations. Across scenarios, the DWR-based adaptive sampling yields:

• Substantially accelerated convergence for the goal functional error compared to uniform or residual-based sampling.
• Lower final error magnitudes for the functional outputs, even in settings with severe solution non-smoothness or high dimensionality.
• Highly effective error estimation in the QoI from early stages of training, despite the non-convexity and high-dimensionality of neural solution spaces.

Figure 1 below depicts the progression from raw pointwise error, to DWR-based error estimators, to their localized, adaptivity-enabling variants, highlighting the improved focus and usable localization properties of the developed indicator.

Figure 1: (a) Pointwise functional error, (b) corresponding pointwise error estimator, (c) corresponding pointwise improved error estimator with localization.

The distinction between uniform and DWR-adaptive sampling is illustrated in Figure 2, showing visible concentration of points in regions most relevant to the goal output.

Figure 2: Comparison of uniform sampling (left) vs. sampling using the DWR measure $p_\mu$ (right).

Across studied cases, DWR-based adaptivity uniformly produces either faster error decay, significantly improved final accuracy levels, or both, over naive or residual-based sampling strategies.

Analysis and Implications

Theoretical Impact

The presented framework demonstrates that DWR error control and localization—well-developed in numerical analysis—can be fully realized in neural function spaces. The absence of linear (Galerkin) structure in neural network function sets, rather than being a hindrance, alleviates concerns about orthogonality degeneracies that limit standard mesh-based DWR estimators. This observation underscores a fundamental difference in approximation theory and estimator construction in the context of nonlinear, non-convex neural network sets.

Practical Implications

• Efficient learning for QoIs: The adaptive scheme enables resource-efficient attainment of accuracy in functionals, a common demand in engineering, scientific computing, and uncertainty quantification tasks.
• Meshless high-dimensional effectiveness: The strategy preserves one of PINNs’ main selling points—scaling beyond low-dimensional or mesh-amenable geometries—by avoiding any reliance on mesh-based structures.
• Early stopping and quality control: The estimator can serve as a practical stopping criterion during training, as it reliably tracks the functional error from early epochs.

Future Directions

Open research challenges and possible advancements include:

• Theoretical underpinnings of estimator effectiveness: The consistently high performance of neural DWR estimators, even with rough adjoint/primal approximations, requires mathematical clarification.
• Integration of sampling weights in Deep Ritz loss: Further theoretical and empirical optimization may be attained by explicit weighting in the energy functional, as suggested by importance sampling research.
• Modeling the estimated sampling measure: Approximating or learning the sampling measure itself via transport maps or rearrangement techniques (e.g., Knothe–Rosenblatt), as mentioned in recent work, could further boost efficiency and generality.

Conclusion

This paper establishes a general, mesh-free, neural network-native methodology for goal-oriented a posteriori error estimation and adaptive sampling in the context of PINNs and the Deep Ritz method. By transferring DWR principles to the neural function space and localizing the estimator, the authors demonstrate both strong empirical gains in functional error control and illuminate new directions for the analysis of adaptive machine learning for scientific computing. Analytical and technical questions—especially regarding the unique properties of neural variational solvers—remain, suggesting a fruitful domain for further foundational investigation and practical advances in adaptive scientific machine learning.