RAMS: Residual-based adversarial-gradient moving sample method for scientific machine learning in solving partial differential equations

Abstract: Physics-informed neural networks (PINNs) and neural operators, two leading scientific machine learning (SciML) paradigms, have emerged as powerful tools for solving partial differential equations (PDEs). Although increasing the training sample size generally enhances network performance, it also increases computational costs for physics-informed or data-driven training. To address this trade-off, different sampling strategies have been developed to sample more points in regions with high PDE residuals. However, existing sampling methods are computationally demanding for high-dimensional problems, such as high-dimensional PDEs or operator learning tasks. Here, we propose a residual-based adversarial-gradient moving sample (RAMS) method, which moves samples according to the adversarial gradient direction to maximize the PDE residual via gradient-based optimization. RAMS can be easily integrated into existing sampling methods. Extensive experiments, ranging from PINN applied to high-dimensional PDEs to physics-informed and data-driven operator learning problems, have been conducted to demonstrate the effectiveness of RAMS. Notably, RAMS represents the first efficient adaptive sampling approach for operator learning, marking a significant advancement in the SciML field.