CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method (2110.15832v2)

Abstract: In this study, novel physics-informed neural network (PINN) methods for coupling neighboring support points and their derivative terms which are obtained by automatic differentiation (AD), are proposed to allow efficient training with improved accuracy. The computation of differential operators required for PINNs loss evaluation at collocation points are conventionally obtained via AD. Although AD has the advantage of being able to compute the exact gradients at any point, such PINNs can only achieve high accuracies with large numbers of collocation points, otherwise they are prone to optimizing towards unphysical solution. To make PINN training fast, the dual ideas of using numerical differentiation (ND)-inspired method and coupling it with AD are employed to define the loss function. The ND-based formulation for training loss can strongly link neighboring collocation points to enable efficient training in sparse sample regimes, but its accuracy is restricted by the interpolation scheme. The proposed coupled-automatic-numerical differentiation framework, labeled as can-PINN, unifies the advantages of AD and ND, providing more robust and efficient training than AD-based PINNs, while further improving accuracy by up to 1-2 orders of magnitude relative to ND-based PINNs. For a proof-of-concept demonstration of this can-scheme to fluid dynamic problems, two numerical-inspired instantiations of can-PINN schemes for the convection and pressure gradient terms were derived to solve the incompressible Navier-Stokes (N-S) equations. The superior performance of can-PINNs is demonstrated on several challenging problems, including the flow mixing phenomena, lid driven flow in a cavity, and channel flow over a backward facing step. The results reveal that for challenging problems like these, can-PINNs can consistently achieve very good accuracy whereas conventional AD-based PINNs fail.

• The paper introduces CAN-PINN, a novel framework that couples automatic and numerical differentiation to significantly improve the speed and accuracy of physics-informed neural networks.
• CAN-PINN achieves 1-2 orders of magnitude faster training and reliably high accuracy compared to traditional methods, even when trained with sparse data.
• The method is shown to be effective in solving various fluid dynamics problems governed by Navier-Stokes equations and accurately inferring parameters in inverse modeling tasks.

CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method

The paper "CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method" presents a novel approach to enhance the efficacy and accuracy of physics-informed neural networks (PINNs) through a coupled differentiation scheme that merges automatic differentiation (AD) and numerical differentiation (ND). PINNs are renowned for integrating the governing physics, typically modeled by differential equations, directly into the neural network's architecture, thereby constraining the network to comply with physical laws. This technique has shown promise in solving both forward and inverse problems involving ODEs and PDEs without relying heavily on large datasets.

The primary innovation in this work is the CAN-PINN framework, which leverages the complementary strengths of AD and ND in the computation of the training loss. While AD is beneficial for precisely calculating gradients, its dependence on dense collocation points makes conventional PINNs computationally demanding and often ineffective under sparse sampling. Conversely, ND enables efficient training under sparse conditions but becomes limited in accuracy due to interpolation errors. By fusing these methods into the CAN-PINN approach, the authors claim significant improvements in training speed and numerical accuracy—even surpassing ND-based PINNs by 1-2 orders of magnitude and achieving reliably high accuracy where AD-based techniques typically fail.

This methodology is demonstrated through a series of fluid dynamics problems governed by the Navier-Stokes equations, with numerical instantiations tailored for convection and pressure gradient terms. The results convincingly illustrate the reduced dispersion and dissipation errors in CAN-PINNs compared to traditional ND-based schemes. For example, the solution of flow mixing phenomena, lid-driven cavity flow, and channel flow over a backward-facing step revealed that CAN-PINNs consistently produced accurate results across varied levels of collocation density—something that conventional AD-based PINNs struggled with. The framework also excelled in inverse modelling tasks, accurately inferring parameters such as the Reynolds number from sparse data.

The contributions of this paper have practical implications for the deployment of PINNs in computational fluid dynamics, offering a pathway to tackle complex differential equations with high efficiency and reliability. Theoretically, by marrying the precision of AD with the flexibility of ND, CAN-PINNs represent an advancement in the neural computation of physics-informed problems, potentially paving the way for more robust PINNs that can handle higher-dimensional and irregular domains.

Future work could explore extending this framework to incorporate other numerical schemes via Taylor series expansions, thus broadening the applicability of PINNs across diverse scientific computing domains. The authors acknowledge that selecting appropriate numerical schemes and hyperparameters in CAN-PINNs is non-trivial, highlighting an avenue for research into automated or adaptive selection mechanisms to bolster their predefined methodology. Moreover, investigations into alternative sampling strategies, as discussed in the paper, could further enhance the adaptability of CAN-PINN models to different geometries and complex physical phenomena.

In conclusion, the CAN-PINN framework signifies a meaningful stride in physics-informed neural network research, offering a compelling alternative to current methods by leveraging coupled differentiation to improve computational efficiency and accuracy in solving differential equations. This advancement firmly positions CAN-PINNs as a robust tool in tackling fluid dynamics problems and inverse modeling tasks, setting a foundation for future exploration into more specialized and adaptive PINN architectures.