Can Physics-Informed Neural Networks beat the Finite Element Method? (2302.04107v1)

Abstract: Partial differential equations play a fundamental role in the mathematical modelling of many processes and systems in physical, biological and other sciences. To simulate such processes and systems, the solutions of PDEs often need to be approximated numerically. The finite element method, for instance, is a usual standard methodology to do so. The recent success of deep neural networks at various approximation tasks has motivated their use in the numerical solution of PDEs. These so-called physics-informed neural networks and their variants have shown to be able to successfully approximate a large range of partial differential equations. So far, physics-informed neural networks and the finite element method have mainly been studied in isolation of each other. In this work, we compare the methodologies in a systematic computational study. Indeed, we employ both methods to numerically solve various linear and nonlinear partial differential equations: Poisson in 1D, 2D, and 3D, Allen-Cahn in 1D, semilinear Schr\"odinger in 1D and 2D. We then compare computational costs and approximation accuracies. In terms of solution time and accuracy, physics-informed neural networks have not been able to outperform the finite element method in our study. In some experiments, they were faster at evaluating the solved PDE.

• The paper demonstrates that FEM delivers lower computational times and smaller errors than PINNs across multiple benchmark PDEs.
• The paper employs a rigorous methodology by comparing both methods on linear and nonlinear equations including Poisson, Allen-Cahn, and Schrödinger equations.
• The paper highlights that while PINNs offer fast evaluations post-training, their high computational cost calls for enhanced optimization strategies.

Physics-Informed Neural Networks vs. Finite Element Method: A Comparative Study

The paper "Can Physics-Informed Neural Networks beat the Finite Element Method?" embarks on a detailed computational analysis comparing Physics-Informed Neural Networks (PINNs) with the classical Finite Element Method (FEM) for solving partial differential equations (PDEs). As PDEs are fundamental to modeling a wide range of phenomena across physical, biological, and socioeconomic domains, efficient and accurate numerical solutions are crucial. FEM has long been the traditional choice for this, whereas PINNs present a novel, data-driven alternative emerging from advances in deep learning.

Overview of PDEs and Numerical Methods

PDEs are integral in mathematically describing systems' behavior over space and time. For decades, FEM has been widely employed due to its convergence guarantees and well-established error estimators. However, FEM's reliance on spatial discretization often leads to the curse of dimensionality, especially in large-scale and high-dimensional problems.

PINNs offer an innovative approach by integrating neural networks with physical laws expressed as differential equations, eliminating the need for discretization. Their formulation allows for the solution of general PDEs without significant degradation in high-dimensional settings. Despite their theoretical promise, PINNs' performance relative to FEM has predominantly been assessed in isolation until now.

Methodological Framework

The authors set out to empirically compare PINNs and FEM across numerous linear and nonlinear PDEs: the Poisson equation in 1D, 2D, and 3D; the Allen-Cahn equation in 1D; and the semilinear Schrödinger equation in 1D and 2D. They measure and contrast the computational cost and approximation accuracy of each method, focusing on both solution and evaluation phases.

FEM solutions were facilitated using FEniCS, leveraging its capabilities for efficient and stable computation. PINNs involved training deep neural networks to satisfy PDE residuals and boundary conditions, employing both the Adam and L-BFGS optimizers for rigorous training pursuits.

Key Findings and Observations

1. Computational Efficiency: Across all PDEs, FEM consistently demonstrated superior efficiency in solution time. PINNs required significantly more computational resources and training epochs, highlighting an area requiring optimization for practical deployment.
2. Accuracy in Solution: In terms of accuracy, FEM solutions generally outperformed PINN approximations, achieving lower relative errors with fewer computational demands. This suggests that while PINNs may provide feasible solutions, traditional methods reliably deliver precision, especially in classical settings.
3. Evaluation Speed: PINNs showcased their strength in evaluation speed on new data points post-training, particularly noticeable in higher-dimensional spaces. Nonetheless, this advantage waned when overall computational costs, including training time, were considered.
4. Dimensional Flexibility: The transition from 2D to 3D spaces did not significantly impact PINNs, asserting their potential in scalable computations across dimensions—a significant merit where FEM might face challenges.
5. Critical Challenges with Nonlinear PDEs: Notably, PINNs wrestled with the Allen-Cahn equation for low $\epsilon$ values, where solutions exhibited sharp transitions—an area where FEM's robustness in handling discontinuities presents evident advantages.

Implications and Future Directions

This paper elicits nuanced insights into the practical viability of PINNs versus FEM. While PINNs integrate seamlessly across dimensional scales and demonstrate speed post-training, their application remains limited by high computational costs and not-universal accuracy. Contrarily, FEM retains its position as a trustworthy, efficient solution for many current engineering and physical science problems despite high-dimensional constraints.

For PINNs to compete robustly, there's a need for improved network architectures, optimization strategies, and possibly hybrid methods that blend deep learning with the stability assurances of FEM. Future research could explore such integrations, focusing on adaptivity and parameterized PDEs—domains where PINNs' adaptability might present breakthroughs over classical methods.

Ultimately, while innovations like PINNs invigorate the sphere of numerical analysis with new possibilities, this comprehensive paper reaffirms that classical methods, with their known strengths, remain indispensable tools in many areas of PDE-driven modeling, at least within the test cases examined.