Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations (1711.10561v1)

Abstract: We introduce physics informed neural networks -- neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. In this two part treatise, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct classes of algorithms, namely continuous time and discrete time models. The resulting neural networks form a new class of data-efficient universal function approximators that naturally encode any underlying physical laws as prior information. In this first part, we demonstrate how these networks can be used to infer solutions to partial differential equations, and obtain physics-informed surrogate models that are fully differentiable with respect to all input coordinates and free parameters.

• The paper introduces physics-informed neural networks (PINNs) that integrate PDE constraints into deep learning frameworks to achieve accurate, data-driven solutions.
• It employs a continuous time model using automatic differentiation to closely approximate PDE solutions, exemplified by the Burgers' equation with minimal training data.
• Discrete time models leveraging implicit Runge-Kutta schemes enable efficient temporal stepping, demonstrating robustness on complex systems like the Allen-Cahn equation.

Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

The paper "Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations" by Maziar Raissi, Paris Perdikaris, and George Em Karniadakis introduces a novel computational approach, termed physics informed neural networks (PINNs), for solving and discovering solutions to a broad class of nonlinear partial differential equations (PDEs). This work establishes a framework that unifies deep learning with the governing principles of physical phenomena, presenting two primary methodologies: continuous time and discrete time models.

Summary

The authors present PINNs as neural networks trained to solve supervised learning tasks while adhering to physical laws codified as nonlinear PDEs. The methods are tailored to handle both data-driven solutions and data-driven discovery of PDEs. PINNs capitalize on embedding prior physical knowledge as constraints during the training process, thereby enhancing data efficiency and robustness.

Continuous Time Models

The continuous time model approximates the solution of the PDE directly using a deep neural network, which is then constrained to satisfy the PDE through automatic differentiation. A thorough evaluation exemplifies this with the Burgers' equation, demonstrating the network's capability to accurately infer solutions even with a limited number of training points. This approach highlights its robustness and capability to circumvent overfitting.

Numerical Experimentation: Burgers' Equation

Using only 100 training data points from boundary and initial conditions, a 9-layer network captured the nonlinear behavior of the Burgers' equation with a relative error of $6.7 \times 10^{-4}$. This result surpasses previous methods in both efficiency and accuracy. The inclusion of mean squared error terms for both the data points and the PDE constraints (at collocation points) enforces convergence, even with high expressiveness in the network architecture.

Discrete Time Models

To address computational concerns in higher dimensions, the authors employ a structured neural network representation based on the Runge-Kutta time-stepping schemes. This discretization allows the network to step through time efficiently while conserving the imposed PDE constraints.

Numerical Experimentation: High-Order Implicit Runge-Kutta

By leveraging implicit high-order Runge-Kutta schemes with up to 500 stages, the authors predict the solution of the Burgers’ equation over large temporal spans (e.g., $\Delta t = 0.8$) with minimal error accumulation, circumventing conventional stability and step-size constraints. This is further evidenced by a relatve error of $8.2 \times 10^{-4}$.

Case Study: Allen-Cahn Equation

Further validation is provided for the Allen-Cahn equation, a reaction-diffusion system, showcasing PINNs' adaptability to handle varying nonlinearities and periodic boundary conditions. The method accurately predicted the intricate dynamics, including the formation of sharp internal layers, starting from noisy initial data.

Implications and Future Outlook

The methodology showcased by PINNs offers a paradigm shift in scientific computation, particularly for problems where traditional methods face practical limitations due to high-dimensionality or data scarcity. The integration of deep learning with physical laws stands to benefit numerous applications, from predictive modeling in engineering systems to real-time forecasting in geosciences.

Despite significant advances, aspects like uncertainty quantification (UQ) remain underexplored within the context of PINNs. Future research is poised to develop methods that incorporate UQ, analogous to approaches utilized in Gaussian Process-based techniques, thus enabling a predictive framework that accounts for inherent data uncertainties.

Conclusion

This paper lays the groundwork for a versatile and scalable framework that bridges the gap between advanced machine learning and classical mathematical physics. The promising results and the open-source availability of codes entice further explorations and refinements, likely propelling developments in both theoretical and applied arenas of computational science.