DGM: A deep learning algorithm for solving partial differential equations (1708.07469v5)

Abstract: High-dimensional PDEs have been a longstanding computational challenge. We propose to solve high-dimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential operator, initial condition, and boundary conditions. Our algorithm is meshfree, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, the neural network is trained on batches of randomly sampled time and space points. The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to $200$ dimensions. The algorithm is also tested on a high-dimensional Hamilton-Jacobi-BeLLMan PDE and Burgers' equation. The deep learning algorithm approximates the general solution to the Burgers' equation for a continuum of different boundary conditions and physical conditions (which can be viewed as a high-dimensional space). We call the algorithm a "Deep Galerkin Method (DGM)" since it is similar in spirit to Galerkin methods, with the solution approximated by a neural network instead of a linear combination of basis functions. In addition, we prove a theorem regarding the approximation power of neural networks for a class of quasilinear parabolic PDEs.

• The paper presents the Deep Galerkin Method (DGM), a meshfree deep learning approach that solves high-dimensional PDEs with relative errors as low as 0.22% in some cases.
• It employs a Galerkin-inspired framework that trains neural networks on randomly sampled time and space points to overcome the curse of dimensionality.
• Numerical tests on free boundary, Hamilton-Jacobi-Bellman, and Burgers' equations demonstrate DGM’s practical accuracy and strong theoretical convergence guarantees.

An Overview of the Deep Galerkin Method for Solving High-Dimensional PDEs

The paper, "DGM: A Deep Learning Algorithm for Solving Partial Differential Equations," authored by Justin Sirignano and Konstantinos Spiliopoulos, presents a novel deep learning approach aimed at addressing the longstanding computational challenge of solving high-dimensional partial differential equations (PDEs). This approach, termed the Deep Galerkin Method (DGM), leverages deep neural networks and operates in a meshfree manner to approximate the solutions of high-dimensional PDEs, including those frequently encountered in physics, engineering, and finance.

Key Contributions

1. Meshfree Approach: The DGM method avoids the necessity of forming a high-dimensional mesh, which becomes computationally intractable as the dimensionality increases. Instead, the neural network is trained on randomly sampled time and space points, thus circumventing the curse of dimensionality.
2. Galerkin-inspired Methodology: Unlike traditional Galerkin methods that use a linear combination of basis functions, DGM utilizes deep neural networks to approximate the PDE solution, capturing the functional form more flexibly and efficiently.
3. Numerical Results Across Domains: The methodology is rigorously tested on high-dimensional free boundary PDEs, the Hamilton-Jacobi-BeLLMan (HJB) PDE, and Burgers' equation, demonstrating its numerical accuracy and efficiency in dimensions up to 200.
4. Approximation Theorem for Neural Networks: The paper also includes a theoretical result showing that neural networks can approximate the solution of quasilinear parabolic PDEs with vanishing L2 error as the number of hidden units increases.

Numerical Examples and Results

Free Boundary PDEs

The paper examines high-dimensional free boundary PDEs, commonly used in financial contexts to price American options. Remarkably, the DGM method achieves significant accuracy even in 200-dimensional spaces. For instance, it demonstrates a relative error of 0.22% when compared to a semi-analytic solution for certain parameter settings in the Black-Scholes model framework.

Hamilton-Jacobi-BeLLMan PDE

DGM is applied to a Hamilton-Jacobi-BeLLMan equation related to the optimal control of a stochastic heat equation. With 21 spatial dimensions, the method produces results with an average percent error of only 0.1%, verified against a semi-analytic solution.

Burgers' Equation Across Different Configurations

To showcase DGM’s versatility, the paper includes experiments on Burgers' equation under various boundary and initial conditions, viscosities, and nonlinear terms. The method accurately captures the solutions’ behavior, including shock and boundary layers, as evidenced by the close alignment with finite difference solutions.

Theoretical Insights and Implications

The paper provides rigorous proofs regarding the convergence of the neural network approximation to the actual PDE solution. Key theoretical insights include:

• Universal Approximation: Theorem 3 of Hornik et al. is used to establish that sufficiently large neural networks can approximate classical solutions to PDEs arbitrarily well.
• L2 Control and Convergence: By extending this result, the authors show that the neural network approximation error in the L2 norm can be made arbitrarily small, strongly implying convergence to the true PDE solution.
• Strong Convergence: Under reasonably mild conditions, the paper demonstrates that neural networks converge strongly in the appropriate function spaces.

Practical and Theoretical Implications

From a practical perspective, DGM opens new avenues for efficiently solving high-dimensional PDEs without resorting to the computationally prohibitive methods traditionally employed. The ability to handle high-dimensional spaces is particularly relevant in fields such as financial engineering, where models often involve numerous correlated assets.

Theoretically, the convergence results lay a solid foundation for further exploration of neural network-based approaches to PDEs. They suggest that deep learning can indeed capture the essential dynamics of high-dimensional PDEs, provided that the architecture and training regimen are properly tailored.

Future Directions

Future research may explore:

• Extension to Other Classes of PDEs: Investigating the applicability of DGM to hyperbolic, elliptic, and partial-integral differential equations.
• Enhanced Architectures and Training Schemes: Developing more specialized neural network architectures or improved training algorithms tailored for specific types of PDEs.
• Real-world Applications: Extending the implementation to solve real-world engineering and scientific problems characterized by high-dimensional PDEs.

In conclusion, the DGM approach introduced by Sirignano and Spiliopoulos represents a promising intersection of deep learning and numerical PDE solving, potentially transforming how high-dimensional problems are approached in computational mathematics and beyond.