Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations (1706.04702v1)

Abstract: We propose a new algorithm for solving parabolic partial differential equations (PDEs) and backward stochastic differential equations (BSDEs) in high dimension, by making an analogy between the BSDE and reinforcement learning with the gradient of the solution playing the role of the policy function, and the loss function given by the error between the prescribed terminal condition and the solution of the BSDE. The policy function is then approximated by a neural network, as is done in deep reinforcement learning. Numerical results using TensorFlow illustrate the efficiency and accuracy of the proposed algorithms for several 100-dimensional nonlinear PDEs from physics and finance such as the Allen-Cahn equation, the Hamilton-Jacobi-BeLLMan equation, and a nonlinear pricing model for financial derivatives.

• The paper introduces a groundbreaking deep learning algorithm that reformulates high-dimensional PDEs as BSDEs by approximating the gradient as a policy function.
• It leverages a neural network architecture with techniques like batch normalization, ReLU activations, and stochastic optimization to recursively approximate values and gradients.
• Numerical experiments in 100-dimensional models, including the Allen-Cahn and Hamilton-Jacobi-Bellman equations, demonstrate relative L1 errors as low as 0.0017 within minutes.

Deep Learning-Based Numerical Methods for High-Dimensional Parabolic PDEs and BSDEs

Abstract

The paper by Weinan E, Jiequn Han, and Arnulf Jentzen proposes an innovative algorithm leveraging deep learning for solving high-dimensional parabolic partial differential equations (PDEs) and backward stochastic differential equations (BSDEs). They draw an analogy between BSDEs and reinforcement learning, with the gradient of the BSDE solution playing the role of the policy function, approximated by a neural network. The algorithm demonstrates effectiveness on $100$-dimensional nonlinear PDEs from physics and finance.

Introduction

The challenge of efficiently solving high-dimensional PDEs, especially those with hundreds of dimensions, has long been a significant barrier in applied mathematics due to the curse of dimensionality. While Monte Carlo methods and other numerical strategies can address specific cases, particularly linear parabolic PDEs, the need for generalized, practical algorithms persists. Recent advancements in deep learning, particularly in high-dimensional spaces, present a promising avenue for overcoming these challenges. The authors aim to explore and exploit the connections between deep learning, reinforcement learning, and BSDEs to solve such high-dimensional PDEs effectively.

Methodology

Their approach hinges on three main steps:

1. Nonlinear Feynman-Kac Formula: Utilizing the formula to reformulate PDEs as equivalent BSDEs.
2. Stochastic Control Interpretation: Viewing BSDEs as stochastic control problems where the solution gradient plays the policy function's role.
3. Deep Learning Approximation: Approximating this high-dimensional policy function using deep neural networks.

By focusing on terminal value problems, the authors align the formulation with backward BSDE dynamics, allowing transformation and application of deep learning via neural networks. The PDEs then serve as the learning problems where neural nets approximate the policy and value functions.

Numerical Implementation

The primary innovation is the recursive approximation of values and gradients using deep neural networks (DNNs). Key components include:

• Network Structure: Four layers including hidden layers with batch normalization and ReLU activation functions.
• Stochastic Optimization: Implementing SGD-type methods and Adam optimizer to minimize the loss function reflecting the terminal condition error.

Examples addressed include the Allen-Cahn equation, Hamilton-Jacobi-BeLLMan equations, and financial derivative pricing models with different borrowing and lending rates.

Numerical Experiments

Allen-Cahn Equation

• Setup: $100$-dimensional, cubic nonlinearity in the PDE.
• Results: Achieves a relative $L^1$ error of $0.0030$ in approximately $650$ seconds.

Hamilton-Jacobi-BeLLMan Equation

• Setup: $100$-dimensional, explicitly solvable via Cole-Hopf transformation.
• Results: Relative $L^1$ error of $0.0017$ in $330$ seconds.

Financial Derivatives with Borrow-Lend Rates

• Setup: Models the pricing of European financial derivatives.
• Results: Relative $L^1$ error of $0.0039$ in $617$ seconds.

These compelling numerical results underline the method's robustness across different complex, high-dimensional PDEs.

Implications and Future Work

The paper establishes a significant step in applying deep learning to numerically approximate high-dimensional PDEs, leveraging the reinforcement learning framework. Practical implications include advancements in finance, physics, and any domain where such high-dimensional PDEs manifest.

Future research could enhance theoretical grounding, extend the approach to other types of PDEs and stochastic processes, and integrate more sophisticated machine learning techniques to improve efficiency and accuracy.

Conclusion

Weinan E, Jiequn Han, and Arnulf Jentzen propose a promising deep learning-based method to solve high-dimensional parabolic PDEs and BSDEs efficiently. Their approach shows substantial improvements in terms of accuracy and computational feasibility, marking a pivotal progression in this challenging field. By bridging reinforcement learning and deep neural networks with high-dimensional PDEs, they pave the way for new methodologies in computational mathematics and its applications.