A unified deep artificial neural network approach to partial differential equations in complex geometries (1711.06464v2)

Abstract: In this paper we use deep feedforward artificial neural networks to approximate solutions to partial differential equations in complex geometries. We show how to modify the backpropagation algorithm to compute the partial derivatives of the network output with respect to the space variables which is needed to approximate the differential operator. The method is based on an ansatz for the solution which requires nothing but feedforward neural networks and an unconstrained gradient based optimization method such as gradient descent or a quasi-Newton method. We show an example where classical mesh based methods cannot be used and neural networks can be seen as an attractive alternative. Finally, we highlight the benefits of deep compared to shallow neural networks and device some other convergence enhancing techniques.

• The paper presents a novel deep ANN framework that accurately approximates PDE solutions without the need for mesh discretization.
• It adapts backpropagation to compute spatial derivatives, automatically enforcing boundary conditions while managing high-dimensional challenges.
• Numerical examples demonstrate that the approach outperforms traditional methods, with deeper networks reducing iterations for desired accuracy.

A Unified Deep ANN Approach to PDEs in Complex Geometries

The paper "A unified deep artificial neural network approach to partial differential equations in complex geometries" by Jens Berg and Kaj Nyström presents a method for solving partial differential equations (PDEs) using deep feedforward artificial neural networks (ANNs). This approach is particularly aimed at addressing challenges posed by complex geometries, which often limit the applicability of traditional mesh-based methods.

Overview

The authors explore the use of deep ANNs to approximate solutions of PDEs, advocating for their potential as an alternative to classical techniques like finite element methods (FEM), finite differences (FDM), and finite volumes (FVM). These traditional methods require discretizing the domain into mesh points, which can be cumbersome in complex geometries. Conversely, the approach proposed in this paper transcends these limitations by employing a mesh-free methodology using ANNs.

Methodological Contributions

The authors' method revolves around an ansatz for the solution that only necessitates standard feedforward neural networks combined with unconstrained, gradient-based optimization methods. They demonstrate modifications to the backpropagation algorithm that allow for computing the partial derivatives of the network output necessary for approximating the differential operator within the PDE.

Key elements of their framework include:

• Modified Backpropagation: The paper details how the backpropagation algorithm is adapted for gradient computation concerning spatial variables. This adaptation is crucial for solving PDEs as it involves derivatives of the ANN outputs.
• Boundary Treatment: By utilizing deep ANNs, the authors propose a novel solution ansatz capable of automatically satisfying boundary conditions. This feature simplifies the solution process, avoiding additional complex computations.
• Dimensionality and Complexity Handling: The method demonstrates strong potential in handling high-dimensional PDEs, where traditional methods become infeasible due to computational constraints.

Numerical Examples and Results

The paper provides a series of numerical examples to demonstrate the efficacy of the proposed method. These examples include linear advection and diffusion problems in both one and two dimensions, extending up to complex 2D geometries like the geographic boundary of Sweden. The results indicate that deep ANNs can achieve satisfactory approximations where classical methods struggle, particularly in irregular and high-complexity domains.

The authors also highlight the added value of deep versus shallow networks, showing that deeper networks significantly reduce the number of iterations required to achieve a given level of accuracy. This observation underscores the importance of network depth in the context of PDE approximation.

Implications and Future Directions

The paper's contributions suggest compelling practical and theoretical implications, primarily in the context of solving PDEs with complex geometries and high-dimensional spaces. The utilization of ANNs introduces flexibility and efficiency, opening new avenues for research in computational mathematics and engineering disciplines.

Looking forward, this work may inspire future research into integrating other forms of neural networks, such as convolutional or recurrent architectures, which could further enhance the modeling of temporal and spatial dependencies within PDEs. Additionally, advanced training techniques like dropout and batch normalization could be employed to tackle challenges such as the vanishing gradient problem in even deeper networks.

Overall, this research illustrates a promising direction for leveraging the power of deep learning in solving complex mathematical models, offering a foundation for further developing neural network solutions in computationally challenging scenarios.