Multilevel domain decomposition-based architectures for physics-informed neural networks (2306.05486v3)

Abstract: Physics-informed neural networks (PINNs) are a powerful approach for solving problems involving differential equations, yet they often struggle to solve problems with high frequency and/or multi-scale solutions. Finite basis physics-informed neural networks (FBPINNs) improve the performance of PINNs in this regime by combining them with an overlapping domain decomposition approach. In this work, FBPINNs are extended by adding multiple levels of domain decompositions to their solution ansatz, inspired by classical multilevel Schwarz domain decomposition methods (DDMs). Analogous to typical tests for classical DDMs, we assess how the accuracy of PINNs, FBPINNs and multilevel FBPINNs scale with respect to computational effort and solution complexity by carrying out strong and weak scaling tests. Our numerical results show that the proposed multilevel FBPINNs consistently and significantly outperform PINNs across a range of problems with high frequency and multi-scale solutions. Furthermore, as expected in classical DDMs, we show that multilevel FBPINNs improve the accuracy of FBPINNs when using large numbers of subdomains by aiding global communication between subdomains.

• The paper extends FBPINNs by incorporating a multilevel domain decomposition strategy to overcome challenges in high-frequency, multi-scale differential equations.
• The paper demonstrates that multilevel FBPINNs outperform traditional PINNs and one-level FBPINNs in scalability and accuracy through rigorous strong and weak scaling tests.
• The study bridges classical Schwarz methods with neural network solvers, offering promising applications in fluid dynamics, geophysics, and other complex systems.

Multilevel Domain Decomposition-Based Architectures for Physics-Informed Neural Networks

The paper presents an extension to the finite basis physics-informed neural networks (FBPINNs), focusing on enhancing their ability to solve high-frequency and multi-scale differential equations. Physics-informed neural networks (PINNs) have shown promise in solving differential equations, leveraging neural networks to approximate solutions directly. However, their efficacy decreases with high-frequency or multi-scale solution spaces. This paper builds upon FBPINNs by integrating multilevel domain decomposition, inspired by classical Schwarz methods, to address the limitations seen in traditional PINNs.

Core Methodology

At the heart of this work is the incorporation of a multilevel approach into the FBPINNs. Traditional FBPINNs utilize an overlapping domain decomposition, employing multiple smaller neural networks, each tasked with a subsection of the solution space. The multilevel strategy introduces several overlapping domain decomposition levels, improving information transfer between subdomains and enhancing the scalability of FBPINNs. This approach aims to improve the accuracy of FBPINNs while maintaining computational efficiency, particularly when handling large numbers of subdomains.

Numerical Results and Performance

The paper evaluates the performance of the proposed multilevel FBPINN architecture through a series of scaling tests on differential equations, including the homogeneous Laplacian, multi-scale Laplacian, and Helmholtz problems in two dimensions. The results demonstrate that multilevel FBPINNs consistently outperform both traditional PINNs and one-level FBPINNs. One critical aspect highlighted is the ability of multilevel FBPINNs to effectively scale with an increased complexity in problem size and solution space, maintaining their performance across varying conditions.

In the strong scaling tests, where model complexity increased while keeping the problem size fixed, these networks showed better accuracy per computational effort. Meanwhile, in weak scaling tests, where the problem complexity and model capacity scaled proportionally, multilevel FBPINNs showed nearly consistent performance despite the heightened problem demands. This performance is noteworthy because it suggests that multilevel structures enable FBPINNs to handle irregular domain shapes and effectively solve high-frequency problems, surpassing traditional neural-based approaches.

Implications and Future Directions

The integration of multilevel domain decomposition with FBPINNs holds significant implications in scientific computing and complex system modeling, offering a more robust framework for solving differential equations with multi-scale properties. By borrowing concepts from domain decomposition methods, such as those in classical numerical solvers, the multilevel FBPINNs potentially bridge the gap between classical numerical analysis and modern machine learning techniques.

Practical implications include potential applications in fields like fluid dynamics, geophysics, and any domain where differential equations with high-frequency components are prevalent. The paper suggests that further investigation into optimal domain decompositions and network architectures could yield even more efficient models. Moreover, incorporating learning-based decomposition strategies or adaptive window functions could enhance versatility and applicability to a broader range of problem types.

Conclusion

This paper represents a substantial development in the use of neural networks for solving differential equations, presenting multilevel FBPINNs as a viable and effective method for increasing the accuracy and efficiency of these tasks. By leveraging multilevel decomposition strategies, this approach offers a promising direction for future research and application in scientific machine learning, addressing current limitations in handling complex, multi-scale problems.