Self-adaptive loss balanced Physics-informed neural networks for the incompressible Navier-Stokes equations (2104.06217v1)

Abstract: There have been several efforts to Physics-informed neural networks (PINNs) in the solution of the incompressible Navier-Stokes fluid. The loss function in PINNs is a weighted sum of multiple terms, including the mismatch in the observed velocity and pressure data, the boundary and initial constraints, as well as the residuals of the Navier-Stokes equations. In this paper, we observe that the weighted combination of competitive multiple loss functions plays a significant role in training PINNs effectively. We establish Gaussian probabilistic models to define the loss terms, where the noise collection describes the weight parameter for each loss term. We propose a self-adaptive loss function method, which automatically assigns the weights of losses by updating the noise parameters in each epoch based on the maximum likelihood estimation. Subsequently, we employ the self-adaptive loss balanced Physics-informed neural networks (lbPINNs) to solve the incompressible Navier-Stokes equations,\hspace{-1pt} including\hspace{-1pt} two-dimensional\hspace{-1pt} steady Kovasznay flow, two-dimensional unsteady cylinder wake, and three-dimensional unsteady Beltrami flow. Our results suggest that the accuracy of PINNs for effectively simulating complex incompressible flows is improved by adaptively appropriate weights in the loss terms. The outstanding adaptability of lbPINNs is not irrelevant to the initialization choice of noise parameters, which illustrates the robustness. The proposed method can also be employed in other problems where PINNs apply besides fluid problems.

• The paper introduces a self-adaptive loss weighting method for PINNs that dynamically balances multiple loss terms using Gaussian probabilistic models.
• It demonstrates that the approach achieves relative L2 errors as low as 1e-4, outperforming traditional static-weight methods in benchmark flow simulations.
• The study paves the way for enhanced convergence and reduced computation times in fluid dynamics and other PDE-governed applications.

Self-adaptive Loss Balanced Physics-informed Neural Networks for the Incompressible Navier-Stokes Equations

The paper addresses the integration of self-adaptive loss balancing in Physics-Informed Neural Networks (PINNs) for solving the incompressible Navier-Stokes equations. PINNs have been leveraged as a surrogate modeling methodology, enabling the solution of partial differential equations (PDEs) by embedding physical laws within neural network frameworks. They offer a promising alternative to traditional computational fluid dynamics (CFD) methods, especially in handling complex geometries and inverse problems.

Key Contributions

The authors propose an adaptive weighting strategy for the loss components in PINNs, crucial for the accurate and efficient simulation of incompressible flows. They introduce a self-adaptive method that incorporates Gaussian probabilistic models within the loss function configuration, allowing for dynamically adjusted weights. This is achieved via the maximum likelihood estimation, where noise parameters representing the aleatoric uncertainty are used to balance different loss terms during network training.

This self-adaptive approach addresses the significant challenge of selecting fixed weights for multiple loss functions, which previously involved manual tuning. The strategy allows the network to automatically adjust to the fluid dynamics context, balancing the PDE residuals, initial and boundary conditions, and observational data. The adaptability and robustness of the proposed method are evaluated through simulations of several canonical flow problems, including the Kovasznay flow, cylinder wake, and Beltrami flow.

Numerical Results and Advancements

Quantitative assessments illustrate meaningful improvements in prediction accuracy compared to traditional PINNs. For each of the benchmark problems, the self-adaptive method resulted in relative L2 errors as low as $10^{-4}$, significantly outperforming static-weight methods which yielded errors around $10^{-3}$ to $10^{-2}$. This achievement demonstrates the capability of the lbPINNs (loss balanced PINNs) in overcoming the limitations of existing network configurations that may become trapped in local optima due to inappropriate loss weight allocations.

Implications and Future Work

This paper emphasizes a notable advancement in the application of PINNs by introducing loss adaptability, paving the way for improved convergence and reduced computation times across a variety of flow simulations. Practically, the method holds potential beyond fluid mechanics, applicable to other physic-based modeling domains where PDEs govern the phenomena.

The work also invites further explorations into the theoretical underpinnings of optimizing loss balance adaptively. Future research could explore a granular understanding of how these parameters influence the convergence dynamics and generalization capabilities of neural networks across more diverse and complex systems. Additional work could also focus on incorporating other forms of uncertainty, like epistemic uncertainty, to further bolster the reliability and efficacy of PINNs in real-world applications.

This paper contributes to the dialogue on enhancing computational methods for fluid dynamics through machine learning, suggesting a trajectory where neural network-based solvers could routinely replace or augment classical numerical methods, particularly for real-time and large-scale simulations.