Invariant deep neural networks under the finite group for solving partial differential equations (2407.20560v2)

Abstract: Utilizing physics-informed neural networks (PINN) to solve partial differential equations (PDEs) becomes a hot issue and also shows its great powers, but still suffers from the dilemmas of limited predicted accuracy in the sampling domain and poor prediction ability beyond the sampling domain which are usually mitigated by adding the physical properties of PDEs into the loss function or by employing smart techniques to change the form of loss function for special PDEs. In this paper, we design a symmetry-enhanced deep neural network (sDNN) which makes the architecture of neural networks invariant under the finite group through expanding the dimensions of weight matrixes and bias vectors in each hidden layers by the order of finite group if the group has matrix representations, otherwise extending the set of input data and the hidden layers except for the first hidden layer by the order of finite group. However, the total number of training parameters is only about one over the order of finite group of the original PINN size due to the symmetric architecture of sDNN. Furthermore, we give special forms of weight matrixes and bias vectors of sDNN, and rigorously prove that the architecture itself is invariant under the finite group and the sDNN has the universal approximation ability to learn the function keeping the finite group. Numerical results show that the sDNN has strong predicted abilities in and beyond the sampling domain and performs far better than the vanilla PINN with fewer training points and simpler architecture.

• The paper proposes a symmetry-enhanced architecture that incorporates finite group invariance, reducing training parameters while maintaining universal approximation properties.
• Rigorous theoretical analysis confirms that the invariant neural network accurately preserves symmetry and enhances extrapolation capabilities over unsampled domains.
• Numerical experiments show that sDNNs significantly outperform traditional PINNs in solving various PDEs, including advection, sine-Gordon, and Poisson equations.

Invariant Deep Neural Networks Under the Finite Group for Solving Partial Differential Equations

This paper investigates a symmetry-enhanced approach for deep neural networks (sDNN) to solve partial differential equations (PDEs) with a focus on integrating finite group symmetry directly into the neural network architecture. By addressing the limitations of Physics-Informed Neural Networks (PINNs), such as poor predictive accuracy beyond the sampling domain, this paper proposes a structured modification to the neural network that leverages the symmetry properties of PDEs.

The authors identify that while PINNs are effective in various scenarios, they struggle to accurately predict solutions outside of the trained (sampling) domain. To mitigate this, sDNNs are proposed, which inherently include the symmetry properties of the PDE's solutions by designing the network to be invariant under finite groups.

Key Contributions

1. Symmetry-Enhanced Architecture: The sDNN is designed to be invariant under the action of a finite group. This is achieved either by expanding the dimensions of weight matrices and bias vectors or by extending the input data and hidden layers. This architecture ensures that the total number of training parameters is significantly reduced, with only about one over the order of the finite group of the original PINN size.
2. Rigorous Theoretical Analysis: The paper rigorously proves that the proposed sDNN maintains the invariance under the finite group and possesses the universal approximation ability to learn functions that keep the finite group properties. This is a significant theoretical assurance that the symmetry properties are not only heuristically included but also mathematically guaranteed.
3. Improved Solution Extrapolation: Numerical experiments demonstrate that sDNNs predict solutions with high accuracy both within and beyond the sampling domain. This is contrasted with vanilla PINNs, which show a marked drop in predictive accuracy outside the training domain.

Numerical Results and Claims

The paper provides robust numerical results across various types of PDEs. Notably:

• For the advection equation with even symmetry, sDNNs significantly outperform PINNs. When sampling from a domain, the predictive accuracy in an extrapolated domain remained consistent with the sampling domain for sDNNs but not for PINNs.
• For the sine-Gordon equation with circulant symmetry, similar improvements in prediction accuracy are noted. The architecture ensures better learning capacity, exemplified by lower loss values and fewer iterations required to converge.
• The paper explores different finite group orders using the Poisson equation and shows that as the order of the symmetry group increases, the network manages to deliver more accurate predictions in larger unsampled domains.
• For PDEs like the nonlinear wave equation and the Korteweg-de Vries (KdV) equation with no straightforward matrix representation for the symmetry group, sDNNs still maintain their edge over PINNs by accurately reflecting the symmetry properties in their solution domains.

Implications and Future Developments

The implications of this research are profound for both theoretical and practical aspects of solving PDEs using neural networks:

1. Theoretical Implications: This work establishes a new paradigm by incorporating finite group symmetry into neural network architecture. This approach opens avenues for further exploration into continuous symmetries and their integration with neural networks, leading to potential advancements in handling more complex differential equations characteristic of various physical phenomena.
2. Practical Utilization: sDNNs provide a pathway for more reliable and accurate models in scientific computing, where PDEs are prevalent. This method can be particularly beneficial in fields requiring high precision and robustness in extrapolated predictions, such as fluid dynamics, material science, and climate modeling.
3. Future Developments: The integration of continuous symmetries represents a promising yet challenging direction. Further research into devising architecture that accommodates infinite group elements of continuous symmetries could significantly enhance the performance and application scope of symmetric neural networks.

In conclusion, this paper presents a well-substantiated advancement in utilizing deep neural networks for PDEs by embedding finite group invariance directly into the network structure. This strategy not only improves accuracy but also significantly enhances the prediction capability beyond the training domain, marking a noteworthy contribution to the domain of numerical methods and scientific computing.