Solving the Initial Value Problem of Ordinary Differential Equations by Lie Group based Neural Network Method (2203.03479v1)

Abstract: To combine a feedforward neural network (FNN) and Lie group (symmetry) theory of differential equations (DEs), an alternative artificial NN approach is proposed to solve the initial value problems (IVPs) of ordinary DEs (ODEs). Introducing the Lie group expressions of the solution, the trial solution of ODEs is split into two parts. The first part is a solution of other ODEs with initial values of original IVP. This is easily solved using the Lie group and known symbolic or numerical methods without any network parameters (weights and biases). The second part consists of an FNN with adjustable parameters. This is trained using the error back propagation method by minimizing an error (loss) function and updating the parameters. The method significantly reduces the number of the trainable parameters and can more quickly and accurately learn the real solution, compared to the existing similar methods. The numerical method is applied to several cases, including physical oscillation problems. The results have been graphically represented, and some conclusions have been made.

• The paper introduces a hybrid method combining Lie group theory with feedforward neural networks to solve ODE IVPs with reduced parameter complexity.
• It decomposes the trial solution into a Lie-derived component and an FNN-based component, improving accuracy and accelerating learning.
• Numerical experiments on coupled nonlinear ODEs, oscillatory problems, and the Duffing equation validate the method’s efficiency and robustness.

Lie Group based Neural Network Method for Solving Initial Value Problems

This paper introduces a novel approach for solving the initial value problems (IVPs) of ordinary differential equations (ODEs) by integrating feedforward neural networks (FNNs) with the Lie group theory of differential equations. The method leverages Lie group expressions to decompose the trial solution into two components: a Lie group-derived solution and an FNN-based solution. The paper demonstrates that this hybrid approach reduces the number of trainable parameters and enhances the learning speed and accuracy compared to existing neural network methods.

Core Methodology

The methodology begins with the decomposition of an ODE's solution using Lie group theory. The trial solution is represented as the sum of two parts:

1. A solution obtained from another ODE, which shares the original IVP's initial values, solved using Lie group methods or other numerical techniques.
2. An FNN with adjustable parameters, trained to minimize an error function using backpropagation.

The method leverages the Lie group theory to find symmetries and transformations that simplify the original ODE. By identifying a suitable Lie group, the solution can be expressed in terms of group parameters, which can then be used to construct the first part of the trial solution. This part captures essential properties of the real solution near the initial point, reducing the workload for the FNN.

Implementation Details

The implementation of the Lie group-based neural network method involves the following steps:

1. Decomposition of the Differential Operator: The differential operator $D$ is decomposed into two parts, $D_1$ and $D_2$, where $D = D_1 + D_2$. $D_1$ is chosen such that the associated IVP is easily solvable.
2. Construction of the Trial Solution: The trial solution $\hat{y}(x)$ is constructed as $\hat{y}(x) = \bar{y}(x) + x\mathcal{N}(x;\theta)$, where $\bar{y}(x)$ is the solution of the associated IVP, and $\mathcal{N}(x;\theta)$ is an FNN.
3. Loss Function Optimization: The FNN is trained by minimizing the loss function $\mathcal{L}(\theta)$, which quantifies the error between the trial solution and the original ODE. The loss function is defined as:

   $$\mathcal{L}(\theta)=\frac{1}{N}\sum_k^N\sum_{i=1}^n\{\frac{d\hat{y}^k_i}{dx}- f_i(\hat{y}^k_1, \hat{y}^k_2,\cdots, \hat{y}^k_n)\}^2$$

   where $\hat{y}^k_i=\hat{y}_i(x_k, \theta)$, and the dataset $S=\{x_k\}_{k=1}^N$ consists of training points in the interval $\mathcal{O}$.

4. Network Architecture: The FNN typically consists of an input layer, one or more hidden layers, and an output layer. The choice of activation functions, number of neurons, and network depth can be tuned to optimize performance.

Numerical Experiments

The paper presents numerical experiments to demonstrate the effectiveness of the proposed method. These include:

• Two Coupled First-Order Nonlinear ODEs: The method is applied to solve a system of two coupled nonlinear ODEs, demonstrating its ability to handle complex systems.
• Linearly Forced Oscillation Problem: The method is used to solve a linearly forced oscillation problem, showcasing its applicability to physical oscillation models.
• Nonlinear Initial Value Problem of Duffing Equation: The method is applied to solve the Duffing equation, a classic example of a nonlinear oscillator, further validating its capability in handling nonlinear dynamics.

The results of these experiments indicate that the Lie group-based neural network method achieves higher accuracy and faster convergence compared to traditional methods, even with small-scale networks and limited training data.

Key Observations

The authors make several key observations:

• The method exhibits excellent generalization and stability performance.
• The method can be extended to solve various problems of ODEs and PDEs by appropriate selection of Lie group expressions and loss functions.
• The NN architectures employed have fewer trainable parameters, indicating that the structure of the trial solutions significantly impacts the quality of the solution.
• The method can be applied to strong nonlinear cases and more accurately detect the severe nonlinearities of physical phenomena.

Implications and Future Directions

The paper combines Lie group theory with neural networks to solve IVPs of ODEs, showing improved efficiency and accuracy. This approach suggests potential future research directions, including:

• Extension to PDEs: Adapting the method to solve IVPs and BVPs of PDEs by combining it with semi-discrete methods or other techniques.
• Optimization of Operator Decomposition: Developing systematic approaches for selecting the operator $D_1$ in the decomposition $D = D_1 + D_2$ to maximize the efficiency and accuracy of the method.
• Theoretical Analysis: Conducting theoretical analysis to establish convergence guarantees and error bounds for the proposed method.

Conclusion

This paper presents a Lie group-based neural network method for solving IVPs of ODEs. The method combines the strengths of Lie group theory and neural networks, resulting in an efficient and accurate approach for solving differential equations. The numerical experiments demonstrate the effectiveness of the method, and the discussion provides insights into its implications and future directions.