Physics-Informed Neural Network Method for Parabolic Differential Equations with Sharply Perturbed Initial Conditions (2208.08635v1)
Abstract: In this paper, we develop a physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. As an example of a parabolic problem, we consider the advection-dispersion equation (ADE) with a point (Gaussian) source initial condition. In the $d$-dimensional ADE, perturbations in the initial condition decay with time $t$ as $t{-d/2}$, which can cause a large approximation error in the PINN solution. Localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. Finally, the PINN solution of parabolic equations is sensitive to the choice of weights in the loss function. We propose a normalized form of ADE where the initial perturbation of the solution does not decrease in amplitude and demonstrate that this normalization significantly reduces the PINN approximation error. We propose criteria for weights in the loss function that produce a more accurate PINN solution than those obtained with the weights selected via other methods. Finally, we proposed an adaptive sampling scheme that significantly reduces the PINN solution error for the same number of the sampling (residual) points. We demonstrate the accuracy of the proposed PINN model for forward, inverse, and backward ADEs.
- Andrew R Barron. Approximation and estimation bounds for artificial neural networks. Machine learning, 14(1):115–133, 1994.
- Automatic differentiation in machine learning: a survey. 2015.
- Physics-informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica, pages 1–12, 2022.
- Scientific machine learning through physics-informed neural networks: Where we are and what’s next, 2022.
- Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
- Deep learning. MIT press, 2016.
- Arlen W Harbaugh. MODFLOW-2005, the US Geological Survey modular ground-water model: the ground-water flow process. US Department of the Interior, US Geological Survey Reston, VA, 2005.
- Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport. Advances in Water Resources, 141:103610, 2020.
- Physics-informed neural network method for forward and backward advection-dispersion equations. Water Resources Research, 57(7):e2020WR029479, 2021.
- Solving the nonlinear schrödinger equation in optical fibers using physics-informed neural network. In Optical Fiber Communication Conference, pages M3H–8. Optica Publishing Group, 2021.
- Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021.
- Daehyon Kim. Normalization methods for input and output vectors in backpropagation neural networks. International journal of computer mathematics, 71(2):161–171, 1999.
- Adam: A method for stochastic optimization, 2017.
- Weight initialization of deep neural networks (dnns) using data statistics. arXiv preprint arXiv:1710.10570, 2017.
- Efficient backprop. In Neural networks: Tricks of the trade, pages 9–48. Springer, 2012.
- On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1):503–528, 1989.
- Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 360:112789, 2020.
- H Nguyen and J Reynen. A space-time least-square finite element scheme for advection-diffusion equations. Computer Methods in Applied Mechanics and Engineering, 42(3):331–342, 1984.
- Gadi Pinkas. Reasoning, nonmonotonicity and learning in connectionist networks that capture propositional knowledge. Artificial Intelligence, 77(2):203–247, 1995.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
- On the convergence and generalization of physics informed neural networks. arXiv e-prints, pages arXiv–2004, 2020.
- Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems. Water Resources Research, 56(5):e2019WR026731, 2020.
- Physics-informed machine learning with conditional karhunen-loève expansions. Journal of Computational Physics, 426:109904, 2021.
- Qingcan Wang et al. Exponential convergence of the deep neural network approximation for analytic functions. arXiv preprint arXiv:1807.00297, 2018.
- Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021.
- Physics-informed machine learning method for large-scale data assimilation problems. Water Resources Research, 58(5):e2021WR031023, 2022.
- Mt3dms: a modular three-dimensional multispecies transport model for simulation of advection, dispersion, and chemical reactions of contaminants in groundwater systems; documentation and user’s guide. 1999.