Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Approximation of Solution Operators for High-dimensional PDEs (2401.10385v1)

Published 18 Jan 2024 in math.NA, cs.LG, cs.NA, and math.OC

Abstract: We propose a finite-dimensional control-based method to approximate solution operators for evolutional partial differential equations (PDEs), particularly in high-dimensions. By employing a general reduced-order model, such as a deep neural network, we connect the evolution of the model parameters with trajectories in a corresponding function space. Using the computational technique of neural ordinary differential equation, we learn the control over the parameter space such that from any initial starting point, the controlled trajectories closely approximate the solutions to the PDE. Approximation accuracy is justified for a general class of second-order nonlinear PDEs. Numerical results are presented for several high-dimensional PDEs, including real-world applications to solving Hamilton-Jacobi-BeLLMan equations. These are demonstrated to show the accuracy and efficiency of the proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (91)
  1. W. F. Ames. Numerical methods for partial differential equations. Academic press, 2014.
  2. Neural operator: Graph kernel network for partial differential equations. In ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2019.
  3. W. Anderson and M. Farazmand. Evolution of nonlinear reduced-order solutions for pdes with conserved quantities. SIAM Journal on Scientific Computing, 44(1):A176–A197, 2022.
  4. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 59(1):345–359, 2019.
  5. K. Atkinson. An Introduction to Numerical Analysis (2nd ed.). John Wiley I& Sons, 1989.
  6. Numerical solution of inverse problems by weak adversarial networks. Inverse Problems, 36(11):115003, 2020.
  7. Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. Journal of Nonlinear Science, pages 1–57, 2017.
  8. J. Berg and K. Nyström. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 317:28–41, 2018.
  9. Data-driven discovery of green’s functions with human-understandable deep learning. Scientific Reports, 12:4824, 03 2022.
  10. Learning green’s functions associated with time-dependent partial differential equations. Journal of Machine Learning Research, 23:1–34, 08 2022.
  11. Neural galerkin schemes with active learning for high-dimensional evolution equations. Journal of Computational Physics, 496:112588, 2024.
  12. Deepm&mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks. Journal of Computational Physics, 436:110296, 2021.
  13. A phase shift deep neural network for high frequency approximation and wave problems. SIAM Journal on Scientific Computing, 42(5):A3285–A3312, 2020.
  14. W. Cai and Z.-Q. J. Xu. Multi-scale deep neural networks for solving high dimensional pdes. arXiv preprint arXiv:1910.11710, 2019.
  15. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
  16. Meta-mgnet: Meta multigrid networks for solving parameterized partial differential equations. Journal of computational physics, 455:110996, 2022.
  17. Deep operator neural networks (deeponets) for prediction of instability waves in high-speed boundary layers. Bulletin of the American Physical Society, 2020.
  18. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022.
  19. On the approximation of functions by tanh neural networks. Neural Networks, 143:732–750, 2021.
  20. M. Dissanayake and N. Phan-Thien. Neural-network-based approximations for solving partial differential equations. Communications in Numerical Methods in Engineering, 10(3):195–201, 1994.
  21. S. Dong and N. Ni. A method for representing periodic functions and enforcing exactly periodic boundary conditions with deep neural networks. Journal of Computational Physics, 435:110242, 2021.
  22. Y. Du and T. A. Zaki. Evolutional deep neural network. Phys. Rev. E, 104:045303, Oct 2021.
  23. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Communications in mathematics and statistics, 5(4):349–380, 2017.
  24. W. E and B. Yu. The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018.
  25. Numerical methods for partial differential equations. Springer Science & Business Media, 2012.
  26. L. C. Evans. Partial differential equations, volume 19 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 1998.
  27. Bcr-net: A neural network based on the nonstandard wavelet form. Journal of Computational Physics, 384:1–15, 2019.
  28. W. Fleming and R. Rishel. Deterministic and Stochastic Optimal Control. Springer, 1975.
  29. Asymptotic expansion as prior knowledge in deep learning method for high dimensional bsdes. Asia-Pacific Financial Markets, pages 1–18, 2017.
  30. Neural control of parametric solutions for high-dimensional evolution pdes. SIAM Journal on Scientific Computing (accepted), 2023.
  31. Structure probing neural network deflation. Journal of Computational Physics, 434:110231, 2021.
  32. Selectnet: Self-paced learning for high-dimensional partial differential equations. Journal of Computational Physics, 441:110444, 2021.
  33. M. Guo and J. S. Hesthaven. Data-driven reduced order modeling for time-dependent problems. Computer Methods in Applied Mechanics and Engineering, 345:75–99, 2019.
  34. Convolutional neural networks for steady flow approximation. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 481–490, New York, NY, USA, 2016. Association for Computing Machinery.
  35. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
  36. Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion monte carlo like approach. Journal of Computational Physics, 423:109792, 2020.
  37. Deep learning with domain adaptation for accelerated projection-reconstruction mr. Magnetic resonance in medicine, 80(3):1189–1205, 2018.
  38. K. Hornik. Approximation capabilities of multilayer feedforward networks. Neural networks, 4(2):251–257, 1991.
  39. Int-deep: A deep learning initialized iterative method for nonlinear problems. Journal of Computational Physics, 419:109675, 2020.
  40. Deep backward schemes for high-dimensional nonlinear pdes. Mathematics of Computation, 89(324):1547–1579, 2020.
  41. A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations. SN partial differential equations and applications, 1(2):1–34, 2020.
  42. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. Journal of Computational Physics, 404:109136, 2020.
  43. C. Johnson. Numerical solution of partial differential equations by the finite element method. Courier Corporation, 2012.
  44. hp-VPINNs: Variational physics-informed neural networks with domain decomposition. Computer Methods in Applied Mechanics and Engineering, 374:113547, 2021.
  45. D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Y. Bengio and Y. LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  46. On universal approximation and error bounds for fourier neural operators. Journal of Machine Learning Research, 22:Art–No, 2021.
  47. Neural operator: Learning maps between function spaces with applications to pdes. J. Mach. Learn. Res., 24(89):1–97, 2023.
  48. M. Kumar and N. Yadav. Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey. Computers & Mathematics with Applications, 62(10):3796–3811, 2011.
  49. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 9(5):987–1000, 1998.
  50. H. Lee and I. S. Kang. Neural algorithm for solving differential equations. Journal of Computational Physics, 91(1):110–131, 1990.
  51. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.
  52. S. Liang and R. Srikant. Why deep neural networks for function approximation? In International Conference on Learning Representations (ICLR), 2017.
  53. S. Liang and H. Yang. Finite expression method for solving high-dimensional partial differential equations. arXiv preprint arXiv:2206.10121, 2022.
  54. Bi-greennet: learning green’s functions by boundary integral network. Communications in Mathematics and Statistics, 11(1):103–129, 2023.
  55. Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains. arXiv preprint arXiv:2007.11207, 2020.
  56. Learning the solution operator of boundary value problems using graph neural networks. arXiv preprint arXiv:2206.14092, 2022.
  57. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3:218–229, 03 2021.
  58. DeepXDE: A deep learning library for solving differential equations. SIAM review, 63(1):208–228, 2021.
  59. T. Luo and H. Yang. Two-layer neural networks for partial differential equations: Optimization and generalization theory. arXiv preprint arXiv:2006.15733, 2020.
  60. Enforcing exact boundary and initial conditions in the deep mixed residual method. CSIAM Transactions on Applied Mathematics, 2(4):748–775, 2021.
  61. Neural networks trained to solve differential equations learn general representations. In Advances in Neural Information Processing Systems, pages 4071–4081, 2018.
  62. Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 360:112789, 2020.
  63. DeepM&Mnet for hypersonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators. Journal of computational physics, 447:110698, 2021.
  64. M. A. Nabian and H. Meidani. A deep learning solution approach for high-dimensional random differential equations. Probabilistic Engineering Mechanics, 57:14–25, 2019.
  65. N. Nüsken and L. Richter. Solving high-dimensional hamilton–jacobi–bellman pdes using neural networks: perspectives from the theory of controlled diffusions and measures on path space. Partial Differential Equations and Applications, 2(4):1–48, 2021.
  66. nPINNs: nonlocal physics-informed neural networks for a parametrized nonlocal universal laplacian operator. algorithms and applications. Journal of Computational Physics, 422:109760, 2020.
  67. fpinns: Fractional physics-informed neural networks. SIAM Journal on Scientific Computing, 41(4):A2603–A2626, 2019.
  68. Neural networks-based backward scheme for fully nonlinear pdes. SN Partial Differ. Equ. Appl., 2(1), 2021.
  69. A. Quarteroni and A. Valli. Numerical approximation of partial differential equations, volume 23. Springer Science & Business Media, 2008.
  70. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  71. A. A. Ramabathiran and P. Ramachandran. Spinn: Sparse, physics-based, and partially interpretable neural networks for pdes. Journal of Computational Physics, 445:110600, 2021.
  72. Convolutional neural operators. In ICLR 2023 Workshop on Physics for Machine Learning, 2023.
  73. Machine learning for fast and reliable solution of time-dependent differential equations. Journal of Computational Physics, 397:108852, 2019.
  74. M. Renardy and R. C. Rogers. An introduction to partial differential equations, volume 13. Springer Science & Business Media, 2006.
  75. On the convergence and generalization of physics informed neural networks. arXiv preprint arXiv:2004.01806, 2020.
  76. J. Sirignano and K. Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics, 375:1339–1364, 2018.
  77. W. A. Strauss. Partial differential equations: An introduction. John Wiley & Sons, 2007.
  78. Learning green’s functions of linear reaction-diffusion equations with application to fast numerical solver. In Proceedings of Mathematical and Scientific Machine Learning, volume 190 of Proceedings of Machine Learning Research, pages 1–16. PMLR, 15–17 Aug 2022.
  79. J. W. Thomas. Numerical partial differential equations: conservation laws and elliptic equations, volume 33. Springer Science & Business Media, 2013.
  80. J. W. Thomas. Numerical partial differential equations: finite difference methods, volume 22. Springer Science & Business Media, 2013.
  81. Multi-scale deep neural network (mscalednn) methods for oscillatory stokes flows in complex domains. arXiv preprint arXiv:2009.12729, 2020.
  82. Is l2superscript𝑙2l^{2}italic_l start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT physics informed loss always suitable for training physics informed neural network? Advances in Neural Information Processing Systems, 35:8278–8290, 2022.
  83. Respecting causality is all you need for training physics-informed neural networks. arXiv preprint arXiv:2203.07404, 2022.
  84. Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Science advances, 7(40):eabi8605, 2021.
  85. U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 2022.
  86. Physics-informed generative adversarial networks for stochastic differential equations. SIAM Journal on Scientific Computing, 42(1):A292–A317, 2020.
  87. Y. Yang and P. Perdikaris. Adversarial uncertainty quantification in physics-informed neural networks. Journal of Computational Physics, 394:136–152, 2019.
  88. D. Yarotsky. Error bounds for approximations with deep relu networks. Neural Networks, 94:103–114, 2017.
  89. Weak adversarial networks for high-dimensional partial differential equations. Journal of Computational Physics, page 109409, 2020.
  90. Physics-informed neural networks for nonhomogeneous material identification in elasticity imaging. arXiv preprint arXiv:2009.04525, 2020.
  91. Y. Zhu and N. Zabaras. Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification. Journal of Computational Physics, 366:415–447, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now