Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Inf-Sup neural networks for high-dimensional elliptic PDE problems (2401.17233v2)

Published 30 Jan 2024 in math.NA and cs.NA

Abstract: Solving high dimensional partial differential equations (PDEs) has historically posed a considerable challenge when utilizing conventional numerical methods, such as those involving domain meshes. Recent advancements in the field have seen the emergence of neural PDE solvers, leveraging deep networks to effectively tackle high dimensional PDE problems. This study introduces Inf-SupNet, a model-based unsupervised learning approach designed to acquire solutions for a specific category of elliptic PDEs. The fundamental concept behind Inf-SupNet involves incorporating the inf-sup formulation of the underlying PDE into the loss function. The analysis reveals that the global solution error can be bounded by the sum of three distinct errors: the numerical integration error, the duality gap of the loss function (training error), and the neural network approximation error for functions within Sobolev spaces. To validate the efficacy of the proposed method, numerical experiments conducted in high dimensions demonstrate its stability and accuracy across various boundary conditions, as well as for both semi-linear and nonlinear PDEs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Andrew R Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information theory, 39(3):930–945, 1993.
  2. Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. Journal of Nonlinear Science, 29:1563–1619, 2019.
  3. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 317:28–41, 2018.
  4. Prediction of aerodynamic flow fields using convolutional neural networks. Computational Mechanics, 64:525–545, 2019.
  5. Mixed finite element methods and applications, volume 44. Springer, 2013.
  6. Russel E Caflisch. Monte-Carlo and quasi-Monte-Carlo methods. Acta Numerica, 7:1–49, 1998.
  7. Physics-informed neural networks (PINNs) for fluid mechanics: A review. Acta Mechanica Sinica, 37(12):1727–1738, 2021.
  8. Deep least-squares methods: An unsupervised learning-based numerical method for solving elliptic PDEs. Journal of Computational Physics, 420:109707, 2020.
  9. Approximations of continuous functionals by neural networks with application to dynamic systems. IEEE Transactions on Neural networks, 4(6):910–918, 1993.
  10. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022.
  11. George Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems, 2(4):303–314, 1989.
  12. On the approximation of functions by tanh neural networks. Neural Networks, 143:732–750, 2021.
  13. Weak physics informed neural networks for approximating entropy solutions of hyperbolic conservation laws. In Seminar für Angewandte Mathematik, Eidgenössische Technische Hochschule, Zürich, Switzerland, Rep, volume 35, page 2022, 2022.
  14. Neural-network-based approximations for solving partial differential equations. Communications in Numerical Methods in Engineering, 10(3):195–201, 1994.
  15. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Communications in Mathematics and Statistics, 5(4):349–380, 2017.
  16. Pierre Grisvard. Elliptic problems in nonsmooth domains. SIAM, 2011.
  17. Error bounds for approximations with deep ReLU neural networks in Ws,psuperscript𝑊𝑠𝑝W^{s,p}italic_W start_POSTSUPERSCRIPT italic_s , italic_p end_POSTSUPERSCRIPT norms. Analysis and Applications, 18(05):803–859, 2020.
  18. A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics. Computer Methods in Applied Mechanics and Engineering, 379:113741, 2021.
  19. Error analysis of deep Ritz methods for elliptic equations. Analysis and Applications, 22(1):57–87, 2024.
  20. Neufenet: Neural finite element solutions with theoretical bounds for parametric PDEs. arXiv preprint arXiv:2110.01601, 2021.
  21. Neural PDE solvers for irregular domains. arXiv preprint arXiv:2211.03241, 2022.
  22. Variational physics-informed neural networks for solving partial differential equations. arXiv preprint arXiv:1912.00873, 2019.
  23. Varnet: Variational neural networks for the solution of partial differential equations. In Learning for Dynamics and Control, pages 298–307. PMLR, 2020.
  24. Solving parametric PDE problems with artificial neural networks. European Journal of Applied Mathematics, 32(3):421–435, 2021.
  25. A theoretical analysis of deep neural networks and parametric PDEs. Constructive Approximation, 55(1):73–125, 2022.
  26. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5):987–1000, 1998.
  27. Fourier neural operator for parametric partial differential equations. International Conference on Learning Representations, 2021.
  28. On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning, pages 6083–6093. PMLR, 2020.
  29. A primal-dual approach for solving conservation laws with implicit in time approximations. Journal of Computational Physics, 472:111654, 2023.
  30. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021.
  31. nn-PINNs: Non-newtonian physics-informed neural networks for complex fluid modeling. Soft Matter, 18(1):172–185, 2022.
  32. Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 360:112789, 2020.
  33. Error analysis for a finite element approximation of elliptic dirichlet boundary control problems. SIAM Journal on Control and Optimization, 51(3):2585–2611, 2013.
  34. A priori and a posteriori error estimates for the Deep Ritz method applied to the Laplace and Stokes problem. Journal of Computational and Applied Mathematics, 421:114845, 2023.
  35. Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs. IMA Journal of Numerical Analysis, 42(2):981–1022, 2022.
  36. Estimates on the generalization error of physics-informed neural networks for approximating PDEs. IMA Journal of Numerical Analysis, 43(1):1–43, 2023.
  37. Arkadi Nemirovski. Prox-method with rate of convergence o⁢(1/t)𝑜1𝑡o(1/t)italic_o ( 1 / italic_t ) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization, 15(1):229–251, 2004.
  38. Solving a class of non-convex min-max games using iterative first order methods. Advances in Neural Information Processing Systems, 32, 2019.
  39. Houman Owhadi. Bayesian numerical homogenization. Multiscale Modeling & Simulation, 13(3):812–828, 2015.
  40. Allan Pinkus. Approximation theory of the MLP model in neural networks. Acta Numerica, 8:143–195, 1999.
  41. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  42. Inferring solutions of differential equations using noisy multi-fidelity data. Journal of Computational Physics, 335:736–746, 2017.
  43. Machine learning of linear differential equations using Gaussian processes. Journal of Computational Physics, 348:683–693, 2017.
  44. Physics-informed neural networks (PINNs) for wave propagation and full waveform inversions. Journal of Geophysical Research: Solid Earth, 127(5):e2021JB023120, 2022.
  45. Nonconvex min-max optimization: Applications, challenges, and recent theoretical advances. IEEE Signal Processing Magazine, 37(5):55–66, 2020.
  46. On quadrature rules for solving partial differential equations using neural networks. Computer Methods in Applied Mechanics and Engineering, 393:114710, 2022.
  47. Monte Carlo statistical methods, volume 2. Springer, 1999.
  48. A constrained integration (Ritz) approach to solving partial differential equations using artificial neural networks. Neurocomputing, 155:277–285, 2015.
  49. Martin Schechter. On Lpsuperscript𝐿𝑝L^{p}italic_L start_POSTSUPERSCRIPT italic_p end_POSTSUPERSCRIPT estimates and regularity II. Mathematica Scandinavica, 13.1: 47–69, 1963.
  50. Error estimates of residual minimization using neural networks for linear PDEs. Journal of Machine Learning for Modeling and Computing, 4(4), 2023.
  51. Jonathan W Siegel. Optimal approximation rates for deep ReLU neural networks on Sobolev and Besov spaces. Journal of Machine Learning Research, 24(357):1–52, 2023.
  52. DGM: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339–1364, 2018.
  53. RMSPROP: divide the gradient by a running average of its recent magnitude. Coursera: Neural networks for machine learning. COURSERA Neural Networks Mach. Learn, 17, 2012.
  54. A deep double deeponet method (D2RM) for solving partial differential equations using neural networks. Computer Methods in Applied Mechanics and Engineering, 405:115892, 2023.
  55. B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
  56. Bing Yu et al. The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018.
  57. Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems. Computer Methods in Applied Mechanics and Engineering, 393:114823, 2022.
  58. Weak adversarial networks for high-dimensional partial differential equations. Journal of Computational Physics, 411:109409, 2020.
  59. A unified framework for the error analysis of physics-informed neural networks. arXiv preprint, arXiv:2311.00529, 2023.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now