Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation (2302.05104v4)

Published 10 Feb 2023 in cs.LG, cs.NA, math.NA, and math.PR

Abstract: In scenarios with limited available data, training the function-to-function neural PDE solver in an unsupervised manner is essential. However, the efficiency and accuracy of existing methods are constrained by the properties of numerical algorithms, such as finite difference and pseudo-spectral methods, integrated during the training stage. These methods necessitate careful spatiotemporal discretization to achieve reasonable accuracy, leading to significant computational challenges and inaccurate simulations, particularly in cases with substantial spatiotemporal variations. To address these limitations, we propose the Monte Carlo Neural PDE Solver (MCNP Solver) for training unsupervised neural solvers via the PDEs' probabilistic representation, which regards macroscopic phenomena as ensembles of random particles. Compared to other unsupervised methods, MCNP Solver naturally inherits the advantages of the Monte Carlo method, which is robust against spatiotemporal variations and can tolerate coarse step size. In simulating the trajectories of particles, we employ Heun's method for the convection process and calculate the expectation via the probability density function of neighbouring grid points during the diffusion process. These techniques enhance accuracy and circumvent the computational issues associated with Monte Carlo sampling. Our numerical experiments on convection-diffusion, Allen-Cahn, and Navier-Stokes equations demonstrate significant improvements in accuracy and efficiency compared to other unsupervised baselines. The source code will be publicly available at: https://github.com/optray/MCNP.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (100)
  1. Brownian-time processes: the pde connection and the half-derivative generator. Annals of Probability, pages 1780–1795, 2001.
  2. Computational fluid dynamics, volume 206. Springer, 1995.
  3. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials & Continua, 59(1):345–359, 2019.
  4. W. F. Bauer. The monte carlo method. Journal of the Society for Industrial and Applied Mathematics, 6(4):438–451, 1958.
  5. A data-driven physics-informed finite-volume scheme for nonclassical undercompressive shocks. Journal of Computational Physics, 437:110324, 2021.
  6. JAX: composable transformations of Python+NumPy programs, 2018.
  7. Message passing neural PDE solvers. In International Conference on Learning Representations, 2022.
  8. Quadratic residual networks: A new class of neural networks for solving forward and inverse problems in physics involving pdes. In Proceedings of the SIAM International Conference on Data Mining, pages 675–683. SIAM, 2021.
  9. Shuhao Cao. Choose a transformer: Fourier or galerkin. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
  10. Physics-informed learning of governing equations from scarce data. Nature Communications, 12(1):1–13, 2021.
  11. Stochastic tools in mathematics and science, volume 1. Springer, 2009.
  12. Vortex methods: theory and practice, volume 8. Cambridge University Press, 2000.
  13. On the partial difference equations of mathematical physics. IBM Journal of Research and Development, 11(2):215–234, 1967.
  14. A feynman-kac-type formula for the deterministic and stochastic wave equations and other pde’s. Transactions of the American Mathematical Society, 360(9):4681–4703, 2008.
  15. A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized pdes. Journal of Scientific Computing, 87:1–36, 2021.
  16. A comparison of a family of eulerian and semi-lagrangian finite element methods for the advection-diffusion equation. WIT Transactions on The Built Environment, 30, 1997.
  17. Self-scalable tanh (stan): Multi-scale solutions for physics-informed neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  18. Can physics-informed neural networks beat the finite element method? arXiv preprint arXiv:2302.04107, 2023.
  19. Monte carlo fpinns: Deep learning method for forward and inverse problems involving high dimensional fractional partial differential equations. Computer Methods in Applied Mechanics and Engineering, 400:115523, 2022.
  20. Multiwavelet-based operator learning for differential equations. In Advances in Neural Information Processing Systems, volume 34, pages 24048–24062, 2021.
  21. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
  22. A derivative-free method for solving elliptic partial differential equations with deep neural networks. Journal of Computational Physics, 419:109672, 2020.
  23. A numerical algorithm for a class of bsdes via the branching process. Stochastic Processes and their Applications, 124(2):1112–1140, 2014.
  24. Deep-neural-network solution of the electronic schrödinger equation. Nature Chemistry, 12(10):891–897, 2020.
  25. Roger Howe. Quantum mechanics and partial differential equations. Journal of Functional Analysis, 38(2):188–254, 1980.
  26. Adaptive moving mesh methods, volume 174. Springer Science & Business Media, 2010.
  27. Meta-auto-decoder for solving parametric partial differential equations. In Advances in Neural Information Processing Systems, 2022.
  28. Numerical simulation of wave breaking. Journal of Physical Oceanography, 41(2):346–364, 2011.
  29. Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations. Journal of Computational Physics, 426:109951, 2021.
  30. Composing partial differential equations with physics-aware neural networks. In International Conference on Machine Learning, pages 10773–10801. PMLR, 2022.
  31. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021.
  32. P.E. Kloeden and E. Platen. Numerical Solution of Stochastic Differential Equations. Stochastic Modelling and Applied Probability. Springer Berlin Heidelberg, 2011.
  33. Fractional laplace motion. Advances in Applied Probability, 38(2):451–464, 2006.
  34. Peter R. Kramer. A review of some monte carlo simulation methods for turbulent systems. Monte Carlo Methods and Applications, 7(3-4):229–244, 2001.
  35. Characterizing possible failure modes in physics-informed neural networks. In Advances in Neural Information Processing Systems, volume 34, pages 26548–26560, 2021.
  36. Partial differential equations with numerical methods, volume 45. Springer, 2003.
  37. HyperdeepONet: learning operator with complex target function space using the limited resources via hypernetwork. In International Conference on Learning Representations, 2023.
  38. Neural-network-based multistate solver for a static schrödinger equation. Physical Review A, 103(3):032405, 2021.
  39. Transformer for partial differential equations’ operator learning. Transactions on Machine Learning Research, 2023.
  40. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1–26, 2023.
  41. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020.
  42. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021.
  43. Physics-informed neural operator for learning partial differential equations. ACM/IMS Journal of Data Science, feb 2024.
  44. What is the fractional laplacian? a comparative review with new results. Journal of Computational Physics, 404:109009, 2020.
  45. Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains. Communications in Computational Physics, 28(5):1970–2001, 2020.
  46. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021.
  47. Sobolev acceleration and statistical optimality for learning elliptic equations via gradient descent. In Advances in Neural Information Processing Systems, volume 35, pages 33233–33247, 2022.
  48. Monte carlo approximations of the neumann problem. Monte Carlo Methods and Applications, 19(3):201–236, 2013.
  49. Xuerong Mao. Stochastic differential equations and applications. Elsevier, 2007.
  50. A neural network multigrid solver for the navier-stokes equations. Journal of Computational Physics, 460:110983, 2022.
  51. A novel sequential method to train physics informed neural networks for allen cahn and cahn hilliard equations. Computer Methods in Applied Mechanics and Engineering, 390:114474, 2022.
  52. A review of vortex methods and their applications: From creation to recent advances. Fluids, 6(2):68, 2021.
  53. Hybrid fem-nn models: Combining artificial neural networks with the finite element method. Journal of Computational Physics, 446:110651, 2021.
  54. N Navaneeth and Souvik Chakraborty. Stochastic projection based approach for gradient free physics informed learning. Computer Methods in Applied Mechanics and Engineering, 406:115842, 2023.
  55. Physics informed wno. Computer Methods in Applied Mechanics and Engineering, 418:116546, 2024.
  56. Parametric deep energy approach for elasticity accounting for strain gradient effects. Computer Methods in Applied Mechanics and Engineering, 386:114096, 2021.
  57. Interpolating between bsdes and pinns–deep learning for elliptic and parabolic boundary value problems. Journal of Machine Learning, 2(1):31–64, 2023.
  58. Higher-order laplace equations and hyper-cauchy distributions. Journal of Theoretical Probability, 28:92–118, 2015.
  59. Integrated finite element neural network (i-fenn) for non-local continuum damage mechanics. Computer Methods in Applied Mechanics and Engineering, 404:115766, 2023.
  60. Backward stochastic differential equations and quasilinear parabolic partial differential equations. In Stochastic Partial Differential Equations and Their Applications, pages 200–217. Springer, 1992.
  61. Forward-backward stochastic differential equations and quasilinear parabolic pdes. Probability Theory and Related Fields, 114(2):123–150, 1999.
  62. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32, 2019.
  63. Huyen Pham. Feynman-kac representation of fully nonlinear pdes and applications. Acta Mathematica Vietnamica, 40:255–269, 2015.
  64. Random vortex dynamics via functional stochastic differential equations. Proceedings of the Royal Society A, 478(2266):20220030, 2022.
  65. U-NO: U-shaped neural operators. Transactions on Machine Learning Research, 2023.
  66. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  67. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, 367(6481):1026–1030, 2020.
  68. Robust sde-based variational formulations for solving linear pdes via deep learning. In International Conference on Machine Learning, pages 18649–18666. PMLR, 2022.
  69. Solving high-dimensional parabolic pdes using the tensor train format. In International Conference on Machine Learning, pages 8998–9009. PMLR, 2021.
  70. Handbook of heat transfer, volume 3. Mcgraw-hill New York, 1998.
  71. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241. Springer, 2015.
  72. Numerical simulation of 2d-vorticity dynamics using particle methods. Computers & Mathematics with Applications, 69(12):1484–1503, 2015.
  73. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 362:112790, 2020.
  74. Learning to simulate complex physics with graph networks. In International Conference on Machine Learning, pages 8459–8468. PMLR, 2020.
  75. Grid-free monte carlo for pdes with spatially varying coefficients. ACM Transactions on Graphics (TOG), 41(4):1–17, 2022.
  76. NOMAD: Nonlinear manifold decoders for operator learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
  77. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624, 2021.
  78. Lordnet: Learning to solve parametric partial differential equations without simulated data. arXiv preprint arXiv:2206.09418, 2022.
  79. On the control of the 2d navier–stokes equations with kolmogorov forcing. Complexity, 2021:1–18, 2021.
  80. Bayesian spline learning for equation discovery of nonlinear dynamics with quantified uncertainty. In Advances in Neural Information Processing Systems, volume 35, pages 6927–6940, 2022.
  81. Deep learning methods for partial differential equations and related parameter identification problems. Inverse Problems, 39(10):103001, 2023.
  82. Factorized fourier neural operators. In International Conference on Learning Representations, 2023.
  83. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations, 2019.
  84. Svd perspectives for augmenting deeponet flexibility and interpretability. Computer Methods in Applied Mechanics and Engineering, 403:115718, 2023.
  85. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17(3):261–272, 2020.
  86. Learning incompressible fluid dynamics from scratch - towards fast, differentiable fluid models that generalize. In International Conference on Learning Representations, 2021.
  87. Teaching the incompressible navier–stokes equations to fast neural surrogate models in three dimensions. Physics of Fluids, 33(4), 2021.
  88. Spline-pinn: Approaching pdes without data using fast, physics-informed hermite-spline cnns. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8529–8538, 2022.
  89. Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Science Advances, 7(40):eabi8605, 2021.
  90. Improved architectures and training algorithms for deep operator networks. Journal of Scientific Computing, 92(2):35, 2022.
  91. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:110768, 2022.
  92. Cenn: Conservative energy method based on neural networks with subdomains for solving variational problems involving heterogeneous and complex geometries. Computer Methods in Applied Mechanics and Engineering, 400:115491, 2022.
  93. E Weinan. Principles of multiscale modeling. Cambridge University Press, 2011.
  94. Self-adaptive loss balanced physics-informed neural networks. Neurocomputing, 496:11–34, 2022.
  95. Li-Ming Yang. Kinetic theory of diffusion in gases and liquids. i. diffusion and the brownian motion. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, pages 94–116, 1949.
  96. Drvn (deep random vortex network): A new physics-informed machine learning method for simulating and inferring incompressible fluid flows. Physics of Fluids, 34(10):107112, 2022.
  97. Xicheng Zhang. Stochastic functional differential equations driven by lévy processes and quasi-linear partial integro-differential equations. The Annals of Applied Probability, 22(6):2505–2538, 2012.
  98. Xicheng Zhang. Stochastic lagrangian particle approach to fractal navier-stokes equations. Communications in Mathematical Physics, 311(1):133–155, 2012.
  99. Learning to solve pde-constrained inverse problems with graph networks. In International Conference on Machine Learning, 2022.
  100. David Zwicker. py-pde: A python package for solving partial differential equations. Journal of Open Source Software, 5(48):2158, 2020.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets