Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Tensor neural networks for high-dimensional Fokker-Planck equations (2404.05615v2)

Published 8 Apr 2024 in math.NA and cs.NA

Abstract: We solve high-dimensional steady-state Fokker-Planck equations on the whole space by applying tensor neural networks. The tensor networks are a linear combination of tensor products of one-dimensional feedforward networks or a linear combination of several selected radial basis functions. The use of tensor feedforward networks allows us to efficiently exploit auto-differentiation (in physical variables) in major Python packages while using radial basis functions can fully avoid auto-differentiation, which is rather expensive in high dimensions. We then use the physics-informed neural networks and stochastic gradient descent methods to learn the tensor networks. One essential step is to determine a proper bounded domain or numerical support for the Fokker-Planck equation. To better train the tensor radial basis function networks, we impose some constraints on parameters, which lead to relatively high accuracy. We demonstrate numerically that the tensor neural networks in physics-informed machine learning are efficient for steady-state Fokker-Planck equations from two to ten dimensions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Easymkl: a scalable multiple kernel learning algorithm. Neurocomputing, 169:215–224. Learning for Visual Semantic Understanding in Big Data ESANN 2014 Industrial Data Processing and Analysis.
  2. Learning “best” kernels from data in Gaussian process regression. with application to aerodynamics. Journal of Computational Physics, 470:111595.
  3. Physics-informed solution of the stationary fokker-plank equation for a class of nonlinear dynamical systems: An evaluation study. arXiv preprint arXiv:2309.16725.
  4. Fisher information and shape-morphing modes for solving the Fokker-Planck equation in higher dimensions. arXiv: 2306.03749.
  5. A new support vector machine with an optimal additive kernel. Neurocomputing, 329:279–299.
  6. Deep splitting method for parabolic PDEs. SIAM Journal on Scientific Computing, 43(5):A3135–A3154.
  7. Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. Journal of Nonlinear Science, 29:1563–1619.
  8. Overcoming the curse of dimensionality in the numerical approximation of Allen–Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations. Journal of Numerical Mathematics, 28(4):197–222.
  9. Solving high-dimensional optimal stopping problems using deep learning. European Journal of Applied Mathematics, 32(3):470–514.
  10. Probability flow solution of the Fokker-Planck equation. Machine Learning: Science and Technology, 4(3):035012.
  11. Machine learning for semi linear pdes. Journal of scientific computing, 79(3):1667–1712.
  12. Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation. Math. Comp., 90(332):2527–2578.
  13. Separable pinn: Mitigating the curse of dimensionality in physics-informed neural networks. arXiv preprint arXiv:2211.08761.
  14. The numerical solution of Fokker-Planck equation with radial basis functions (RBFs) based on the meshless technique of kansa’s approach and Galerkin method. Engineering Analysis with Boundary Elements, 47:38–63.
  15. An efficient data-driven solver for Fokker-Planck equations: algorithm and analysis. Commun. Math. Sci., 20(3):803–827.
  16. Algorithms for solving high dimensional pdes: from nonlinear monte carlo to machine learning. Nonlinearity, 35(1):278.
  17. Genz, A. (1984). Testing multidimensional integration routines. In Proceedings of the International Conference on Tools, Methods and Languages for Scientific and Engineering Computation, pages 81–94. Elsevier North-Holland, Amsterdam.
  18. Stationary density estimation of Itô diffusions using deep learning. SIAM J. Numer. Anal., 61(1):45–82.
  19. Learning dynamical systems from data: a simple cross-validation perspective, part I: Parametric kernel flows. Phys. D, 421:Paper No. 132817, 10.
  20. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510.
  21. Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Communications in mathematics and statistics, 5(4):349–380.
  22. Learning physics-informed neural networks without stacked back-propagation. In International Conference on Artificial Intelligence and Statistics, pages 3034–3047. PMLR.
  23. Henry-Labordere, P. (2017). Deep primal-dual algorithm for bsdes: Applications of machine learning to cva and im. Available at SSRN 3071506.
  24. Tackling the curse of dimensionality with physics-informed neural networks. arXiv preprint arXiv:2307.12306.
  25. Steady states of Fokker-Planck equations: I. Existence. J. Dynam. Differential Equations, 27(3-4):721–742.
  26. Deep backward schemes for high-dimensional nonlinear PDEs. Mathematics of Computation, 89(324):1547–1579.
  27. Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations. Proceedings of the Royal Society A, 476(2244):20190630.
  28. SPF-GMKL: generalized multiple kernel learning with a million kernels. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 750–758.
  29. Three algorithms for solving high-dimensional fully coupled FBSDEs through deep learning. IEEE Intelligent Systems, 35(3):71–84.
  30. The variational formulation of the Fokker-Planck equation. SIAM J. Math. Anal., 29(1):1–17.
  31. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440.
  32. Radial basis functions methods for solving Fokker-Planck equation. Engineering Analysis with Boundary Elements, 36(2):181–189.
  33. Adam: A method for stochastic optimization. ICLR.
  34. Lp-norm multiple kernel learning. The Journal of Machine Learning Research, 12:953–997.
  35. Neural-network methods for boundary value problems with irregular boundaries. IEEE Transactions on Neural Networks, 11(5):1041–1049.
  36. Artificial neural network solver for time-dependent Fokker-Planck equations. Appl. Math. Comput., 457:Paper No. 128185, 24.
  37. Computing the invariant distribution of randomly perturbed dynamical systems using deep learning. J. Sci. Comput., 91(3):Paper No. 77, 17.
  38. Computing high-dimensional invariant distributions from noisy data. J. Comput. Phys., 474:Paper No. 111783, 16.
  39. Neural parametric fokker–planck equation. SIAM Journal on Numerical Analysis, 60(3):1385–1449.
  40. Learning zeros of Fokker-Planck operators. arXiv: 2306.07068.
  41. A stabilized radial basis-finite difference (RBF-FD) method with hybrid kernels. Comput. Math. Appl., 77(9):2354–2368.
  42. Hybrid Gaussian-cubic radial basis functions for scattered data interpolation. Comput. Geosci., 22(5):1203–1218.
  43. A geometric algorithm for scalable multiple kernel learning. In Artificial Intelligence and Statistics, pages 633–642. PMLR.
  44. Fourierformer: Transformer meets generalized Fourier integral theorem. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K., editors, Advances in Neural Information Processing Systems.
  45. Ultra-fast optimization algorithm for sparse multi kernel learning. In Proceedings of the 28th International Conference on Machine Learning, number CONF.
  46. Kernel flows: from learning kernels from data into the abyss. J. Comput. Phys., 389:22–47.
  47. Kernel flows: From learning kernels from data into the abyss. Journal of Computational Physics, 389:22–47.
  48. Raissi, M. (2018). Forward-backward stochastic neural networks: Deep learning of high-dimensional partial differential equations. arXiv preprint arXiv:1804.07010.
  49. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707.
  50. Simplemkl. Journal of Machine Learning Research, 9(83):2491–2521.
  51. Risken, H. The Fokker-Planck equation, volume 18 of Springer Series in Synergetics. Springer-Verlag, Berlin, 2nd edition. Methods of solution and applications.
  52. Solving PDEs with a hybrid radial basis function: power-generalized multiquadric kernel. Adv. Appl. Math. Mech., 14(5):1161–1180.
  53. Large scale multiple kernel learning. The Journal of Machine Learning Research, 7:1531–1565.
  54. Numerical solution of the Fokker-Planck equation using physics-based mixture models. Computer Methods in Applied Mechanics and Engineering, 399:115424.
  55. Adaptive deep density approximation for Fokker-Planck equations. J. Comput. Phys., 457:Paper No. 111080, 19.
  56. More generality in efficient multiple kernel learning. In Proceedings of the 26th annual international conference on machine learning, pages 1065–1072.
  57. Radial basis function neural networks solution for stationary probability density function of nonlinear stochastic systems. Probabilistic Engineering Mechanics, 71:103408.
  58. Tensor neural network and its numerical integration. arXiv preprint arXiv:2207.02754.
  59. Solving schr\\\backslash\”{{\{{o}}\}} dinger equation using tensor neural network. arXiv preprint arXiv:2209.12572.
  60. Learning dynamical systems from data: A simple cross-validation perspective, part v: Sparse kernel flows for 132 chaotic dynamical systems. Physica D: Nonlinear Phenomena, page 134070.
  61. Deep regularization and direct training of the inner layers of neural networks with kernel flows. Phys. D, 426:Paper No. 132952, 7.
  62. A deep learning method for solving Fokker-Planck equations. In Bruna, J., Hesthaven, J., and Zdeborova, L., editors, Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, volume 145 of Proceedings of Machine Learning Research, pages 568–597. PMLR.
  63. Deep learning framework for solving Fokker-Planck equations with low-rank separation representation. Engineering Applications of Artificial Intelligence, 121:106036.
  64. A predictor-corrector deep learning algorithm for high dimensional stochastic partial differential equations. arXiv:2208.09883.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets