Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Neural McKean-Vlasov Processes: Distributional Dependence in Diffusion Processes (2404.09402v1)

Published 15 Apr 2024 in cs.LG, cs.AI, and stat.ML

Abstract: McKean-Vlasov stochastic differential equations (MV-SDEs) provide a mathematical description of the behavior of an infinite number of interacting particles by imposing a dependence on the particle density. As such, we study the influence of explicitly including distributional information in the parameterization of the SDE. We propose a series of semi-parametric methods for representing MV-SDEs, and corresponding estimators for inferring parameters from data based on the properties of the MV-SDE. We analyze the characteristics of the different architectures and estimators, and consider their applicability in relevant machine learning problems. We empirically compare the performance of the different architectures and estimators on real and synthetic datasets for time series and probabilistic modeling. The results suggest that explicitly including distributional dependence in the parameterization of the SDE is effective in modeling temporal data with interaction under an exchangeability assumption while maintaining strong performance for standard It^o-SDEs due to the richer class of probability flows associated with MV-SDEs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Dataset dynamics via gradient flows in probability space. In International Conference on Machine Learning, pages 219–230. PMLR, 2021.
  2. Refining deep generative models via discriminator gradient flow. arXiv preprint arXiv:2012.00780, 2020.
  3. Mohamed Ali Belabbas. On implicit regularization: Morse functions and applications to matrix factorization. arXiv preprint arXiv:2001.04264, 2020.
  4. Iterative multilevel density estimation for mckean-vlasov sdes via projections. arXiv preprint arXiv:1909.11717, 2019.
  5. A stochastic particle method for the mckean-vlasov and the burgers equation. Mathematics of Computation, 66(217):157–192, 1997. ISSN 00255718, 10886842. URL http://www.jstor.org/stable/2153648.
  6. Mean-field stochastic differential equations and associated pdes. The Annals of Probability, 45(2):824–878, 2017. ISSN 00911798, 2168894X. URL http://www.jstor.org/stable/44245559.
  7. Robust and scalable sde learning: A functional perspective. arXiv preprint arXiv:2110.05167, 2021.
  8. Probabilistic theory of mean field games with applications I-II. Springer, 2018.
  9. Long-time behaviour and phase transitions for the mckean–vlasov equation on the torus. Archive for Rational Mechanics and Analysis, 235(1):635–690, 2020.
  10. Classical solutions for a nonlinear fokker-planck equation arising in computational neuroscience. Communications in Partial Differential Equations, 38(3):385–409, 2013.
  11. Solving inverse stochastic problems from discrete particle observations using the fokker–planck equation and physics-informed neural networks. SIAM Journal on Scientific Computing, 43(3):B811–B830, 2021.
  12. Trajectory inference via mean-field langevin in path space. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=Mftcm8i4sL.
  13. Rigorous derivation of the nonlocal reaction-diffusion fitzhugh–nagumo system. SIAM Journal on Mathematical Analysis, 51(1):346–373, 2019.
  14. Solving fredholm integral equations of the first kind via wasserstein gradient flows. arXiv preprint arXiv:2209.09936, 2022.
  15. Emergent behavior in flocks. IEEE Transactions on automatic control, 52(5):852–862, 2007.
  16. Diffusion schrödinger bridge with applications to score-based generative modeling. Advances in Neural Information Processing Systems, 34:17695–17709, 2021.
  17. Nonparametric estimation for interacting particle systems: Mckean–vlasov models. Probability Theory and Related Fields, 182(1):551–613, 2022.
  18. Laetitia Della Maestra and Marc Hoffmann. The lan property for mckean-vlasov models in a mean-field regime. Stochastic Processes and their Applications, 155:109–146, 2023. ISSN 0304-4149. doi: https://doi.org/10.1016/j.spa.2022.10.002. URL https://www.sciencedirect.com/science/article/pii/S0304414922002113.
  19. Empirical approximation to invariant measures for mckean–vlasov processes: mean-field interaction vs self-interaction. arXiv preprint arXiv:2112.14112, 2021.
  20. Dynamic default contagion in heterogeneous interbank systems. SIAM Journal on Financial Mathematics, 12(4):SC83–SC97, 2021.
  21. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378, 2007. doi: 10.1198/016214506000001437. URL https://doi.org/10.1198/016214506000001437.
  22. Parameter estimation for macroscopic pedestrian dynamics models from microscopic data. SIAM Journal on Applied Mathematics, 79(4):1475–1500, 2019.
  23. Scalable reversible generative models with free-form continuous dynamics. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=rJxgknCcK7.
  24. A multiscale 3d chemotaxis assay reveals bacterial navigation mechanisms. Communications biology, 4(1):1–8, 2021.
  25. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
  26. A mckean–vlasov equation with positive feedback and blow-ups. The Annals of Applied Probability, 29(4):2338–2373, 2019.
  27. Learning high-dimensional mckean-vlasov forward-backward stochastic differential equations with general distribution dependence. arXiv preprint arXiv:2204.11924, 2022.
  28. Identifying latent stochastic differential equations. IEEE Transactions on Signal Processing, 70:89–104, 2021.
  29. Inference and sampling of point processes from diffusion excursions. In Uncertainty in Artificial Intelligence, pages 839–848. PMLR, 2023.
  30. Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural networks, 4(2):251–257, 1991.
  31. Mean-field langevin dynamics and energy landscape of neural networks. In Annales de l’Institut Henri Poincaré, Probabilités et Statistiques, volume 57, pages 2043–2065. Institut Henri Poincaré, 2021.
  32. A variational perspective on diffusion-based generative models and score matching. Advances in Neural Information Processing Systems, 34:22863–22876, 2021.
  33. Capital distribution and portfolio performance in the mean-field atlas model. Annals of Finance, 11(2):151–198, 2015.
  34. Model for chemotaxis. Journal of Theoretical Biology, 30(2):225–234, 1971.
  35. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems, 2018. URL https://doi.org/10.48550/arXiv.1807.03039.
  36. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  37. Human trajectory forecasting in crowds: A deep learning perspective. IEEE Transactions on Intelligent Transportation Systems, 23(7):7386–7400, 2021.
  38. Daniel Lacker. Mean field games and interacting particle systems. 2018. URL http://www.columbia.edu/~dl3133/MFGSpring2018.pdf.
  39. A case study on stochastic games on large graphs in mean field and sparse regimes. Mathematics of Operations Research, 47(2):1530–1565, 2022.
  40. Towards a mathematical theory of trajectory inference. arXiv preprint arXiv:2102.09204, 2021.
  41. Thomas M. Liggett. Stochastic models of interacting systems. The Annals of Probability, 25(1):1 – 29, 1997. doi: 10.1214/aop/1024404276. URL https://doi.org/10.1214/aop/1024404276.
  42. Sylvie Méléard. Asymptotic behaviour of some interacting particle systems; mckean-vlasov and boltzmann models. Probabilistic models for nonlinear partial differential equations, pages 42–95, 1996.
  43. Learning mean-field equations from particle data using wsindy. Physica D: Nonlinear Phenomena, 439:133406, 2022. ISSN 0167-2789. doi: https://doi.org/10.1016/j.physd.2022.133406. URL https://www.sciencedirect.com/science/article/pii/S0167278922001543.
  44. On a kinetic fitzhugh–nagumo model of neuronal network. Communications in mathematical physics, 342(3):1001–1042, 2016.
  45. Particle systems with singular interaction through hitting times: application in systemic risk modeling. The Annals of Applied Probability, 2019.
  46. Masked autoregressive flow for density estimation. Advances in neural information processing systems, 30, 2017.
  47. A method of moments estimator for interacting particle systems and their mean field limit. arXiv preprint arXiv:2212.00403, 2022.
  48. Mean-field neural networks: learning mappings on wasserstein space. arXiv preprint arXiv:2210.15179, 2022.
  49. A machine learning framework for solving high-dimensional mean field game and mean field control problems. Proceedings of the National Academy of Sciences, 117(17):9183–9193, 2020.
  50. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191, 2020.
  51. Sinkformers: Transformers with doubly stochastic attention. In International Conference on Artificial Intelligence and Statistics, pages 3515–3530. PMLR, 2022.
  52. Filippo Santambrogio. {{\{{Euclidean, metric, and Wasserstein}}\}} gradient flows: an overview. Bulletin of Mathematical Sciences, 7(1):87–154, 2017.
  53. Parameter estimation for the mckean-vlasov stochastic differential equation. arXiv preprint arXiv:2106.13751, 2021.
  54. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
  55. Approximate solution to the stochastic kuramoto model. Physical Review E, 88(5), nov 2013.
  56. Iterative multilevel particle approximation for mckean–vlasov sdes. The Annals of Applied Probability, 29(4):2230–2265, 2019.
  57. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804–24816, 2021.
  58. Milica Tomašević. A new mckean–vlasov stochastic interpretation of the parabolic-parabolic keller–segel model: The two-dimensional case. The Annals of Applied Probability, 31(1):432–459, 2021.
  59. Cédric Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021.
  60. William H Warren. Collective motion in human crowds. Current directions in psychological science, 27(4):232–240, 2018.
  61. Mean-field nonparametric estimation of interacting particle systems. In Po-Ling Loh and Maxim Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pages 2242–2275. PMLR, 02–05 Jul 2022. URL https://proceedings.mlr.press/v178/yao22a.html.
  62. Event related potentials during object recognition tasks. Brain Research Bulletin, 38(6):531–538, 1995.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets