2000 character limit reached
Long run convergence of discrete-time interacting particle systems of the McKean-Vlasov type (2403.17472v4)
Published 26 Mar 2024 in math.PR
Abstract: We consider a discrete-time system of n coupled random vectors, a.k.a. interacting particles. The dynamics involve a vanishing step size, some random centered perturbations, and a mean vector field which induces the coupling between the particles. We study the doubly asymptotic regime where both the number of iterations and the number n of particles tend to infinity, without any constraint on the relative rates of convergence of these two parameters. We establish that the empirical measure of the interpolated trajectories of the particles converges in probability, in an ergodic sense, to the set of recurrent Mc-Kean-Vlasov distributions. A first application example is the granular media equation, where the particles are shown to converge to a critical point of the Helmholtz energy. A second example is the convergence of stochastic gradient descent to the global minimizer of the risk, in a wide two-layer neural networks using random features.
- Gradient flows in metric spaces and in the space of probability measures. Lectures in Mathematics ETH Zürich. Birkhäuser Verlag, Basel, second edition, 2008.
- Integral inequalities for convex functions of operators on martingales. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, volume 2, pages 223–240. Univ. California Press Berkeley, Calif., 1972.
- M. Benaïm. Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités, XXXIII, volume 1709 of Lecture Notes in Math., pages 1–68. Springer, Berlin, 1999.
- Trend to equilibrium and particle approximation for a weakly selfconsistent vlasov-fokker-planck equation. ESAIM: Mathematical Modelling and Numerical Analysis, 44(5):867–884, 2010.
- P. Billingsley. Convergence of probability measures. Wiley Series in Probability and Statistics: Probability and Statistics. John Wiley & Sons, Inc., New York, second edition, 1999. A Wiley-Interscience Publication.
- N. Bourbaki. General topology. Chapters 5–10. Elements of Mathematics (Berlin). Springer-Verlag, Berlin, 1989. Translated from the French, Reprint of the 1966 edition.
- Nonlinear self-stabilizing processes–i existence, invariant probability, propagation of chaos. Stochastic processes and their applications, 75(2):173–201, 1998.
- Ergodic properties of weak asymptotic pseudotrajectories for semiflows. Journal of Dynamics and Differential Equations, 12:579–598, 2000.
- On the global convergence of gradient descent for over-parameterized models using optimal transport. Advances in neural information processing systems, 31, 2018.
- Propagation of chaos: A review of models, methods and applications. i. models and methods. Kinetic and Related Models, 15(6):895, 2022.
- Probabilistic approach for granular media equations in the non-uniformly convex case. Probability theory and related fields, 140:19–40, 2008.
- Learning with sgd and random features. Advances in neural information processing systems, 31, 2018.
- An elementary approach to uniform in time propagation of chaos. Proceedings of the American Mathematical Society, 148(12):5387–5398, 2020.
- Uniform propagation of chaos and creation of chaos for a class of nonlinear diffusions. Stochastic Analysis and Applications, 37(6):909–935, 2019.
- Conditional propagation of chaos for mean field systems of interacting neurons. Electronic Journal of Probability, 26(none):1 – 25, 2021.
- Giovanni Leoni. A first course in fractional Sobolev spaces, volume 229. American Mathematical Society, 2023.
- Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in Neural Information Processing Systems, 2016.
- Florient Malrieu. Logarithmic sobolev inequalities for some nonlinear pde’s. Stochastic processes and their applications, 95(1):109–132, 2001.
- Florent Malrieu. Convergence to equilibrium for granular media equations and their euler schemes. The Annals of Applied Probability, 13(2):540–560, 2003.
- R. Mañé. Ergodic Theory and Differentiable Dynamics. Ergebnisse der Mathematik und ihrer Grenzgebiete : a series of modern surveys in mathematics. Folge 3. Springer-Verlag, 1987.
- A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences, 115(33):E7665–E7671, 2018.
- Density and gradient estimates for non degenerate brownian sdes with unbounded measurable drift. Journal of Differential Equations, 272:330–369, 2021.
- S. Méléard and S. Roelly-Coppoletta. A propagation of chaos result for a system of particles with moderate interaction. Stochastic Processes and their Applications, 26:317–332, 1987.
- Karl Oelschlager. A martingale approach to the law of large numbers for weakly interacting stochastic processes. Annals of Probability, 12:458–479, 1984.
- Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007.
- Trainability and accuracy of artificial neural networks: An interacting particle system approach. Communications on Pure and Applied Mathematics, 75(9):1889–1935, 2022.
- Mean field analysis of neural networks: A central limit theorem. Stochastic Processes and their Applications, 130(3):1820–1852, 2020.
- A convergence theory for svgd in the population limit under talagrand’s inequality t1. In International Conference on Machine Learning, pages 19139–19152. PMLR, 2022.
- Alain-Sol Sznitman. Nonlinear reflecting diffusion process, and the propagation of chaos and fluctuations associated. Journal of functional analysis, 56(3):311–336, 1984.
- Julian Tugaut. Convergence to the equilibria for self-stabilizing processes in double-well landscape. The Annals of Probability, 41(3A):1427 – 1460, 2013.
- A Yu Veretennikov. On ergodic measures for mckean-vlasov stochastic equations. In Monte Carlo and Quasi-Monte Carlo Methods 2004, pages 471–486. Springer, 2006.
- Cédric Villani. Mathematics of granular materials. Journal of statistical physics, 124(2-4):781–822, 2006.
- C. Villani. Optimal transport, volume 338 of Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag, Berlin, 2009. Old and new.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.