Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A calculus for Markov chain Monte Carlo: studying approximations in algorithms (2310.03853v2)

Published 5 Oct 2023 in math.PR, math.ST, stat.CO, and stat.TH

Abstract: Markov chain Monte Carlo (MCMC) algorithms are based on the construction of a Markov chain with transition probabilities leaving invariant a probability distribution of interest. In this work, we look at these transition probabilities as functions of their invariant distributions, and we develop a notion of derivative in the invariant distribution of a MCMC kernel. We build around this concept a set of tools that we refer to as Markov chain Monte Carlo Calculus. This allows us to compare Markov chains with different invariant distributions within a suitable class via what we refer to as mean value inequalities. We explain how MCMC Calculus provides a natural framework to study algorithms using an approximation of an invariant distribution, and we illustrate this by using the tools developed to prove convergence of interacting and sequential MCMC algorithms. Finally, we discuss how similar ideas can be used in other frameworks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. “Optimal scaling of MCMC beyond Metropolis” In Advances in Applied Probability 55, 2023, pp. 492–509
  2. Luigi Ambrosio, Nicola Gigli and Giuseppe Savaré “Gradient flows: in metric spaces and in the space of probability measures” Birkhäuser Verlag, 2005
  3. Christophe Andrieu, Laird A Breyer and Arnaud Doucet “Convergence of simulated annealing using Foster-Lyapunov criteria” In Journal of Applied Probability 38, 2001, pp. 975–994
  4. “On nonlinear Markov chain Monte Carlo” In Bernoulli 17, 2011, pp. 987–1014
  5. “Complexity of Gibbs samplers through Bayesian asymptotics” In arXiv:2304.06993, 2023
  6. Yves F Atchadé “A cautionary tale on the efficiency of some adaptive Monte Carlo schemes” In Annals of Applied Probability 20, 2010, pp. 841–868
  7. Anthony A Barker “Monte carlo calculations of the radial distribution functions for a proton-electron plasma” In Australian Journal of Physics 18, 1965, pp. 119–134
  8. Bernard Bercu, Pierre Del Moral and Arnaud Doucet “Fluctuations of interacting Markov chain Monte Carlo methods” In Stochastic Processes and their Applications 122, 2012, pp. 1304–1331
  9. “Dynamic conditional independence models and Markov chain Monte Carlo methods” In Journal of the American Statistical Association 92, 1997, pp. 1403–1412
  10. Patrick Billingsley “Convergence of probability measures” New York: John Wiley & Sons, 1999
  11. Anthony Brockwell, Pierre Del Moral and Arnaud Doucet “Sequentially interacting Markov chain Monte Carlo methods” In Annals of Statistics 38, 2010, pp. 3387–3411
  12. “The master equation and the convergence problem in mean field games” Princeton, NJ: Princeton University Press, 2019
  13. “An introduction to sequential Monte Carlo” Springer, 2020
  14. Pierre Del Moral “Feynman-Kac formulae: genealogical and interacting particle systems with applications” New York: Springer-Verlag, 2004
  15. Pierre Del Moral and Arnaud Doucet “Interacting Markov chain Monte Carlo methods for solving nonlinear measure-valued equations” In Annals of Applied Probability 20, 2010, pp. 593–639
  16. “Limit theorems for weighted samples with applications to sequential Monte Carlo methods” In Annals of Statistics 36, 2008, pp. 2344–2376
  17. Richard M Dudley “Real Analysis and Probability” Cambridge University Press, 2002
  18. Axel Finke, Arnaud Doucet and Adam M Johansen “Limit theorems for sequential MCMC methods” In Advances in Applied Probability 52, 2020, pp. 377–403
  19. Gersende Fort, Eric Moulines and Pierre Priouret “Convergence of adaptive and interacting Markov chain Monte Carlo algorithms” In Annals of Statistics 39, 2011, pp. 3262–3289
  20. “A central limit theorem for adaptive and interacting Markov chains” In Bernoulli 20, 2014, pp. 457–485
  21. Marylou Gabrié, Grant M Rotskoff and Eric Vanden-Eijnden “Efficient Bayesian sampling using normalizing flows to assist Markov chain Monte Carlo Methods” In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021
  22. Marco A Gallegos-Herrada, David Ledvinka and Jeffrey S Rosenthal “Equivalences of Geometric Ergodicity of Markov Chains” In Journal of Theoretical Probability to appear, 2023, pp. 1–27
  23. Andrew Golightly and Darren J Wilkinson “Bayesian sequential inference for nonlinear multivariate diffusions” In Statistics and Computing 16, 2006, pp. 323–338
  24. Keith W Hastings “Monte Carlo sampling methods using Markov chains and their applications” In Biometrika 57, 1970, pp. 97–109
  25. “Weak differentiability of product measures” In Mathematics of Operations Research 35, 2010, pp. 27–51
  26. Peter J Huber and Elvezio M Ronchetti “Robust statistics” Hoboken, NJ: John Wiley & Sons, 2009
  27. Søren Fiig Jarner and Ernst Hansen “Geometric ergodicity of Metropolis algorithms” In Stochastic Processes and their Applications 85, 2000, pp. 341–361
  28. Irving Kaplansky “Set Theory and Metric Spaces” AMS Chelsea Publishing, 1972
  29. Achim Klenke “Probability theory: a comprehensive course” Springer London, 2013
  30. “An Adaptive and Scalable Multi-Object Tracker Based on the Non-Homogeneous Poisson Process” In IEEE Transactions on Signal Processing 71 IEEE, 2023, pp. 105–120
  31. Youssef M Marzouk, Habib N Najm and Larry A Rahn “Stochastic spectral methods for efficient Bayesian solution of inverse problems” In Journal of Computational Physics 224, 2007, pp. 560–586
  32. Youssef M Marzouk and Dongbin Xiu “A Stochastic Collocation Approach to Bayesian Inference in Inverse Problems” In Communications in Computational Physics 6, 2009, pp. 826–847
  33. Kerrie L Mengersen and Richard L Tweedie “Rates of convergence of the Hastings and Metropolis algorithms” In Annals of Statistics 24, 1996, pp. 101–121
  34. “Equation of state calculations by fast computing machines” In Journal of Chemical Physics 21, 1953, pp. 1087–1092
  35. Sean P Meyn and Richard L Tweedie “Markov chains and stochastic stability” Cambridge: Cambridge University Press, 2009
  36. “Normalizing flows for probabilistic modeling and inference” In Journal of Machine Learning Research 22, 2021, pp. 2617–2680
  37. Christian Ritter and Martin A Tanner “Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler” In Journal of the American Statistical Association 87, 1992, pp. 861–868
  38. Gareth O Roberts and Jeffrey S Rosenthal “General state space Markov chains and MCMC algorithms” In Probability surveys 1, 2004, pp. 20–71
  39. Gareth O Roberts and Richard L Tweedie “Exponential convergence of Langevin distributions and their discrete approximations” In Bernoulli 2, 1996, pp. 341–363
  40. Filippo Santambrogio “Optimal transport for applied mathematicians: Calculus of variations, PDEs, and modeling” Birkhäuser/Springer, 2015
  41. “On MCMC-based particle methods for Bayesian filtering: Application to multitarget tracking” In 2009 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), 2009, pp. 360–363
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now