2000 character limit reached
Parallel Affine Transformation Tuning of Markov Chain Monte Carlo (2401.16567v2)
Published 29 Jan 2024 in stat.ME and stat.ML
Abstract: The performance of Markov chain Monte Carlo samplers strongly depends on the properties of the target distribution such as its covariance structure, the location of its probability mass and its tail behavior. We explore the use of bijective affine transformations of the sample space to improve the properties of the target distribution and thereby the performance of samplers running in the transformed space. In particular, we propose a flexible and user-friendly scheme for adaptively learning the affine transformation during sampling. Moreover, the combination of our scheme with Gibbsian polar slice sampling is shown to produce samples of high quality at comparatively low computational cost in several settings based on real-world data.
- Time bounds for selection. Journal of Computer and System Sciences, 7(4):448–461, 1973.
- Transport elliptical slice sampling. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 3664–3676. PMLR, 2023.
- Air Markov chain Monte Carlo. arXiv preprint arXiv:1801.09309, 2018.
- Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47(4):547–553, 2009.
- MCMC methods for functions: Modifying old algorithms to make them faster. Statistical Science, 28(3):424–446, 2013.
- Learn from thy neighbor: Parallel-chain and regional adaptive MCMC. Journal of the American Statistical Association, 104(488):1454–1466, 2009.
- Bayesian Data Analysis (3rd ed.). Chapman and Hall, 2013.
- Adaptive independent Metropolis-Hastings by fast estimation of mixtures of normals. Journal of Computational and Graphical Statistics, 19(2):243–259, 2010.
- On sampling with approximate transport maps. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 11698–11733. PMLR, 2023.
- An adaptive Metropolis algorithm. Bernoulli, 7(2):223–242, 2001.
- Geodesic slice sampling on the sphere. arXiv preprint arXiv:2301.08056, 2023.
- Reversibility of elliptical slice sampling revisited. arXiv preprint arXiv:2301.02426, 2023.
- Quantifying the effectiveness of linear preconditioning in Markov chain Monte Carlo. arXiv preprint arXiv:2312.04898, 2023.
- The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15:1593–1623, 2014.
- Hofmann, H. Statlog (german credit data). UCI Machine Learning Repository, 1994.
- Generalized elliptical slice sampling with regional pseudo-priors. arXiv preprint arXiv:1903.05309, 2019.
- Dimension-independent Markov chain Monte Carlo on the sphere. Scandinavian Journal of Statistics, 2023.
- The geometry of logconcave functions and sampling algorithms. Random Structures and Algorithms, 30(3):307–358, 2007.
- MacKay, D. Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2003.
- Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 21(6):1087–1092, 1953.
- Elliptical slice sampling. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, volume 9, pp. 541–548. PMLR, 2010.
- Müller, P. A generic approach to posterior integration and Gibbs sampling. Technical report, Purdue University, 1991.
- Neal, R. M. Bayesian Learning for Neural Networks, volume 118 of Lecture Notes in Statistics. Springer, New York, 1996.
- Neal, R. M. Slice sampling. The Annals of Statistics, 31(3):705–767, 2003.
- Parallel MCMC with generalized elliptical slice sampling. Journal of Machine Learning Research, 15:2087–2112, 2014.
- Transport map accelerated Markov chain Monte Carlo. SIAM/ASA Journal on Uncertainty Quantification, 6(2):645–682, 2018.
- Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. MIT Press, 2006.
- The polar slice sampler. Stochastic Models, 18(2):257–280, 2002.
- Examples of adaptive MCMC. Journal of Computational and Graphical Statistics, 18(2):349–367, 2009.
- Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli, 2(4):341–363, 1996.
- Rudolf, D. Explicit error bounds for Markov chain Monte Carlo. Dissertationes Mathematicae, 485:1–93, 2012.
- Dimension-independent spectral gap of polar slice sampling. Statistics and Computing, 34(article 20), 2024.
- Robust random walk-like Metropolis-Hastings algorithms for concentrating posteriors. arXiv preprint arXiv:2202.12127, 2022.
- Schär, P. Wasserstein contraction and spectral gap of slice sampling revisited. Electronic Journal of Probability, 28(article 136), 2023.
- A dimension-independent bound on the Wasserstein contraction rate of geodesic slice sampling on the sphere for uniform target. arXiv preprint arXiv:2309.09097, 2023.
- Gibbsian polar slice sampling. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 30204–30223. PMLR, 2023.
- Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the Annual Symposium on Computer Applications in Medical Care, pp. 261–265, 1988.
- Breast cancer Wisconsin (diagnostic). UCI Machine Learning Repository, 1995.
- Automated factor slice sampling. Journal of Computational and Graphical Statistics, 23(2):543–563, 2014.
- Tierney, L. Markov chains for exploring posterior distributions. The Annals of Statistics, 22(4):1701–1728, 1994.
- Stereographic Markov chain Monte Carlo. arXiv preprint arXiv:2205.12112, 2022.