Papers
Topics
Authors
Recent
Search
2000 character limit reached

Towards a turnkey approach to unbiased Monte Carlo estimation of smooth functions of expectations

Published 29 Mar 2024 in stat.ME and stat.CO | (2403.20313v3)

Abstract: Given a smooth function $f$, we develop a general approach to turn Monte Carlo samples with expectation $m$ into an unbiased estimate of $f(m)$. Specifically, we develop estimators that are based on randomly truncating the Taylor series expansion of $f$ and estimating the coefficients of the truncated series. We derive their properties and propose a strategy to set their tuning parameters -- which depend on $m$ -- automatically, with a view to make the whole approach simple to use. We develop our methods for the specific functions $f(x)=\log x$ and $f(x)=1/x$, as they arise in several statistical applications such as maximum likelihood estimation of latent variable models and Bayesian inference for un-normalised models. Detailed numerical studies are performed for a range of applications to determine how competitive and reliable the proposed approach is.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Unbiased Monte Carlo: Posterior estimation for intractable/infinite-dimensional models. Bernoulli, 24(3):1726 – 1786.
  2. A stochastic algorithm for probabilistic independent component analysis. Annals of Applied Statistics, 6(1):125–160.
  3. Efficient optimization of loops and limits with randomized telescoping sums. In International Conference on Machine Learning, pages 534–543. PMLR.
  4. Billingsley, P. (1995). Probability and Measure. John Wiley & Sons.
  5. Unbiased Monte Carlo computation of smooth functions of expectations via Taylor expansions. In 2015 Winter Simulation Conference (WSC), pages 360–367. IEEE.
  6. Unbiased Monte Carlo for optimization and functions of expectations via multi-level randomization. In 2015 Winter Simulation Conference (WSC), pages 3656–3667. IEEE.
  7. Unbiased multilevel Monte Carlo: Stochastic optimization, steady-state simulation, quantiles, and other applications. arXiv preprint arXiv:1904.09929.
  8. Boas Jr, R. P. (1965). Tannery’s Theorem. Mathematics Magazine, 38(2):66–66.
  9. Importance weighted autoencoders. In 4th International Conference on Learning Representations (ICLR).
  10. Bayesian inference for exponential random graph models. Social Networks, 33(1):41–55.
  11. On the optimal design of the randomized unbiased Monte Carlo estimators. Operations Research Letters, 49(4):477–484.
  12. Waste-free sequential Monte Carlo. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):114–148.
  13. Convergence of a stochastic approximation version of the EM algorithm. Annals of Statistics, pages 94–128.
  14. An introduction to the bootstrap, volume 57 of Monographs on Statistics and Applied Probability. New York, NY: Chapman & Hall.
  15. Giles, M. B. (2008). Multilevel Monte Carlo path simulation. Operations Research, 56(3):607–617.
  16. Exact estimation for Markov chain equilibrium expectations. Journal of Applied Probability, 51(A):377–389.
  17. The asymptotic efficiency of simulation estimators. Operations Research, 40(3):505–520.
  18. Hoeffding, W. (1948). A non-parametric test of independence. Annals of Mathematical Statistics, 19(4):546–557.
  19. On nonnegative unbiased estimators. Annals of Statistics.
  20. Properties of the stochastic approximation EM algorithm with mini-batch sampling. Statistics and Computing, 30(6):1725–1739.
  21. SUMO: Unbiased estimation of log marginal probability for latent variable models. In International Conference on Learning Representations (ICLR).
  22. On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods. Statistical Science, 30(4):443–467.
  23. McLeish, D. (2011). A general method for debiasing a Monte Carlo estimator. Monte Carlo Methods and Applications, 17(4):301–315.
  24. Owen, A. B. (1997). Scrambled net variance for integrals of smooth functions. Annals of Statistics, 25(4):1541–1562.
  25. Robust action and the rise of the Medici, 1400-1434. American Journal of Sociology, 98(6):1259–1319.
  26. Papaspiliopoulos, O. (2011). Monte Carlo probabilistic inference for diffusion processes: A methodological framework. In Barber, D., Cemgil, A. T., and Chiappa, S., editors, Bayesian Time Series Models, pages 82–103. Cambridge University Press.
  27. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4):838–855.
  28. Tighter variational bounds are not necessarily better. In International Conference on Machine Learning, pages 4277–4285. PMLR.
  29. Unbiased estimation with square root convergence for SDE models. Operations Research, 63(5):1026–1043.
  30. A stochastic approximation method. Annals of Mathematical Statistics, pages 400–407.
  31. On multilevel Monte Carlo unbiased gradient estimation for deep latent variable models. In International Conference on Artificial Intelligence and Statistics, pages 3925–3933. PMLR.
  32. Vihola, M. (2018). Unbiased estimators and multilevel Monte Carlo. Operations Research, 66(2):448–462.
  33. Optimal distributions for randomized unbiased estimators with an infinite horizon and an adaptive algorithm. arXiv preprint arXiv:2304.07797.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 3 tweets with 4 likes about this paper.

Sign Up for Free