2000 character limit reached
Fast Bootstrapping Nonparametric Maximum Likelihood for Latent Mixture Models (2402.18748v1)
Published 28 Feb 2024 in stat.ME
Abstract: Estimating the mixing density of a latent mixture model is an important task in signal processing. Nonparametric maximum likelihood estimation is one popular approach to this problem. If the latent variable distribution is assumed to be continuous, then bootstrapping can be used to approximate it. However, traditional bootstrapping requires repeated evaluations on resampled data and is not scalable. In this letter, we construct a generative process to rapidly produce nonparametric maximum likelihood bootstrap estimates. Our method requires only a single evaluation of a novel two-stage optimization algorithm. Simulations and real data analyses demonstrate that our procedure accurately estimates the mixing density with little computational cost even when there are a hundred thousand observations.
- H. T. Huynh and L. Nguyen, “Nonparametric maximum likelihood estimation using neural networks,” Pattern Recognition Letters, vol. 138, pp. 580–586, 2020.
- Y. Li and Z. Ye, “Boosting in univariate nonparametric maximum likelihood estimation,” IEEE Signal Processing Letters, vol. 28, pp. 623–627, 2021.
- V. Bhatia and B. Mulgrew, “Non-parametric likelihood based channel estimator for Gaussian mixture noise,” Signal Processing, vol. 87, no. 11, pp. 2569–2586, 2007.
- L. Feng, R. Ma, and L. H. Dicker, “Nonparametric maximum likelihood approximate message passing,” in 2017 51st Annual Conference on Information Sciences and Systems (CISS). IEEE, 2017, pp. 1–6.
- B. W. Silverman, M. C. Jones, J. D. Wilson, and D. W. Nychka, “A smoothed EM approach to indirect estimation problems, with particular, reference to stereology and emission tomography,” Journal of the Royal Statistical Society. Series B (Methodological), vol. 52, no. 2, pp. 271–324, 1990.
- Y. Vardi, L. A. Shepp, and L. Kaufman, “A statistical model for positron emission tomography,” Journal of the American Statistical Association, vol. 80, no. 389, pp. 8–20, 1985.
- B. Efron, “Empirical Bayes deconvolution estimates,” Biometrika, vol. 103, no. 1, pp. 1–20, 2016.
- H. Jiang and A. Guntuboyina, “A nonparametric maximum likelihood approach to mixture of regression,” arXiv preprint arXiv:2108.09816, 2021.
- Z. Fan, L. Guan, Y. Shen, and Y. Wu, “Gradient flows for empirical Bayes in high-dimensional linear models,” arXiv preprint arXiv:2312.12708, 2023.
- G. J. McLachlan, S. X. Lee, and S. I. Rathnayake, “Finite mixture models,” Annual Review of Statistics and Its Application, vol. 6, pp. 355–378, 2019.
- J. Kiefer and J. Wolfowitz, “Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters,” The Annals of Mathematical Statistics, vol. 27, no. 4, pp. 887–906, 1956.
- B. G. Lindsay, “Mixture models: Theory, geometry and applications,” in NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5, 1995, pp. 1–163.
- N. Laird, “Nonparametric maximum likelihood estimation of a mixing distribution,” Journal of the American Statistical Association, vol. 73, no. 364, pp. 805–811, 1978.
- C. H. Zhang, “Compound decision theory and empirical Bayes methods,” Annals of Statistics, vol. 31, no. 2, pp. 379–390, 2003.
- R. Koenker and I. Mizera, “Convex optimization, shape constraints, compound decisions, and empirical Bayes rules,” Journal of the American Statistical Association, vol. 109, no. 506, pp. 674–685, 2014.
- L. Feng and L. H. Dicker, “Approximate nonparametric maximum likelihood for mixture models: A convex optimization approach to fitting arbitrary multivariate mixing distributions,” Computational Statistics & Data Analysis, vol. 122, pp. 80–91, 2018.
- L. Liu, M. Levine, and Y. Zhu, “A functional EM algorithm for mixing density estimation via nonparametric penalized likelihood maximization,” Journal of Computational and Graphical Statistics, vol. 18, no. 2, pp. 481–504, 2009.
- R. Koenker and J. Gu, “Comment: Minimalist g-modeling,” Statistical Science, vol. 34, no. 2, pp. 209–213, 2019.
- M. Knott and P. Tzamourani, “Bootstrapping the estimated latent distribution of the two-parameter latent trait model,” British Journal of Mathematical and Statistical Psychology, vol. 60, no. 1, pp. 175–191, 2007.
- D. B. Rubin, “The Bayesian bootstrap,” The Annals of Statistics, vol. 9, no. 1, p. 130434, 1981.
- M. A. Newton and A. E. Raftery, “Approximate Bayesian inference with the weighted likelihood bootstrap,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 56, no. 1, pp. 3–26, 1994.
- M. A. Newton, N. G. Polson, and J. Xu, “Weighted Bayesian bootstrap for scalable posterior distributions,” Canadian Journal of Statistics, vol. 49, no. 2, pp. 421–437, 2021.
- B. Efron, “Bootstrap methods: Another look at the jackknife,” The Annals of Statistics, vol. 7, no. 1, pp. 1–26, 1979.
- M. Shin, S. Wang, and J. S. Liu, “Generative multiple-purpose sampler for weighted M-estimation,” Journal of Computational and Graphical Statistics (to appear), 2023.
- D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
- I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in Neural Information Processing Systems, vol. 27, 2014.
- X. Zhou, Y. Jiao, J. Liu, and J. Huang, “A deep generative approach to conditional sampling,” Journal of the American Statistical Association, vol. 118, no. 543, pp. 1837–1848, 2023.
- S. Liu, X. Zhou, Y. Jiao, and J. Huang, “Wasserstein generative learning of conditional distribution,” arXiv preprint arXiv:2112.10039, 2021.
- Y. Qiu and X. Wang, “ALMOND: Adaptive latent modeling and optimization via neural networks and langevin diffusion,” Journal of the American Statistical Association, vol. 116, no. 535, pp. 1224–1236, 2021.
- G. O. Roberts and R. L. Tweedie, “Exponential convergence of Langevin distributions and their discrete approximations,” Bernoulli, vol. 2, no. 4, pp. 341–363, 1996.
- K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. 359–366, 1989.
- K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural networks, vol. 4, no. 2, pp. 251–257, 1991.
- F. Emmert-Streib, Z. Yang, H. Feng, S. Tripathi, and M. Dehmer, “An introductory review of deep learning for prediction models with big data,” Frontiers in Artificial Intelligence, vol. 3, 2020.
- X. Lian, M. Wang, and J. Liu, “Finite-sum composition optimization via variance reduced gradient descent,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, A. Singh and J. Zhu, Eds., vol. 54. PMLR, 20–22 Apr 2017, pp. 1159–1167.
- R. A. Levine and G. Casella, “Implementations of the Monte Carlo EM algorithm,” Journal of Computational and Graphical Statistics, vol. 10, no. 3, pp. 422–439, 2001.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
- R. Koenker and J. Gu, “REBayes: An R package for empirical Bayes mixture methods,” Journal of Statistical Software, vol. 82, no. 8, p. 1–26, 2017.
- R. Norberg, “Experience rating in group life insurance,” Scandinavian Actuarial Journal, vol. 1989, no. 4, pp. 194–224, 1989.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.