Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dirichlet process mixtures of block $g$ priors for model selection and prediction in linear models (2411.00471v1)

Published 1 Nov 2024 in stat.ME and cs.LG

Abstract: This paper introduces Dirichlet process mixtures of block $g$ priors for model selection and prediction in linear models. These priors are extensions of traditional mixtures of $g$ priors that allow for differential shrinkage for various (data-selected) blocks of parameters while fully accounting for the predictors' correlation structure, providing a bridge between the literatures on model selection and continuous shrinkage priors. We show that Dirichlet process mixtures of block $g$ priors are consistent in various senses and, in particular, that they avoid the conditional Lindley ``paradox'' highlighted by Som et al.(2016). Further, we develop a Markov chain Monte Carlo algorithm for posterior inference that requires only minimal ad-hoc tuning. Finally, we investigate the empirical performance of the prior in various real and simulated datasets. In the presence of a small number of very large effects, Dirichlet process mixtures of block $g$ priors lead to higher power for detecting smaller but significant effects without only a minimal increase in the number of false discoveries.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Bayesian robustness modelling of location and scale parameters. Scandinavian Journal of Statistics 38, 691–711.
  2. Antoniak, C. E. (1974). Mixtures of dirichlet processes with applications to bayesian nonparametric problems. The annals of statistics pp. 1152–1174.
  3. On the beta prime prior for scale parameters in high-dimensional Bayesian regression models. arXiv preprint arXiv:1807.06539 .
  4. Criteria for Bayesian model choice with application to variable selection. The Annals of Statistics 40, 1550–1577.
  5. The formal definition of reference priors. Annals of Statistics 37, 905–938.
  6. The intrinsic Bayes factor for linear models. In Bayesian Statistics 5, Eds. A. P. D. J. M. Bernardo, J. O. Berger & A. F. M. Smith, pp. 25–44. Oxford Univ. Press.
  7. Bayes factors and marginal distributions in invariant situations. Sankhyā: The Indian Journal of Statistics, Series A pp. 307–321.
  8. Bertoin, J. (2006). Random fragmentation and coagulation processes, volume 102. Cambridge University Press.
  9. The horseshoe+ estimator of ultra-sparse signals. Bayesian Analysis 12, 1105–1131.
  10. Dirichlet–Laplace priors for optimal shrinkage. Journal of the American Statistical Association 110, 1479–1490.
  11. Ferguson distributions via pólya urn schemes. The annals of statistics 1, 353–355.
  12. Group inverse-gamma gamma shrinkage for sparse linear models with block-correlated regressors. Bayesian Analysis 1, 1–30.
  13. Hyper-g𝑔gitalic_g priors for generalized linear models. Bayesian Analysis 6, 387–410.
  14. Estimating optimal transformations for multiple regression and correlation. Journal of the American statistical Association 80, 580–598.
  15. Inference with normal-gamma prior distributions in regression problems. Bayesian Analysis 5, 171–188.
  16. The horseshoe estimator for sparse signals. Biometrika 97, 465–480.
  17. Objective Bayesian model selection in Gaussian graphical models. Biometrika 96, 497–512.
  18. Objective bayesian variable selection. Journal of the American Statistical Association 101, 157–167.
  19. Prior distributions for objective Bayesian analysis. Bayesian Analysis 13, 627–679.
  20. Consistent fractional Bayes factor for nested normal linear models. Journal of statistical planning and inference 97, 305–321.
  21. A horseshoe mixture model for bayesian screening with an application to light sheet fluorescence microscopy in brain imaging. The Annals of Applied Statistics 17, 2639–2658.
  22. Ferguson, T. S. (1973). A bayesian analysis of some nonparametric problems. The annals of statistics pp. 209–230.
  23. Robust Bayesian graphical modeling using Dirichlet t-distributions. Bayesian Analysis 9, 521–550.
  24. Methods and tools for Bayesian variable selection and model averaging in normal linear regression. International Statistical Review 86, 237–258.
  25. Power-expected-posterior priors for variable selection in Gaussian linear models. Bayesian Analysis 10, 75–107.
  26. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102, 359–378.
  27. Gordy, M. B. (1998). A generalization of generalized Beta distributions. Technical report, Division of Research and Statistics, Division of Monetary Affairs, Federal Reserve.
  28. Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732.
  29. Alternative prior distributions for variable selection with very many more variables than observations. University of Kent Technical Report .
  30. Hans, C. (2009). Bayesian lasso regression. Biometrika 96, 835–845.
  31. Adaptive lasso for sparse high-dimensional regression models. Statistica Sinica pp. 1603–1618.
  32. On the use of non-local prior densities in Bayesian hypothesis tests. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 72, 143–170.
  33. Bayesian model selection in high-dimensional settings. Journal of the American Statistical Association 107, 649–660.
  34. A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. Journal of the American Statistical Association 90, 928–934.
  35. Continuous shrinkage prior revisited: a collapsing behavior and remedy. arXiv preprint arXiv:2007.02192 .
  36. Bayesian adaptive lasso. Annals of the Institute of Statistical Mathematics 66, 221–244.
  37. Variable selection using shrinkage priors. Computational Statistics & Data Analysis 107, 107–119.
  38. Mixtures of g-priors in generalized linear models. Journal of the American Statistical Association 113, 1828–1845.
  39. Mixtures of g-priors for Bayesian variable selection. Journal of the American Statistical Association 103, 410–423.
  40. Rejection sampling for an extended gamma distribution. Unpublished manuscript .
  41. Neal, R. M. (2000). Markov chain sampling methods for dirichlet process mixture models. Journal of computational and graphical statistics 9, 249–265.
  42. O’Hagan, A. (1995). Fractional Bayes factors for model comparison. Journal of the Royal Statistical Society: Series B (Methodological) 57, 99–118.
  43. The Bayesian lasso. Journal of the American Statistical Association 103, 681–686.
  44. Local shrinkage rules, lévy processes and regularized regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 74, 287–311.
  45. Bayesian inference for logistic models using Pólya–Gamma latent variables. Journal of the American Statistical Association 108, 1339–1349.
  46. Effect of model space priors on statistical inference with model uncertainty. The New England Journal of Statistics in Data Science pp. 1–10.
  47. Laplace power-expected-posterior priors for logistic regression. Bayesian Analysis 1, 1–24.
  48. Rodríguez, A. (2013). On the jeffreys prior for the multivariate ewens distribution. Statistics & Probability Letters 83, 1539–1546.
  49. Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. The Annals of Statistics pp. 2587–2619.
  50. Sethuraman, J. (1994). A constructive definition of dirichlet priors. Statistica sinica pp. 639–650.
  51. Som, A. (2014). Paradoxes and Priors in Bayesian Regression. Ph.D. thesis, The Ohio State University.
  52. A conditional Lindley paradox in Bayesian linear models. Biometrika 103, 993–999.
  53. Tipping, M. E. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research 1, 211–244.
  54. Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, Eds. P. K. Goel & A. Zellner, pp. 233–243. Amsterdam: North-Holland/Elsevier.
  55. Posterior odds ratios for selected regression hypotheses. Trabajos de Estadística y de Investigaciów Operativa 31, 585–603.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets