- The paper introduces a novel Gibbs sampling scheme using auxiliary variables, ensuring conjugacy and efficient estimation with the horseshoe prior.
- The method utilizes a scale mixture representation of the half-Cauchy distribution to enable straightforward Gibbs sampling with inverse-gamma conditionals.
- Comparative simulations show the sampler is up to 40 times faster and achieves higher effective sample sizes than traditional slice sampling approaches.
A Simple Sampler for the Horseshoe Estimator
In "A simple sampler for the horseshoe estimator," Makalic and Schmidt present a novel Bayesian sampler designed to streamline sampling procedures within the context of linear regression models employing the horseshoe prior. The horseshoe model is widely recognized for its global-local shrinkage capabilities, effectively managing small coefficients indicative of noise while preserving substantial coefficients that represent signal strength. This characteristic sets it apart from classic Bayesian methodologies, such as the lasso and ridge approaches, which uniformly apply shrinkage across all coefficients.
One of the key contributions of this paper is the derivation of a new sampling scheme utilizing auxiliary variables. This approach confers multiple benefits, chief among them being the conjugacy it ensures for all parameters. This property makes Gibbs sampling particularly straightforward to implement, facilitating efficient posterior distribution sampling without the difficulties associated with non-standard conditional posterior distributions.
The authors elucidate the scale mixture representation of the half-Cauchy distribution, a cornerstone of their sampling method. This representation allows them to recast the horseshoe hierarchy such that efficient Gibbs sampling is feasible. The conditional posterior distributions of all parameters, apart from the regression coefficients, are inverse-gamma distributed, for which robust samplers are readily available. This yields a significant improvement over existing approaches that rely on slice sampling for the hyperparameters.
Additionally, the paper discusses the extension of Gibbs sampling to logistic regression and other models. By using the Pólya-gamma data augmentation method for modeling logistic regression, the authors extend the utility of the horseshoe prior to models where binary outcomes or count data are prominent, further broadening its applicability in statistical modeling.
The authors provide a compelling comparison of their implementation (bhs) against the monomvn package, known for its conventional slice sampling approach. Their simulations indicate that for larger datasets and a higher number of predictors, their method outperformed monomvn in terms of execution speed, showcasing an example where it was approximately 40 times faster for a dataset with 1,000 observations and 1,000 predictors. Moreover, their implementation demonstrated superior sampling efficiency, with higher effective sample sizes across varying levels of thinning compared to monomvn.
The methodological advancement introduced in this paper not only improves operational efficiency but also contributes to the theoretical understanding of the horseshoe model. The conjugacy and sampling efficiency conferred by the auxiliary variable strategy bolster its appeal for practitioners focused on high-dimensional data analysis. Furthermore, the ease with which marginal likelihoods can be computed utilizing Chib's algorithm due to the known normalizing constants offers a distinct tactical advantage.
Future work in this domain may yield further extensions to other hierarchical models and explore potential optimizations in computational procedures, especially in cases dealing with immense datasets. This paper's methodological contributions enhance the toolkit available for Bayesian regression, promoting the manageable application of the horseshoe model in modern statistical analysis.