Horseshoe Priors: Bayesian Shrinkage

• Horseshoe priors are a Bayesian shrinkage framework using a global–local Gaussian scale mixture to concentrate mass near zero while preserving significant signals.
• The methodology employs heavy-tailed distributions to balance aggressive noise compression with minimal shrinkage for large coefficients, achieving near-minimax risk.
• Variants such as regularized and heavy-tailed forms extend applications to regression, neural networks, and nonparametric function estimation, making the approach versatile.

Searching arXiv for recent and foundational papers on horseshoe priors and variants. The horseshoe family of priors is a class of Bayesian shrinkage priors built for sparse or nearly sparse inference. Its canonical form is a global–local Gaussian scale mixture in which a common global scale controls overall shrinkage while coefficient-specific local scales allow a small number of signals to escape that shrinkage. Across the literature, the family is characterized by strong concentration near zero together with heavy tails, so that weak coordinates can be aggressively compressed while large coordinates are not over-shrunk (Ghosh et al., 2015). Subsequent work has extended this core idea in several directions: heavier-tailed variants for ultra-sparse normal means, regularized forms that decouple sparsity from large-signal regularization, subspace and nonlinear-function shrinkage priors, structured versions acting on differences or graph-like quantities, and application-specific formulations in inverse problems, Bayesian neural networks, graphical models, nonparametric regression, and multiple testing (Womack et al., 2019, Piironen et al., 2017, Shin et al., 2016, Duda et al., 2024, Uribe et al., 2022, Ghosh et al., 2017, Banerjee et al., 29 Jun 2026).

1. Canonical global–local construction

In its standard form, the horseshoe prior is written as

$$\beta_j \mid \lambda_j,\tau \sim \mathcal{N}(0,\tau^2\lambda_j^2),\qquad \lambda_j \sim C^+(0,1),\qquad \tau \sim C^+(0,1),$$

with $\lambda_j$ the local scale and $\tau$ the global scale (Piironen et al., 2017, Banerjee et al., 29 Jun 2026). In grouped formulations, the same structure is applied to vectors rather than scalar coefficients; for example, in Bayesian neural networks a whole incoming weight vector to a hidden unit can share a local scale, with a layer-specific global scale (Ghosh et al., 2017).

A standard shrinkage representation uses

$\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$

so that posterior means take the form

$$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$

in normal means settings (Pas et al., 2014, Liang et al., 8 Feb 2025). Small $\kappa_j$ corresponds to little shrinkage, whereas $\kappa_j\approx 1$ corresponds to near-total shrinkage. Under the canonical scaling used in one asymptotic analysis, $\kappa\sim \mathrm{Beta}(1/2,1/2)$, making the prior U-shaped on the shrinkage scale and thereby favoring both extreme shrinkage and negligible shrinkage rather than intermediate regimes (Polson et al., 1 Apr 2026).

The family’s defining geometry is the combination of an unbounded spike near zero with heavy tails. For the canonical horseshoe marginal density $\pi_H(\theta)$, one analysis reports the two-sided bounds

$$\frac{K}{2}\log\!\left(1+\frac{4}{\theta^2}\right) < \pi_H(\theta) < K\log\!\left(1+\frac{2}{\theta^2}\right),\qquad K=(2\pi^3)^{-1/2},$$

hence

$\lambda_j$0

which is interpreted as a logarithmic pole at the origin together with Cauchy-class tails (Polson et al., 1 Apr 2026). A related default-prior analysis emphasizes that horseshoe and horseshoe+ belong to a regularly varying global–local class whose tails remain stable under many nonlinear transformations (Bhadra et al., 2015).

This global–local construction underlies the family’s canonical use in sparse normal means, regression, and latent Gaussian models. It is also the template from which later variants depart.

2. Sparse estimation, posterior concentration, and asymptotic optimality

A major theoretical line studies the horseshoe family in the sparse normal means model

$\lambda_j$1

with $\lambda_j$2 nearly black, meaning at most $\lambda_j$3 nonzero coordinates out of $\lambda_j$4 (Pas et al., 2014). In this regime, the minimax squared $\lambda_j$5-risk is of order

$\lambda_j$6

more sharply

$\lambda_j$7

in the formulation used for the horseshoe estimator (Pas et al., 2014).

For the horseshoe posterior mean $\lambda_j$8, one result shows

$\lambda_j$9

and for $\tau$0, $\tau$1, this attains the minimax rate up to constants (Pas et al., 2014). The same work proves posterior contraction and posterior-variance bounds of the same order, showing that the full posterior, not merely the posterior mean, contracts around the truth at the sparse minimax scale (Pas et al., 2014).

A broader asymptotic analysis studies one-group priors of the form

$\tau$2

with $\tau$3 slowly varying and $\tau$4 (Ghosh et al., 2015). Within this class, the horseshoe-type priors are those with $\tau$5, a subclass that includes the horseshoe, three-parameter beta normal mixtures with $\tau$6, and the generalized double Pareto with shape parameter $\tau$7 (Ghosh et al., 2015). For this class, Bayes estimators attain the minimax $\tau$8-risk up to constants, and for the horseshoe-type subclass the asymptotic constant is correct, yielding exact asymptotic minimaxity (Ghosh et al., 2015).

That same paper proves optimal posterior concentration for the broader class and shows that the natural testing rules induced by the posterior shrinkage weight are asymptotically Bayes optimal under sparsity (ABOS) in the sense of Bogdan et al.’s asymptotic testing framework (Ghosh et al., 2015). Another synthesis paper ties these finite-sample and asymptotic facts to a moderate deviation perspective, arguing that the horseshoe’s logarithmic singularity at zero and Cauchy-class tails together explain super-efficiency for nulls, robustness for signals, and ABOS-type risk behavior (Polson et al., 1 Apr 2026).

These results establish the horseshoe family as a continuous-shrinkage analogue of spike-and-slab behavior: no exact zeros are created by the prior itself, but the induced shrinkage profile can mimic sparse selection while retaining tractable continuous posterior structure.

3. Variant families and prior engineering

Several later papers modify the canonical horseshoe to sharpen specific aspects of its behavior.

Piironen and Vehtari identify two practical shortcomings of the original horseshoe: the lack of a systematic prior specification for the global parameter $\tau$9, and the inability to regularize large coefficients separately from sparse shrinkage (Piironen et al., 2017). They introduce the effective number of nonzero parameters,

$\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$0

and derive an interpretable scale choice

$\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$1

for a prior guess $\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$2 of the number of relevant coefficients in standardized Gaussian regression (Piironen et al., 2017). They also propose the regularized horseshoe,

$\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$3

which behaves like the original horseshoe near zero but imposes a finite slab width $\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$4 on large coefficients, making it a continuous counterpart of spike-and-slab with finite slab width rather than the infinitely wide slab associated with the original horseshoe (Piironen et al., 2017).

Another extension is the Heavy-tailed Horseshoe (HTHS), proposed for ultra-sparse normal means (Womack et al., 2019). Instead of fixing the gamma shape at $\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$5 in the local-scale hierarchy, HTHS randomizes that shape with a local decision parameter $\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$6, leading under a uniform prior on $\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$7 to the exact marginal

$\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$8

a log-Cauchy density (Womack et al., 2019). The induced shrinkage-profile prior is more singular at both 0 and 1 than the standard horseshoe, and the resulting signal prior has heavier tails: $\kappa_j=\frac{1}{1+\lambda_j^2\tau^2},$9 rather than the $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$0-type tails of HS/HS+ (Womack et al., 2019). The paper argues that this yields stronger “spikiness and slabbiness,” better Kullback–Leibler risk bounds at the origin, and better recovery in ultra-sparse regimes, while noting that advantages may diminish when sparsity is only moderate (Womack et al., 2019).

A different direction is functional shrinkage. The functional horseshoe prior (fHS) targets subspace proximity rather than coordinate sparsity (Shin et al., 2016). For a basis expansion $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$1 and a target subspace with projection matrix $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$2, the prior is

$$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$3

with shrinkage factor $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$4 (Shin et al., 2016). Here the prior shrinks the fitted function toward a parametric subspace rather than shrinking basis coefficients toward zero. A nonlinear extension, non-linear functional shrinkage (NLFS), replaces the fixed projection by a Jacobian-based local projection $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$5 onto the tangent space of a nonlinear function family such as Hill curves: $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$6 (Duda et al., 2024). This is a horseshoe-family prior in function space rather than parameter space, with $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$7 encoding how strongly the fit is shrunk toward the target nonlinear family (Duda et al., 2024).

These variants show that the horseshoe family is not a single prior but a design language: a way to combine strong shrinkage near a structured low-complexity object with enough tail mass to preserve departures supported by the data.

4. Structured and transformed horseshoe priors

A recurring pattern in later work is to apply horseshoe shrinkage not to coefficients themselves but to transformed quantities.

In Bayesian fused lasso modeling via horseshoe priors, the horseshoe is placed on successive differences $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$8 rather than directly on $$E(\beta_j\mid y_j,\tau)=\bigl(1-E(\kappa_j\mid y_j,\tau)\bigr)y_j$$9 (Kakikawa et al., 2022). Coefficients still receive Laplace shrinkage, but the fused component is

$\kappa_j$0

with half-Cauchy induced local and global scales on the difference prior (Kakikawa et al., 2022). The stated purpose is to prevent over-shrinkage of true nonzero differences while still favoring equality of adjacent coefficients. The same paper extends this idea to Bayesian HORSES, where horseshoe priors are imposed on all pairwise differences $\kappa_j$1, yielding structured shrinkage for coefficient grouping (Kakikawa et al., 2022).

In edge-preserving Bayesian inversion, the prior is placed on finite differences $\kappa_j$2 rather than on the signal $\kappa_j$3 itself (Uribe et al., 2022). The conditionally Gaussian Markov random field prior is

$\kappa_j$4

with horseshoe-type half-Cauchy hyperpriors on the global scale $\kappa_j$5 and local weights $\kappa_j$6 (Uribe et al., 2022). This makes small increments strongly shrunk while permitting a small number of large jumps at edges. Through inverse-gamma augmentation, the model admits closed-form Gibbs updates for all hyperparameters and a high-dimensional Gaussian update for $\kappa_j$7 (Uribe et al., 2022).

Structured transformations also appear in semi-parametric expert Bayesian networks. There the horseshoe prior is not placed on edge indicators or linear coefficients but on Gaussian-process amplitude parameters $\kappa_j$8 associated with parent-specific nonlinear components (Weng et al., 2024). The model uses

$\kappa_j$9

as a horseshoe-inspired half-Cauchy scale prior on GP amplitudes (Weng et al., 2024). The role is to shrink nonlinear departures from an expert-specified linear graph, thereby selecting nonlinear edge modifications or additions indirectly through amplitude thresholding (Weng et al., 2024). The paper explicitly notes that this is not the canonical local–global horseshoe hierarchy; rather, it is a weighted shrinkage penalty on GP amplitudes, with different $\kappa_j\approx 1$0 values for expert and non-expert edge classes functioning as a differential grouped shrinkage scheme (Weng et al., 2024).

In independent component estimation, horseshoe-type reasoning appears through latent Gaussian scale mixtures for super-Gaussian source priors (Datta et al., 2024). The working hierarchy is based on the hyperbolic secant prior,

$\kappa_j\approx 1$1

with an optional scale variable $\kappa_j\approx 1$2 (Datta et al., 2024). The paper does not deploy the canonical horseshoe directly, but interprets this latent scale-mixture construction as horseshoe-type, emphasizing the same themes of heavy-tailed adaptive shrinkage, Gaussian conditional structure, and scalable EM/MCMC inference (Datta et al., 2024).

These formulations illustrate a general principle: the horseshoe family often acts most naturally on the latent object whose sparsity is scientifically meaningful—differences, increments, group-wise scales, nonlinear departures, or latent-source amplitudes—rather than on raw coefficients alone.

5. Nonparametric and function-space formulations

Nonparametric regression has generated a distinct branch of horseshoe-family research in which the target is adaptation over smoothness classes rather than finite-dimensional sparsity.

The functional horseshoe prior adapts between a parametric subspace and a full spline space (Shin et al., 2016). When the truth belongs to the target subspace, the posterior contracts at the parametric rate $\kappa_j\approx 1$3 in empirical $\kappa_j\approx 1$4; when it does not, the posterior contracts at the near-minimax nonparametric rate $\kappa_j\approx 1$5, under the paper’s spline-basis assumptions (Shin et al., 2016). In additive models, independent component-wise fHS priors yield a grouped function-selection mechanism through the posterior of the shrinkage factors $\kappa_j\approx 1$6, and the paper reports smaller MSE and stronger variable-selection performance than standard horseshoe priors on spline coefficients in several settings (Shin et al., 2016).

A later nonparametric study places the horseshoe inside the broader class of oversmoothed heavy-tailed (OT) priors (Agapiou et al., 21 May 2025). In the Gaussian white noise model, the coefficient prior has the form

$\kappa_j\approx 1$7

with $\kappa_j\approx 1$8 drawn from a heavy-tailed density satisfying mild regularity assumptions; horseshoe is covered by taking $\kappa_j\approx 1$9 so that $\kappa\sim \mathrm{Beta}(1/2,1/2)$0 (Agapiou et al., 21 May 2025). The paper proves that such OT priors, including horseshoe OT priors, achieve near-minimax $\kappa\sim \mathrm{Beta}(1/2,1/2)$1 posterior contraction over Sobolev balls up to logarithmic factors, and adaptive near-minimax contraction over Besov balls under $\kappa\sim \mathrm{Beta}(1/2,1/2)$2-loss, including the sparse zone, when combined with suitable deterministic oversmoothing scales (Agapiou et al., 21 May 2025). It also proves that polynomial deterministic scales are insufficient for full adaptation, thereby identifying oversmoothed deterministic scaling rather than heavy tails alone as the decisive ingredient (Agapiou et al., 21 May 2025).

The nonlinear subspace-shrinkage extension NLFS carries these ideas into nonlinear function classes such as Hill models (Duda et al., 2024). Here the horseshoe-family prior controls the squared distance

$\kappa\sim \mathrm{Beta}(1/2,1/2)$3

between a flexible spline fit and the local tangent space of the nonlinear family (Duda et al., 2024). Empirically, the shrinkage factor $\kappa\sim \mathrm{Beta}(1/2,1/2)$4 is close to 1 when the target family is correct and near 0 when it is misspecified, making the prior act as a continuous adaptive model-selection device between shape-constrained and unconstrained fits (Duda et al., 2024).

This function-space literature broadens the meaning of “horseshoe family.” In these formulations the prior is no longer about exact or nearly exact zero coordinates; instead it is about sparse departure from a low-dimensional function class.

6. Applications, testing, and decision-theoretic developments

The horseshoe family has become a general-purpose prior mechanism across machine learning and applied Bayesian modeling.

In Bayesian neural networks, a group-wise horseshoe prior is placed on the incoming weights of each hidden unit: $\kappa\sim \mathrm{Beta}(1/2,1/2)$5 making entire hidden units shrink toward inactivity when their shared scale collapses (Ghosh et al., 2017). The paper reports that one can over-specify network width and let the prior prune irrelevant units without loss of predictive performance, provided non-centered parameterization is used (Ghosh et al., 2017).

For default Bayesian analysis of nonlinear functionals in high-dimensional normal means, horseshoe and horseshoe+ are presented as proper regularly varying priors that avoid Efron’s transformed-prior pathologies of diffuse Gaussian priors (Bhadra et al., 2015). In the sum-of-squares example $\kappa\sim \mathrm{Beta}(1/2,1/2)$6, replacing a fixed Gaussian variance by a half-Cauchy global scale yields an induced prior

$\kappa\sim \mathrm{Beta}(1/2,1/2)$7

which no longer pushes mass away from the origin as strongly as the $\kappa\sim \mathrm{Beta}(1/2,1/2)$8 law induced by a vague Gaussian prior (Bhadra et al., 2015). The broader claim is that regular variation makes horseshoe-family priors suitable as default priors for low-dimensional nonlinear functionals embedded in high-dimensional noise (Bhadra et al., 2015).

Multiple testing has prompted a distinct decision-theoretic branch. Because continuous shrinkage priors do not induce exact zeros, they do not directly produce posterior inclusion probabilities. One 2026 paper proposes posterior sign-based summaries

$\kappa\sim \mathrm{Beta}(1/2,1/2)$9

and tail-conditioned analogues $\pi_H(\theta)$0, as continuous-prior surrogates for $\pi_H(\theta)$1-values and $\pi_H(\theta)$2-values (Banerjee et al., 29 Jun 2026). In the sparse normal means model with empirical-Bayes horseshoe, the $\pi_H(\theta)$3-value rule attains the exact minimax combined risk $\pi_H(\theta)$4, while the $\pi_H(\theta)$5-value rule achieves asymptotic FDR control at target level $\pi_H(\theta)$6 together with asymptotically optimal FNR (Banerjee et al., 29 Jun 2026).

A 2025 paper attacks the same problem from a different angle through the frequentist-assisted horseshoe (FAHS) (Liang et al., 8 Feb 2025). There the canonical horseshoe is retained locally, but the global parameter is calibrated using a Benjamini–Hochberg estimate of the number of discoveries: $\pi_H(\theta)$7 depending on the variant (Liang et al., 8 Feb 2025). The rest of the procedure remains standard horseshoe posterior thresholding. The stated goal is finite-sample FDR control in high-dimensional testing while preserving the horseshoe’s heavy-tailed local shrinkage (Liang et al., 8 Feb 2025).

Predictive inference has also entered the horseshoe literature. In sparse Gaussian sequence prediction, one paper proves exact asymptotic minimaxity of horseshoe predictive densities under Kullback–Leibler loss when the sparsity level is known and $\pi_H(\theta)$8 is set at oracle order $\pi_H(\theta)$9, $$\frac{K}{2}\log\!\left(1+\frac{4}{\theta^2}\right) < \pi_H(\theta) < K\log\!\left(1+\frac{2}{\theta^2}\right),\qquad K=(2\pi^3)^{-1/2},$$0 (Zhai et al., 17 Apr 2026). It introduces a Gaussian-mixture representation of the posterior predictive density—termed “Horseshoe spectroscopy”—in which the local-scale posterior exhibits a phase transition at

$$\frac{K}{2}\log\!\left(1+\frac{4}{\theta^2}\right) < \pi_H(\theta) < K\log\!\left(1+\frac{2}{\theta^2}\right),\qquad K=(2\pi^3)^{-1/2},$$1

producing predictive switching behavior between strongly shrunk and essentially unshrunk Gaussian predictive components (Zhai et al., 17 Apr 2026). A hierarchical version with $$\frac{K}{2}\log\!\left(1+\frac{4}{\theta^2}\right) < \pi_H(\theta) < K\log\!\left(1+\frac{2}{\theta^2}\right),\qquad K=(2\pi^3)^{-1/2},$$2 is shown to adapt to effective sparsity under a theta-min condition (Zhai et al., 17 Apr 2026).

Across these domains, the practical appeal of the horseshoe family is remarkably consistent: one obtains continuous shrinkage with strong noise suppression, robust handling of large signals, and latent Gaussian structure enabling scalable inference.

7. Conceptual synthesis and unresolved issues

Several broad themes recur across the literature.

First, the family’s central mechanism is stable across formulations: a globally controlled degree of shrinkage, local adaptivity, and heavy tails. Whether the object being shrunk is a regression coefficient, a hidden unit, a finite difference, a GP amplitude, or a function-space deviation, the design logic is the same. The horseshoe prior acts as a continuous alternative to explicit model selection, often mimicking spike-and-slab behavior without discrete latent indicators (Ghosh et al., 2015, Ghosh et al., 2017).

Second, the choice of global scale is decisive. Sparse asymptotics in normal means suggest $$\frac{K}{2}\log\!\left(1+\frac{4}{\theta^2}\right) < \pi_H(\theta) < K\log\!\left(1+\frac{2}{\theta^2}\right),\qquad K=(2\pi^3)^{-1/2},$$3 should be of order $$\frac{K}{2}\log\!\left(1+\frac{4}{\theta^2}\right) < \pi_H(\theta) < K\log\!\left(1+\frac{2}{\theta^2}\right),\qquad K=(2\pi^3)^{-1/2},$$4 or $$\frac{K}{2}\log\!\left(1+\frac{4}{\theta^2}\right) < \pi_H(\theta) < K\log\!\left(1+\frac{2}{\theta^2}\right),\qquad K=(2\pi^3)^{-1/2},$$5 (Pas et al., 2014, Ghosh et al., 2015). Piironen and Vehtari’s effective-nonzero-parameter analysis provides a practical calibration principle in regression (Piironen et al., 2017). Multiple-testing work shows that good testing performance hinges on getting the global scale right, either via empirical Bayes concentration theory or explicit frequentist calibration (Banerjee et al., 29 Jun 2026, Liang et al., 8 Feb 2025).

Third, the canonical horseshoe is not the only prior in the family, and not always the preferred one. The Heavy-tailed Horseshoe sharpens both the spike and the slab for ultra-sparse problems (Womack et al., 2019). The regularized horseshoe stabilizes weakly identified likelihoods by imposing finite slab width (Piironen et al., 2017). Functional and nonlinear functional horseshoes redirect shrinkage from coefficients to structured function-space deviations (Shin et al., 2016, Duda et al., 2024). Application-specific horseshoe-inspired priors, such as the GP-amplitude shrinkage in semi-parametric Bayesian networks, may depart from the canonical local–global hierarchy while still exploiting the family’s heavy-tail/near-zero geometry (Weng et al., 2024).

Fourth, several limitations remain explicit in the literature. Some implementations called “horseshoe” are only horseshoe-inspired and omit the full local–global hierarchy, latent-scale posterior inference, or regularized slab components (Weng et al., 2024). In nonparametric settings, heavy tails alone are insufficient for full adaptation; deterministic oversmoothing scales can be essential (Agapiou et al., 21 May 2025). In hierarchical models, inference can be difficult because of funnel-like posterior geometry, making non-centered parameterizations or auxiliary-variable schemes important (Ghosh et al., 2017, Piironen et al., 2017). For multiple testing and predictive adaptation, the strongest guarantees often require idealized sparse-sequence assumptions or theta-min conditions (Banerjee et al., 29 Jun 2026, Zhai et al., 17 Apr 2026).

Taken together, these works define the horseshoe family less as a single distribution than as a coherent statistical architecture. Its canonical member is the half-Cauchy global–local prior on Gaussian variances, but the family now includes heavier-tailed, regularized, grouped, structured, functional, and nonlinear-subspace variants. What unifies them is the attempt to place very high prior mass near a low-complexity configuration while leaving enough tail mass for substantial departures. In sparse estimation, this low-complexity object is zero; in fusion, it is equality of differences; in inverse problems, it is piecewise constancy; in functional shrinkage, it is membership in a subspace or nonlinear model class. The horseshoe family’s continuing significance lies in how broadly that principle has proved portable.