Dirichlet–Laplace Priors in Sparse Inference

• Dirichlet–Laplace priors are global–local shrinkage methods designed for high-dimensional sparse Bayesian inference, combining simplex-constrained weights with Laplace mixtures.
• They achieve optimal posterior contraction rates and robust variable selection in regression and normal means models via conjugate-friendly Gibbs sampling.
• Practical implementations use corrected sampling algorithms and alternative parameterizations to enhance computational efficiency and accurate uncertainty quantification.

The Dirichlet–Laplace (DL) prior is a global–local shrinkage prior designed for high-dimensional Bayesian inference under sparsity. It combines a simplex-constrained vector of local scales (Dirichlet weights) with Laplace or Gaussian scale mixtures, producing pronounced shrinkage near zero while retaining heavy tails for large signals. The DL prior delivers optimal posterior contraction rates, efficient computation via conjugate-friendly Gibbs sampling, and robust variable selection in regression and normal means models. Theoretical advances and practical implementation strategies clarify its analytic properties and correct subtle errors in earlier simulation algorithms.

1. Hierarchical Model and Marginal Representation

Consider the normal means model, $y_i = \theta_i + \varepsilon_i$, $\varepsilon_i \sim N(0,1)$, for $i = 1, \ldots, n$. The DL prior introduces three layers of latent variables:

• Local scale parameters: $\psi_j \sim \mathrm{Exp}(1/2)$ for $j = 1, \ldots, n$.
• Simplex-constrained weights: $$\phi = (\phi_1, \ldots, \phi_n) \sim \mathrm{Dirichlet}(a, \ldots, a)$$.
• Global scale: $\tau \sim \mathrm{Gamma}(n a, 1/2)$.

The hierarchical prior for $\theta_j$ is:

$$\theta_j | \psi_j, \phi_j, \tau \sim N(0, \psi_j \phi_j^2 \tau^2).$$

Integrating over $\psi_j$ produces $DE(0, \phi_j \tau)$ double-exponential (Laplace) marginals. Further marginalization over $\phi$ and $\tau$ yields a density for $\theta_j$ sharply peaked at zero and with exponential or polynomial tails, depending on $a$ and the integration over Dirichlet and Gamma components (Bhattacharya et al., 2014, Bhattacharya et al., 2012).

An alternative parameterization replaces Dirichlet and Gamma with independent Gamma variables: letting $\lambda_j = \phi_j \tau$, $\lambda_j \sim \mathrm{Gamma}(a, 1/2)$, so

$$\theta_j | \psi_j, \lambda_j \sim N(0, \psi_j \lambda_j^2).$$

This version facilitates implementation and eliminates redundancy (Gruber et al., 16 Aug 2025).

2. Global–Local Shrinkage Mechanism

DL priors effect shrinkage by combining a sum-to-one constraint on local weights (the simplex) with a global scale. This architecture produces "joint shrinkage", suppressing all but a few signals via the spike at zero. For very sparse signals (e.g., under $a = 1/n$), the DL prior matches the concentration properties of discrete spike-and-slab mixtures, yet retains computational tractability (Bhattacharya et al., 2012).

Comparative properties:

• Laplace (Bayesian Lasso): Exponential peak at zero, lighter tails — tends to over-shrink large coefficients.
• Horseshoe: Heavy tails ($1/\beta^2$) but weaker central spike.
• Dirichlet–Laplace: Adjustable spike via $a$ (small $a$ gives near point-mass), exponential-to-Pareto tails (Zhang et al., 2016).

3. Theoretical Guarantees and Posterior Concentration

DL priors achieve minimax posterior contraction rates in sparse high-dimensional settings. In the normal means model, with $q_n$ true nonzero coordinates in $\theta^0$, set $s_n = q_n \log(n/q_n)$. Under $a_n = n^{-(1+\delta)}$ (typically $a = 1/n$), the posterior contracts at rate $s_n$:

$$\mathbb{E}_{\theta^0} \left[ \Pi\left( \|\theta - \theta^0\|_2^2 \leq M s_n \mid y \right) \right] \rightarrow 1 \quad \text{as} \ n \rightarrow \infty,$$

provided $\|\theta^0\|_2^2 \leq q_n (\log n)^4$ (Bhattacharya et al., 2014). The induced support size $|\mathrm{supp}_{\epsilon_n}(\theta)|$ for $\epsilon_n = s_n/n$ remains proportional to $q_n$, avoiding over-selection of spurious features.

In linear regression ($p_n \to \infty$), consistent posterior contraction and selection hold under $q_n = o(n/\log n)$ and suitable shrinking $a_n$ (Zhang et al., 2016).

4. Posterior Computation: Gibbs Samplers and Algorithmic Corrections

DL posterior draws are computable via blocked Gibbs sampling exploiting the scale-mixture Gaussian and normalized random measure identities. Key update steps:

• $\theta$ update: Jointly normal, conditionally independent given $\psi_j,\phi_j,\tau$ or $\psi_j,\lambda_j$.
• Local scale update: $\psi_j \mid \theta_j, \phi_j, \tau$ sampled as inverse-Gaussian.
• Global scale update: $\tau \mid \phi,\theta$ as generalized inverse-Gaussian.
• Simplex weights update: $\phi_j$ via normalized random measure draws, i.e., $T_j \sim \mathrm{giG}(a-1, 1, 2|\theta_j|)$, $\phi_j = T_j/\sum_k T_k$ (Bhattacharya et al., 2012, Gruber et al., 16 Aug 2025).

An error in the original Bhattacharya et al. (2015) sampler concerned the order of updates: sampling $\psi$ before $(\phi, \tau)$, and using conditionals that had already integrated out $\psi$, violates the marginal-conditional factorization and leads to incorrect stationary distributions (Gruber et al., 16 Aug 2025, Onorati et al., 7 Jul 2025). Corrected algorithms respect the sequence $\phi \rightarrow \tau \rightarrow \psi$ within blocks (or use the $\lambda_j$ parameterization for independent updates).

Per iteration cost is $O(n)$, dominated by fast inverse-Gaussian and GIG draws. Mixing is efficient, but global shrinkage parameters (e.g., $\tau$, $\sum \lambda_j$) require diagnostic monitoring.

5. Applications in Regression, Variable Selection, and Performance

The DL prior supports high-dimensional linear regression, sparse estimation, and variable selection tasks. In regression,

$$Y = X\beta + \varepsilon, \quad \varepsilon \sim N(0, \sigma^2 I_n)$$

assigns each coefficient $\beta_j$ a DL prior, facilitating sparse recovery via penalized credible regions:

• Compute joint posterior credible ellipsoids;
• Find the sparsest $\beta$ in the region via $L_0$ minimization or penalization;
• Hyperparameters (e.g., Dirichlet $a$) can be objectively tuned by minimizing discrepancy between induced $R^2$ distribution and a target Beta prior (Zhang et al., 2016).

Simulation and microarray analyses confirm that DL priors outperform Bayesian Lasso, Laplace, and horseshoe alternatives in terms of mean squared error, selection support recovery, and interpretability, especially in settings where signals are sparse and correlations moderate to high (Bhattacharya et al., 2014, Zhang et al., 2016).

6. Practical Recommendations and Implementation Notes

For efficient and correct computation:

• Always use the corrected Gibbs sampler or the simplified $\lambda$-based scheme to correctly target the posterior (Gruber et al., 16 Aug 2025, Onorati et al., 7 Jul 2025).
• Employ specialized C/C++ routines for GIG and inverse-Gaussian draws.
• Monitor mixing of global shrinkage parameters; consider re-parameterizations or thinning to improve autocorrelations.
• For $a \ll 1$, e.g., $a = 1/n$, numerical instabilities may arise; the alternative $\lambda$-parametrization tends to be more robust.
• Parallelize coordinatewise updates for large $n$.
• Validate implementation by “Getting it right” tests or by comparison with small-$n$ marginal samplers.

An R package “DirLapl” implements the corrected sampler (Onorati et al., 7 Jul 2025).

7. Impact and Current Best Practices

DL priors blend computational tractability, theoretical optimality under sparsity, and flexibility via global–local architecture. Best practices include setting $a = 1/n$ or $1/p$, using gamma global scales for closed-form updates, and validating MCMC chains against the target posterior (Bhattacharya et al., 2014, Gruber et al., 16 Aug 2025).

When embedded in regression or penalized credible region frameworks, DL priors yield consistent support recovery, improved prediction error, and are applicable to massive-dimensional data, including genetics and microarray analyses (Zhang et al., 2016). Theoretical results and corrected algorithms ensure reliability of uncertainty quantification and practical recommendations for robust variable selection.

A plausible implication is that, owing to the spike-at-zero and heavy tails, DL priors should be preferred in high-dimensional sparse modeling where both accurate zero selection and minimal shrinkage of true signals are required. All substantive theoretical properties and large-sample guarantees remain valid under corrected samplers (Gruber et al., 16 Aug 2025, Onorati et al., 7 Jul 2025).