Dirichlet-Laplace Prior in Sparse Bayesian Models

• The Dirichlet-Laplace prior is a continuous Bayesian shrinkage prior that combines a Laplace kernel with a Dirichlet-constrained allocation of local scales to promote joint sparsity in high-dimensional parameter vectors.
• It employs a normal-scale mixture representation and latent variables, enabling efficient Gibbs sampling with corrected blocking strategies for accurate posterior computation.
• The DL prior achieves minimax posterior contraction and supports variable selection in sparse normal-means and regression settings by balancing global and local shrinkage through its hyperparameter choices.

Searching arXiv for recent and foundational papers on the Dirichlet-Laplace prior. The Dirichlet–Laplace prior is a continuous Bayesian shrinkage prior for high-dimensional sparse parameter vectors. It combines a Laplace kernel with a Dirichlet simplex of coefficient-specific scale fractions and a global scale, thereby inducing a form of coordinated shrinkage across coordinates that is designed to concentrate strongly around sparse vectors while retaining conjugacy-friendly latent-variable representations for posterior computation (Bhattacharya et al., 2014). In the normal-means and regression settings, it is typically written either in a Laplace form,

$$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau), \qquad \phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$

or in an equivalent Gaussian scale-mixture form with exponential local mixing variables. The prior was introduced to address a specific deficiency of many standard continuous shrinkage priors: acceptable univariate shrinkage behavior does not by itself guarantee strong joint prior concentration around sparse vectors in $\mathbb{R}^n$ (Bhattacharya et al., 2012).

1. Definition and hierarchical structure

The DL prior arises as a specialization of a broader kernel-Dirichlet construction in which

$$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$

with $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$. Specializing $\mathcal K$ to the Laplace kernel yields

$$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$

where $\mathrm{DE}(\tau)$ denotes the mean-zero double-exponential density

$$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$

The fully Bayesian specification assigns

$$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$

and is denoted $\theta \sim \mathrm{DL}_a$ in the original development (Bhattacharya et al., 2014).

An equivalent normal-mixture representation follows from the standard Laplace-as-normal-exponential mixture identity: $\mathbb{R}^n$0 This representation places the DL prior within the global-local shrinkage family, but with a distinctive dependence structure: the effective coordinate scale is $\mathbb{R}^n$1, and the simplex constraint forces the $\mathbb{R}^n$2 to compete for a finite total scale budget (Bhattacharya et al., 2012).

A further equivalent parameterization is obtained by defining $\mathbb{R}^n$3 or $\mathbb{R}^n$4. Under the stated hierarchy,

$\mathbb{R}^n$5

independently, and the prior can be rewritten as

$\mathbb{R}^n$6

This reparameterization removes the explicit simplex constraint from posterior simulation while preserving the same prior law on $\mathbb{R}^n$7 (Onorati et al., 7 Jul 2025).

The hyperparameter $\mathbb{R}^n$8 is the Dirichlet concentration parameter. Small $\mathbb{R}^n$9 concentrates $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$0 near the corners of the simplex, so a few coordinates receive relatively large scale mass while many receive very small scales. In the original formulation, $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$1 is singled out theoretically, while $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$2 is recommended as a robust default in settings where many small nonzero effects may be present (Bhattacharya et al., 2012).

2. Sparsity mechanism and relation to other shrinkage priors

The central innovation of the DL prior is not the Laplace kernel itself, but the simplex-constrained allocation of scale across coordinates. Standard global-local priors use independent local scales $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$3, whereas the DL prior introduces a Dirichlet vector $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$4 satisfying $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$5. This creates a budget constraint: if some coordinates receive large $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$6, many others must receive tiny $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$7. The resulting shrinkage is therefore joint rather than merely marginal (Bhattacharya et al., 2014).

This simplex mechanism was introduced to approximate spike-and-slab behavior without a point mass at zero. Spike-and-slab priors can achieve minimax-optimal posterior contraction, but their posterior computation requires exploration over a model space of size $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$8. The DL prior remains fully continuous and avoids discrete subset search, yet is constructed to place much more mass near sparse vectors than ordinary i.i.d. Laplace or many other continuous shrinkage priors (Bhattacharya et al., 2012).

Relative to the Bayesian lasso, the distinction is structural rather than cosmetic. The Bayesian lasso uses Laplace marginals induced by independent exponential local scales; the DL prior uses Laplace kernels modulated by simplex-normalized local scale weights. The original papers argue that this difference changes both the marginal behavior near zero and the joint geometry of the prior on $$\theta_j \mid \phi_j,\tau \sim \mathcal K(\cdot\,;\phi_j\tau), \qquad \phi=(\phi_1,\dots,\phi_n)\in \mathcal S^{n-1}, \qquad \phi\sim \mathrm{Dir}(a,\dots,a),$$9, and that the dependent scales $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$0 are what produce the large improvement in concentration around sparse vectors (Bhattacharya et al., 2012).

Relative to the horseshoe and related heavy-tailed global-local priors, the DL prior remains a global-local prior but with dependent local scales. The horseshoe is described positively in the original work and conjectured there to have optimal or near-optimal contraction on a broad domain, but the DL prior differs in that its sparsity mechanism is not solely a consequence of marginal singularity at zero or heavy tails; it is driven by the joint simplex constraint on scale allocation (Bhattacharya et al., 2014).

3. Marginal behavior, prior concentration, and asymptotic theory

The DL prior induces a singularity at zero in the marginal density of each coordinate after integrating out the Dirichlet weights. If $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$1 and $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$2, then the marginal distribution of $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$3 is unbounded with a singularity at zero for any $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$4. In the special case $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$5, the marginal is a wrapped Gamma distribution with density proportional to

$$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$6

Thus the prior combines an infinite spike at zero with exponential tails (Bhattacharya et al., 2012).

In the fully Bayesian DL prior, the marginal density of a coordinate can be written as

$$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$7

where $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$8 is the modified Bessel function of the second kind. This representation is used in the theoretical analysis of small-ball probabilities and posterior contraction (Bhattacharya et al., 2014).

The motivating theoretical claim is that many standard continuous shrinkage priors have poor joint concentration around sparse vectors. For sparse $$\mathcal S^{n-1}=\{\phi_j\ge 0,\ \sum_{j=1}^n \phi_j=1\}$$9, the original analysis shows that global-only priors and Bayesian-lasso-type priors can assign exponentially small mass to neighborhoods of sparse signals, leading to suboptimal posterior contraction. By contrast, under the DL prior with $\mathcal K$0, if $\mathcal K$1 has one nonzero entry and $\mathcal K$2, then

$\mathcal K$3

which is substantially larger than the $\mathcal K$4 upper bound established there for the Bayesian lasso (Bhattacharya et al., 2012).

The flagship asymptotic result is a minimax posterior contraction theorem in the sparse normal-means model. If

$\mathcal K$5

for small $\mathcal K$6, and if

$\mathcal K$7

then for some constant $\mathcal K$8,

$\mathcal K$9

If $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$0, the same conclusion holds provided $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$1 (Bhattacharya et al., 2014).

A related result establishes posterior compressibility. Since continuous priors do not produce exact zeros, the approximate support is defined by

$$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$2

For $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$3, the posterior places asymptotically negligible mass on vectors with more than $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$4 practically nonzero coordinates for some $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$5, under the same normal-means conditions used for contraction (Bhattacharya et al., 2014). This formalizes the idea that the posterior under the DL prior remains approximately low-dimensional even without exact sparsity.

4. Posterior computation and the correction of the original Gibbs sampler

The original DL papers proposed a Gibbs sampler based on the hierarchy

$$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$6

cycling through updates of $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$7, $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$8, $$\theta_j \mid \phi_j,\tau \sim \mathrm{DE}(\phi_j\tau),$$9, and $\mathrm{DE}(\tau)$0 (Bhattacharya et al., 2014). Later work showed that the published ordering of the blocked latent-variable updates was incorrect: it sampled from conditional distributions in the wrong order and therefore did not, in general, target the intended posterior distribution (Gruber et al., 16 Aug 2025).

The error is a blocking issue rather than a mistake in isolated full-conditional formulas. In the normal-means setting, the relevant factorization is

$\mathrm{DE}(\tau)$1

Hence a valid blocked Gibbs draw must sample in the order

$\mathrm{DE}(\tau)$2

Sampling instead in the order $\mathrm{DE}(\tau)$3, then $\mathrm{DE}(\tau)$4, then $\mathrm{DE}(\tau)$5, while conditioning on stale values, changes the transition kernel. The corrected papers state explicitly that the stationary distribution $\mathrm{DE}(\tau)$6 of the flawed algorithm satisfies

$\mathrm{DE}(\tau)$7

and therefore does not preserve posterior invariance (Onorati et al., 7 Jul 2025).

A preferred correction is to eliminate the redundant $\mathrm{DE}(\tau)$8 parameterization and work directly with $\mathrm{DE}(\tau)$9, where $$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$0 independently. In the normal-means model

$$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$1

the corrected sampler uses

$$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$2

$$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$3

and, with $$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$4,

$$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$5

The same redundancy-free strategy extends to Gaussian linear regression, with the $$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$6 and $$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$7 updates scaled by $$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$8 (Onorati et al., 7 Jul 2025).

The practical consequences are substantive. Later simulation studies reported that the incorrect sampler can over-shrink nonzero signals and distort the posterior shape for truly zero coefficients. In the normal-means experiments summarized there, the corrected algorithm uniformly outperformed the original one when $$f(y)=\frac{1}{2\tau}e^{-|y|/\tau}, \qquad y\in\mathbb R.$$9, and the earlier impression that $$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$0 “over-shrinks” was argued to be an artifact of the incorrect sampler (Onorati et al., 7 Jul 2025). The corrigendum emphasized that the prior itself and the original theoretical properties remain valid; the defect concerns posterior simulation only (Gruber et al., 16 Aug 2025).

5. Regression formulations and variable selection

The DL prior was adapted from normal means to Gaussian linear regression in work on Bayesian variable selection via penalized credible regions. In that setting,

$$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$1

and the DL prior is placed on the regression coefficients as

$$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$2

with

$$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$3

The equivalent normal-mixture representation is

$$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$4

which yields conditionally conjugate Gaussian updates for $$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$5 (Zhang et al., 2016).

This regression formulation was used inside a two-stage variable-selection framework. First, the full model is fitted under a continuous shrinkage prior. Second, one searches for the sparsest coefficient vector inside a joint posterior credible region. With posterior mean $$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$6 and covariance $$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$7, the credible set is taken as

$$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$8

and the sparsified estimator is obtained by minimizing

$$\phi \sim \mathrm{Dir}(a,\dots,a), \qquad \tau \sim \mathrm{gamma}(na,1/2),$$9

In this approach, the DL prior is used for model fitting and posterior uncertainty quantification, while exact sparsification is produced by the penalized credible-region search rather than by the prior itself (Zhang et al., 2016).

The asymptotic results in that paper distinguish posterior consistency from selection consistency. Under assumptions including $\theta \sim \mathrm{DL}_a$0, bounded eigenvalues of $\theta \sim \mathrm{DL}_a$1, bounded true coefficients, and a sparsity regime $\theta \sim \mathrm{DL}_a$2, the DL prior yields a consistent posterior if

$\theta \sim \mathrm{DL}_a$3

for finite $\theta \sim \mathrm{DL}_a$4 and finite $\theta \sim \mathrm{DL}_a$5. Under additional minimum-signal assumptions and

$\theta \sim \mathrm{DL}_a$6

the penalized credible-region procedure is selection consistent: $\theta \sim \mathrm{DL}_a$7 (Zhang et al., 2016).

The same work also proposed prior elicitation through the induced distribution of $\theta \sim \mathrm{DL}_a$8. Writing $\theta \sim \mathrm{DL}_a$9, the prior-implied coefficient of determination is

$\mathbb{R}^n$00

The proposed strategy is to choose hyperparameters so that the induced prior on $\mathbb{R}^n$01 is close to a target distribution, typically $\mathbb{R}^n$02. For the normal prior a closed-form Kullback–Leibler solution is derived; for the DL prior the recommendation is simulation-based tuning over candidate values of $\mathbb{R}^n$03 (Zhang et al., 2016).

6. Hyperparameter choice, limitations, and common conceptual confusions

The most consequential tuning parameter is the Dirichlet concentration $\mathbb{R}^n$04. In the original DL theory, $\mathbb{R}^n$05 is the canonical sparse choice and underpins the strongest concentration results. At the same time, simulation studies in the original work reported that $\mathbb{R}^n$06 can over-shrink when there are many relatively small signals, and recommended $\mathbb{R}^n$07 as a robust default choice (Bhattacharya et al., 2012). Later computational work qualifies that conclusion: it argues that earlier impressions of excessive shrinkage at $\mathbb{R}^n$08 were partly artifacts of the incorrect Gibbs sampler, and that with the corrected sampler $\mathbb{R}^n$09 often performs best in the normal-means experiments reported there (Onorati et al., 7 Jul 2025). The tension is therefore not a contradiction in the prior definition but a combination of genuine shrinkage trade-offs and previously flawed posterior computation.

The theoretical scope of the strongest original DL results is also specific. The minimax contraction and posterior compressibility theorems are proved in the sparse normal-means setting, while the regression consistency results in the penalized credible-region framework require $\mathbb{R}^n$10 asymptotically rather than ultra-high-dimensional $\mathbb{R}^n$11 regimes (Bhattacharya et al., 2014). This suggests that the DL prior is theoretically mature in some sparse models, but its full asymptotic behavior outside those settings was not established in the cited works.

A recurring source of confusion is terminological. The classical Dirichlet distribution is also prominent in Bayesian statistics, but in a wholly different role. In discrete Bayesian-network learning, a separate characterization theorem shows that under multinomial sampling, global and local parameter independence, parameter modularity, and hypothesis equivalence, the only positive prior over multinomial parameters is Dirichlet (Geiger et al., 2013). That result concerns the classical Dirichlet prior on multinomial probabilities. It does not discuss the modern Dirichlet–Laplace prior, does not involve Laplace mixing or global-local shrinkage, and is conceptually unrelated to high-dimensional sparse regression.

The DL prior is therefore best understood as a specific continuous sparsity prior: a Laplace-kernel global-local model in which the local scales are not independent but are coordinated through a Dirichlet simplex. Its defining feature is that dependence among local scales is used to encode sparsity as a joint geometric property of the prior rather than as a purely marginal phenomenon (Bhattacharya et al., 2012).