Dependent Dirichlet Priors

• Dependent Dirichlet priors are a generalization of the Dirichlet process that introduce structured dependence across probability measures using covariate-indexed processes, latent hierarchies, and normalized CRMs.
• They enable flexible Bayesian hierarchical modeling and nonparametric inference in applications like dynamic clustering, belief networks, and graphical models while retaining tractable marginal distributions.
• Advanced inference techniques such as blocked MCMC, slice-sampling, and optimal linear estimators are employed to efficiently compute posterior distributions under these complex, correlated structures.

Dependent Dirichlet priors generalize the classical Dirichlet distribution and Dirichlet process by introducing statistical dependence among collections of marginally Dirichlet measures or vectors. This construction enables flexible Bayesian hierarchical modeling, nonparametric modeling of partially exchangeable structures, and efficient information sharing in multivariate and structured probability models. Key applications include dependent mixture models, dynamic clustering, belief network parameterization, and graphical models. By embedding dependence through covariate-indexed stick-breaking, latent processes, normalization of dependent completely random measures, or coordinated hyperparameterization, these priors retain tractable marginals while enabling complex correlation structures across measures.

1. Foundations and Core Constructions

Dependent Dirichlet priors emerge when the independence assumption between Dirichlet-distributed random measures or probability vectors is relaxed, resulting in joint laws with specified marginal Dirichlet distributions and positive cross-correlation. A canonical general form is as follows. Let $\{G_x : x \in \mathcal{X}\}$ denote a family of random probability measures indexed by covariate $x$, with each $G_x \sim \mathrm{DP}(\alpha, G_0)$ marginally, but the collection $\{G_x\}$ is dependent in a specified manner (Foti et al., 2012).

Principal constructions include:

• Covariate-indexed stick-breaking: The stick-breaking weights $V_k(x)$ are modeled as dependent stochastic processes (e.g., Gaussian processes mapped to Beta marginals), yielding MacEachern's multiple-p DDP (Foti et al., 2012). For each $x$,

$$G_x = \sum_{k=1}^\infty \pi_k(x) \delta_{\theta_k}, \quad \pi_k(x) = V_k(x) \prod_{j<k}(1 - V_j(x)),$$

with $V_k(x) \sim \mathrm{Beta}(1, \alpha)$ for each $k, x$ and dependence across $x$ controlled by the process' kernel.

• Latent process constructions: Dependence is induced by hierarchical models involving latent Dirichlet and multinomial processes. For a dependent Dirichlet process (DDP) indexed by $x$0,

$x$1

where $x$2 encodes neighborhood structure (Nieto-Barajas, 2021).

• Normalization of dependent CRMs: Dependent Dirichlet processes arise by normalizing dependent gamma or $x$3-stable completely random measures, with dependence introduced via the underlying Poisson random measures (Lijoi et al., 2014). In the bivariate Dirichlet process, atoms split into "common" and "idiosyncratic" via mass partitioning, yielding controllable marginal and joint behavior.
• Mixture and graphical models: Dirichlet-type priors associated with graphical model structures (G-Dirichlet), belief network kernels (dependent Dirichlet/multiplicative DD priors), or pairwise mixture models (PDDP, PDGSBP) provide additional context-specific dependence mechanisms (Danielewska et al., 2023, Hooper, 2012, Hatjispyros et al., 2017).

2. Parametrizations: Stick-breaking, Latent, and Graphical

Mechanisms for introducing dependence include:

• Gaussian Process/Copula Stick-breaking: Via $x$4, set $x$5 (Foti et al., 2012, Iorio et al., 2019). Autoregressive copula constructions (e.g., time-indexed AR(1) Gaussian processes mapped through a Beta inverse CDF) provide explicit control over temporal or spatial dependence:

$x$6

with $x$7 for $x$8 the standard normal CDF (Iorio et al., 2019).

• Latent multinomial hierarchies: Each local random measure $x$9 borrows information from its neighbors’ latent multinomial counts, inducing partial exchangeability and allowing tunable correlation via parameter $G_x \sim \mathrm{DP}(\alpha, G_0)$0. The structure accommodates time series, spatial, and graph-structured models (Nieto-Barajas, 2021).
• Beta-Binomial stick-breaking dependence: Introducing Markov structure in the stick-breaking variables, e.g., the Beta-Binomial stick-breaking (BBSB) process, where $G_x \sim \mathrm{DP}(\alpha, G_0)$1, with correlation $G_x \sim \mathrm{DP}(\alpha, G_0)$2 (Gil-Leyva et al., 2019).
• Graphical neutrality: In G-Dirichlet priors, the factorization coordinates $G_x \sim \mathrm{DP}(\alpha, G_0)$3 (derived from separating the clique and separator structure of a graph $G_x \sim \mathrm{DP}(\alpha, G_0)$4) are mutually independent Beta random variables, and the support is characterized by a positivity constraint on the clique polynomial of $G_x \sim \mathrm{DP}(\alpha, G_0)$5 (Danielewska et al., 2023).

3. Theoretical Properties and Dependence Quantification

Dependent Dirichlet priors are designed to retain marginals in the Dirichlet family while encoding specified correlation across measures.

• Marginals and Support: In all major models, $G_x \sim \mathrm{DP}(\alpha, G_0)$6 for each fixed $G_x \sim \mathrm{DP}(\alpha, G_0)$7. In discrete settings, G-Dirichlet laws reduce to product Beta or classic Dirichlet under extreme graph choices (Danielewska et al., 2023).
• Correlation Structures:
  • For latent multinomial processes, the correlation between $G_x \sim \mathrm{DP}(\alpha, G_0)$8 and $G_x \sim \mathrm{DP}(\alpha, G_0)$9 is

  $\{G_x\}$0

  (Nieto-Barajas, 2021). - For dependent CRMs with partition parameter $\{G_x\}$1, as $\{G_x\}$2 the processes become independent; as $\{G_x\}$3 all measures coincide (Lijoi et al., 2014).

• Partial Exchangeability: Generalizations of the Pólya urn and Blackwell–MacQueen schemes show that conditional distributions incorporate latent or common ancestors, leading to partially exchangeable arrays and tractable partition structures (Lijoi et al., 2014, Nieto-Barajas, 2021, Hooper, 2012).
• Inference and Conjugacy: Many constructions (latent multinomial, dependent CRMs, Fleming–Viot-driven processes) yield closed-form or finite-mixture posteriors, supporting analytic or tractable MCMC inference (Papaspiliopoulos et al., 2016, Lijoi et al., 2014, Nieto-Barajas, 2021). For some choices, only optimal linear estimators or approximations are available (e.g., dependent Dirichlet priors for belief network CP-tables) (Hooper, 2012).

4. Prominent Examples and Special Cases

Several notable specializations are mathematically and practically significant:

Prior Class	Marginal Law	Dependence Mechanism
Bivariate Dirichlet (CRM)	Dir$\{G_x\}$4	Mass splitting via Poisson process (parameter $\{G_x\}$5)
Beta–Binomial Stick-breaking	Dir$\{G_x\}$6	Markov chain in stick-breaks (parameter $\{G_x\}$7)
Latent Multinomial DDP	Dir$\{G_x\}$8	Shared anchor DP and latent multinomial counts
MacEachern DDP (Multiple-p)	Dir$\{G_x\}$9	Covariate-indexed Beta stick-breaks
G-Dirichlet Prior (Graphical)	Dir$V_k(x)$0	Graph-based factorization (cliques, separators)
Pairwise Dependent DP (PDDP)	Dir$V_k(x)$1	Symmetrized mixture of shared components
Dependent Dirichlet for CP-tables	Dir$V_k(x)$2	Overlapping Gamma components in parameterization

This diversity enables broad adaptation to time series (Judd et al., 13 Apr 2026, Iorio et al., 2019, Papaspiliopoulos et al., 2016), spatial data, belief networks (Hooper, 2012), graphical models (Danielewska et al., 2023), and partially exchangeable groupings (Hatjispyros et al., 2017, Nieto-Barajas, 2021).

5. Posterior Inference and Computation

The tractability of posterior inference varies with the construction:

• Finite mixture/analytic conjugacy: In simple latent or CRM-based models, posterior laws after seeing data reduce to a finite mixture of Dirichlet processes or vectors, with updated base measures and concentration parameters computed by "adding counts" from observations (Papaspiliopoulos et al., 2016, Nieto-Barajas, 2021, Lijoi et al., 2014). Filtering algorithms can be written recursively and facilitate exact sequential inference.
• Gibbs and Blocked Sampling: Blocked MCMC, slice-sampling, and particle filters are common for models with infinite-dimensional or function-valued weights (e.g., AR-copula DDP (Iorio et al., 2019), stick-breaking GPs (Foti et al., 2012)).
• Optimal Linear Estimators: For dependent Dirichlet priors in Bayesian networks, full conjugacy is lost, but minimum-MSE estimators among all linear combinations of neighboring CP-table proportions and prior means are available closed-form (Hooper, 2012).
• Graphical Models: Sampling under graphical Dirichlet priors leverages independent Beta local coordinates associated with DAGs/moral graphs, with forward–backward or elimination orderings providing both normalization and simulation schemes (Danielewska et al., 2023).

6. Applications and Practical Impact

Dependent Dirichlet priors enable models with dependence across time, space, or structured indices, with prominent applications including:

• Dynamic mixtures and clustering: Modeling data with smoothly evolving or abrupt changes in clustering, temporal birth–death of clusters, or nonstationary heterogeneity (Iorio et al., 2019, Papaspiliopoulos et al., 2016, Judd et al., 13 Apr 2026).
• Hierarchical random measures: Grouped and spatially dependent data, where local measures share a global component for partial exchangeability or shrinkage (Nieto-Barajas, 2021).
• Graphical and belief network parameterization: Efficiently sharing information across high-dimensional conditional probability tables, especially when data for some configurations is sparse (Hooper, 2012).
• Borrowing of strength and cross-group inference: Pairwise dependent measures support informative shrinkage and improved estimation in small-sample/high-dimension regimes (Hatjispyros et al., 2017, Lijoi et al., 2014).

Empirical evidence indicates that such priors provide substantive improvements—especially in scenarios with strong domain-driven similarities, informative neighborhoods, or the need to accommodate both shared and unique features in populations or processes.

7. Variants, Limitations, and Further Directions

There is a spectrum of model variants—ranging in marginal exactness, locality of dependence, and computational scalability:

• Marginal exactness: Multiple-p DDP and latent multinomial DDP maintain exact Dirichlet marginals; kernel stick-breaking and local DP constructions sacrifice marginals for increased flexibility or computational efficiency (Foti et al., 2012).
• Order of dependence: Markov, AR($V_k(x)$3), and higher-order dependence can be accommodated in time series or spatial models by lattice or graph-based neighborhood specifications (Nieto-Barajas, 2021).
• Practical constraints: In high-dimensional or long-range dependent settings, mixture posteriors can have intractably large support without pruning or approximation (Papaspiliopoulos et al., 2016). For some belief network or graphical models, optimality of linear estimators provides a practical alternative (Hooper, 2012).

Open directions involve non-Markovian dynamics (e.g., subordinated Wright–Fisher priors (Judd et al., 13 Apr 2026)), scaling to massive data, and extending analytic conjugacy to new classes of dependent random measures and generalized species sampling laws. The unifying principle remains the harmonization of marginal Dirichlet structure with interpretable, domain-relevant dependence across measures.