---
title: Pitman-Yor Process
url: https://www.emergentmind.com/topics/pitman-yor-process-95e4e7aa-5873-4a2e-8543-ea2e430d5318
type: topic
---

# Pitman-Yor Process

The Pitman-Yor process (PYP) is a fundamental two-parameter family of discrete random probability measures that generalizes the Dirichlet process by inducing power-law tails in the sizes of partition blocks. It arises in Bayesian nonparametrics, combinatorial stochastic processes, genetics, and statistical mechanics. The PYP admits multiple representations: stick-breaking construction, exchangeable partition probability functions (EPPFs), Chinese restaurant process (CRP), and subordinator embedding. Its flexibility in modeling heavy-tailed distributions makes it a canonical prior for applications where the cluster-size distribution is empirically found to follow power laws.

## 1. Mathematical Definition and Core Representations

Let $H$ be a nonatomic base measure on a space $\Theta$, $d \in [0,1)$ a discount parameter, and $\alpha > -d$ a concentration parameter. The Pitman-Yor process is the law of a random probability measure $G \sim \mathrm{PY}(d, \alpha, H)$ such that, for any partition $(A_1, ..., A_k)$ of $\Theta$, $(G(A_1), ..., G(A_k))$ has the law of the first $k$ masses of a Poisson-Dirichlet distribution $\operatorname{PD}(d, \alpha)$ [1210.4184][1609.06783].

The canonical stick-breaking representation is:
\[
V_k \sim \mathrm{Beta}(1-d, \alpha + k d), \qquad
\pi_k = V_k \prod_{l=1}^{k-1} (1 - V_l), \qquad
\theta_k \sim H,
\]
\[
G = \sum_{k=1}^\infty \pi_k\,\delta_{\theta_k}
\]
If $d=0$, $\mathrm{PY}(0, \alpha, H)$ coincides with the Dirichlet process [1210.8094][2003.12200].

## 2. Partition Structure and Predictive Rules

A sample $(\theta_1, ..., \theta_N) \sim G$ induces a random partition with the following predictive probabilities:
- If current clusters are labeled $\{\phi_c\}$ with counts $n_c$:
\[
P(\theta_{M} = \phi_{c} | \{\theta_{i}\}, d, \alpha) = \frac{n_c - d}{(M-1)+\alpha}
\]
\[
P(\theta_{M}\ \text{new} | \{\theta_{i}\}, d, \alpha) = \frac{\alpha + d\, t}{(M-1)+\alpha}
\]
Here, $t$ is the current number of clusters [1210.4184][1810.06227][1609.06783].

The EPPF is:
\[
p(n_1,...,n_k) = \frac{\prod_{i=1}^{k-1} (\alpha + i d)}
{(\alpha + 1)_{n-1} \prod_{j=1}^k (1-d)_{n_j-1}}
\]
where $(x)_n=x(x+1)...(x+n-1)$ [1602.08382][1501.00208].

For $d=0$ the clustering reduces to logarithmic growth (Dirichlet process); for $d>0$ the expected number of clusters grows as $n^d$ (power law) [1602.08382][2102.06059][2208.14255].

## 3. Extensions: Kernel Pitman-Yor and Hierarchical Constructions

### Kernel Pitman-Yor Process (KPYP)
To model spatial or temporal dependencies, the KPYP modifies the stick-breaking by allowing each stick's discount parameter to depend on an external predictor $x$:
\[
d_k(x) = k(x, \varphi_k)
\]
where $k(\cdot, \cdot)$ is a kernel (e.g., Gaussian/RBF), and $\{\varphi_k\}$ are latent cluster locations. The local mixing measure at $x$ is:
\[
G_x = \sum_{k=1}^\infty \pi_k(x)\delta_{\theta_k}
\]
where
\[
V_k(x) \sim \mathrm{Beta}(1-d_k(x), \alpha + k d_k(x)),\ 
\pi_k(x) = V_k(x) \prod_{l<k}(1 - V_l(x))
\]
This results in predictor-dependent random probability measures, enabling further spatial/temporal adaptivity [1210.4184].

### Hierarchical Pitman-Yor Processes
Deep hierarchies are formed by recursively drawing child measures from parent PYPs:
\[
G_0 \sim \operatorname{PY}(d_0, \theta_0, H)
\]
\[
G_j | G_0 \sim \operatorname{PY}(d_1, \theta_1, G_0)
\]
This is central to hierarchical topic models and language models with power-law frequency behavior, and is the foundation for the Chinese restaurant franchise representation in collapsed Gibbs-sampling algorithms [1609.06783][1501.00208].

## 4. Limit Theorems and Asymptotics

The law of large numbers holds: empirical measures $\mu_n = \frac{1}{n}\sum_{i=1}^n \delta_{X_i}$ from a PYP converge almost surely to the base measure $H$.

For $d \to 0$, the Pitman-Yor process approaches the Dirichlet process.

Large deviations as $d \to 1$ exhibit phase transitions and non-Gaussian fluctuations. The first weight $P_1$ has limiting concentration for $\alpha \to 1$, and as $\alpha \to 0$, all mass concentrates at one atom [1602.08382].

Species sampling: For $\sigma > 0$, the sample contains $K_n = O(n^\sigma)$ unique values, fundamentally changing cluster richness compared to logarithmic behavior of the Dirichlet process [2102.06059][2208.14255].

## 5. Bayesian Inference, Estimation, and Algorithms

Posterior inference adapts standard DP machinery:
- Gibbs/CRP sampler for cluster assignment:
  - Assign existing cluster or create new cluster with probabilities as in Section 2.
- Stick-breaking truncated samplers can control total variation approximation error via random truncation, with sampling cost scaling in the truncation point $\tau(\epsilon)$, whose asymptotics are governed by polynomially tilted stable laws [1806.10867].
- Variational Bayesian approximation is tractable for KPYP using coordinate ascent in the truncated stick-breaking -- explicit update formulas are available for location-specific sticks, cluster indicators, atoms, and concentration parameter [1210.4184].

Empirical Bayes and full Bayes inference on the type parameter $\sigma$ achieves asymptotic normality with rate $1/\sqrt{n^\sigma_0}$ where $\sigma_0$ is the true power-law exponent [2208.14255], and Bernstein-von Mises theorems provide posterior Gaussianity after bias correction for discrete data [2102.06059].

Estimation of additive functionals (e.g., Shannon entropy) is analytically tractable for PYP priors, with closed-form posterior mean and variance expressions, and the Pitman-Yor Mixture (PYM) estimator achieves frequentist consistency for a broad class of models [1302.0328].

## 6. Applications and Generalizations

The Pitman-Yor process is widely used in:
- Nonparametric density estimation: As kernel mixture prior achieves nearly parametric posterior contraction rates over adaptive smoothness classes [1210.8094].
- Bayesian entropy estimation for infinite discrete spaces [1302.0328].
- Power-law topic and language modeling (Hierarchical PYP topic model, n-gram models with heavy tails) [1609.06783].
- Functional and spatial clustering, speaker identification, and spatio-temporal point processes (via KPYP) [1210.4184].
- Classification via species sampling frameworks, including explicit representation of diversity [1908.07186].
- Product space modeling: Enriched Pitman-Yor processes (EPY) provide nested product models with independent clustering in each space, admitting “square-breaking” stick representations and unifying mixture-of-mixtures and spike-and-slab Bayesian priors [2003.12200].

## 7. Theoretical Connections and Related Classes

The PYP is interrelated with:
- Poisson-Dirichlet processes (the ranked size sequence underlying the PYP weights).
- Stable subordinators and generalized gamma processes: Stick-breaking constructions, bridge representations, and their conditioning extend to PG($\alpha, \zeta$) and EPG($\alpha, \zeta$) classes, covering all PD($\alpha, \theta$) via appropriate mixing.
- Indian Buffet Process (IBP): Two-parameter PYP is the combinatorial engine behind the power-law three-parameter IBP for exchangeable feature allocation [1501.00208].
- Coagulation-fragmentation chains: Markov structure induced by successively deleting or merging atoms in the PD($\alpha, \theta$) partitions [1308.6570][1908.07186].

Theoretical tools from subordinator calculus, large deviations, and regenerative composition structures underpin much of the rigorous analysis [1602.08382][1308.6570][1908.07186].

---

The Pitman-Yor process and its extensions serve as archetypal priors for nonparametric Bayesian inference in models that require more flexible, heavy-tailed clustering distributions than the Dirichlet process admits. Their analytic tractability, rich asymptotic theory, and broad applicability justify their central role in modern statistical learning [1210.4184][1609.06783][1806.10867][1602.08382][1210.8094][2102.06059][1908.07186][1810.06227][1302.0328][1308.6570][1501.00208][2208.14255][2003.12200].

Source: https://www.emergentmind.com/topics/pitman-yor-process-95e4e7aa-5873-4a2e-8543-ea2e430d5318