Nonparametric Bayesian Two-Sample Test

• Nonparametric Bayesian two-sample tests are methodologies that assess whether two independent samples come from the same distribution by integrating over uncertainty using flexible nonparametric priors.
• They leverage approaches like Dirichlet process mixtures and optional Pólya trees to adaptively model complex, multimodal distributions and local differences in data.
• Approximation techniques, including recursive algorithms and Monte Carlo integration, enable practical inference despite the combinatorial complexity of evaluating marginal likelihoods.

A nonparametric Bayesian two-sample test is a statistical methodology designed to determine whether two independently sampled datasets originate from the same underlying probability distribution, without imposing restrictive parametric assumptions. In the Bayesian framework, such tests integrate over uncertainty in the latent distributions using flexible nonparametric priors. The most prominent approaches are based on Dirichlet process mixtures (DPM), optional Pólya trees and their generalizations, and measures based on functionals such as the Kolmogorov distance or kernel-based metrics. Below is an in-depth overview of theoretical foundations, modeling, computational strategies, and comparative strengths of nonparametric Bayesian two-sample tests, centered on the rigorous developments and formulations in the literature (0906.4032, 1011.1253, Labadi et al., 2014).

1. Bayesian Formulation of the Two-Sample Problem

Let $X = \{ x_1, ..., x_{m_1} \} \sim q_1$ and $Y = \{ y_1, ..., y_{m_2} \} \sim q_2$ be samples from unknown distributions $q_1$ and $q_2$, respectively. The null and alternative hypotheses are:

• $H_0$: $q_1 = q_2 = q$
• $H_1$: $q_1 \ne q_2$

The Bayesian solution chooses between these hypotheses by evaluating the marginal likelihoods and computing the Bayes factor: $\chi = \frac{P(X, Y \mid H_1)}{P(X, Y \mid H_0)}$ If $\chi > 1$, data favor the alternative. Under nonparametric Bayesian modeling, the prior over distributions $q_1$ and $q_2$ is chosen to be flexible enough to encode broad structure, typically via DPM or random-partition measures.

2. Dirichlet Process Mixtures as Nonparametric Priors

The Dirichlet process (DP) is a measure-valued stochastic process $DP(\alpha, G_0)$ where $\alpha$ is the concentration parameter and $G_0$ the base measure. As a prior over densities, it can be used in mixture models—yielding DPM models that can approximate arbitrary densities. For finite mixtures: $$p(x^{(i)} \mid \varphi) = \sum_{j=1}^C p(x^{(i)} \mid \theta_j) p(c_i = j \mid \zeta)$$ Mixing proportions $\zeta$ have a Dirichlet prior: $$p(\zeta \mid \alpha) = \frac{\Gamma(\alpha)}{[\Gamma(\alpha/C)]^C} \prod_{j=1}^C \zeta_j^{\alpha/C-1}$$ Letting $C \to \infty$ yields the DPM, supporting infinite mixtures and very flexible density learning.

The marginal likelihood for data $D$ under a DPM prior is: $$p(D \mid \alpha, \beta) = \sum_{v \in V} p(v \mid \alpha)\, p(D \mid v, \beta)$$ with $V$ the set of all partitions of $D$. The sum is combinatorially large, but can be approximated efficiently with recursive or clustering-based algorithms.

3. Bayes Factor Computation Under DPM Priors

For the two-sample test, the marginal likelihoods are: $$P(X \mid \alpha, \beta) = \int P(X \mid q_1)\, P(q_1 \mid \alpha, \beta)\, dq_1$$

$$P(Y \mid \alpha, \beta) = \int P(Y \mid q_2)\, P(q_2 \mid \alpha, \beta)\, dq_2$$

$$P(X, Y \mid \alpha, \beta) = \int P(X, Y \mid q)\, P(q \mid \alpha, \beta)\, dq$$

Thus, the nonparametric Bayes factor is: $$\chi = \frac{P(X \mid \alpha, \beta) P(Y \mid \alpha, \beta)}{P(X, Y \mid \alpha, \beta)}$$ where all terms integrate over the space of densities under the DPM prior. This procedure does not require parametric assumptions, and the DPM prior ensures consistent estimation for a wide range of densities.

4. Optional Pólya Trees and Joint Random Measures

The optional Pólya tree (OPT) prior generalizes DPM by defining random measures through recursive partitioning. The coupling optional Pólya tree (co-OPT) (1011.1253) extends this to model two random measures $Q_1$, $Q_2$ simultaneously, introducing "coupling variables" $C(A)$ at each node $A$ in the partition tree:

• If $C(A) = 1$, the two distributions are coupled (identical) on $A$.
• If $C(A) = 0$, independent splits are assigned to $Q_1$ and $Q_2$ on $A$.

The recursive construction generates, for data in node $A$,

$$q_1(x_1 \mid A) q_2(x_2 \mid A) = C(A) q_0^A(x_1, x_2) + (1 - C(A)) \sum_j \lambda_j(A) \frac{D(n_1^j + \alpha_1^j) D(n_2^j + \alpha_2^j)}{D(\alpha_1^j) D(\alpha_2^j)} \prod_i q_1(x_1 \mid A_i^j) q_2(x_2 \mid A_i^j)$$

where all weights and assignments are random under the prior.

The co-OPT framework thus directly targets both global and local differences, as decoupling occurs adaptively in the tree only where data support heterogeneity.

5. Approximate Inference Strategies

Because marginal likelihoods under DPM or co-OPT are generally intractable due to the combinatorial number of partitions, approximation is essential. Key approaches include:

• Recursive algorithms: Marginal likelihoods are computed via tree recursion, terminating early according to thresholds (e.g., node size).
• Bayesian Hierarchical Clustering (BHC): As an efficient O($n^2$) method for Dirichlet process marginal likelihood computation.
• Monte Carlo: When necessary, Monte Carlo integration or sampling over tree paths can approximate posteriors.
• Parallelization: Since distinct branches of the recursive tree are independent given their parent, computation can be easily parallelized.

6. Advantages, Limitations, and Comparison to Parametric Methods

Advantages:

• Flexibility: DPM and Pólya tree priors can represent complex—and multimodal—distributions, adapting to data heterogeneity.
• Integrated Uncertainty: Bayesian inference marginalizes over unknown densities, yielding robust assessment of evidence under limited data.
• Local Structure: Partition-based models (co-OPT) reveal regions of the sample space where differences (or similarities) between distributions are present.

Limitations:

• Computational Cost: Inference, even with approximations, is more intensive than in parametric settings, due to exponential tree growth.
• Tuning Sensitivity: Bayes factors and recursive splits are influenced by hyperparameters (e.g., DP concentration, partition rules).
• Approximation Error: Quality of inference depends on the accuracy and stability of recursion, early stopping rules, or clustering approximations.

Compared to parametric Bayesian two-sample tests (e.g., in the exponential family), these nonparametric approaches are strictly more general: the parametric Bayes factor

$$\chi = \frac{h(\eta,\nu) h(\eta+m_1+m_2, \nu+u(X)+u(Y))}{h(\eta+m_1,\, \nu+u(X)) h(\eta+m_2,\, \nu+u(Y))}$$

is only valid under exponentially structured data and is closed-form. However, misfit of the model leads to dramatic power loss or miscalibration. Nonparametric Bayes methods, in contrast, retain consistency and power in general settings without making model-specific assumptions.

7. Empirical and Practical Considerations

Simulation studies (1011.1253) show that nonparametric Bayesian two-sample tests outperform classical tests like Kolmogorov–Smirnov and Cramer–von Mises under high-dimensional and local-alternative settings, and are competitive with dependent Dirichlet process models or nonparametric distance statistics. For example, in high-dimensional contingency tables (e.g., $2^{15}$ cells), co-OPT achieves higher power and lower sample size requirements compared to L2 distance–based tests.

Typical use-cases include:

• Testing equality of high-dimensional distributions where traditional empirical CDF-based tests fail due to "curse of dimensionality".
• Discovering not only presence, but also local structure (regions) of distributional differences.
• Scenarios with limited or noisy data: integrated uncertainty in density estimation provides more calibrated inference.

The choice of the nonparametric prior (DPM, Pólya tree, co-OPT) should reflect practical trade-offs between computational tractability, interpretability, and the dimensionality or granularity of the hypothesized differences.

Summary

A nonparametric Bayesian two-sample test leverages flexible priors (notably Dirichlet process mixtures and Pólya tree–based partitions) to infer, via the Bayes factor, whether two independent samples are generated from identical or distinct distributions. By marginalizing over latent densities, these methods accommodate arbitrary distributional complexity and yield robust inference. Recent advances, such as co-OPT priors, further enhance local-difference recovery and high-dimensional tractability. Computational challenges are addressed via recursive algorithms, clustering approximations, and parallel processing. Compared to both parametric Bayesian and frequentist alternatives, nonparametric Bayesian tests deliver superior adaptability and power in settings where the form of the underlying distributions is unknown or highly complex (0906.4032, 1011.1253).