Importance conditional sampling for Pitman-Yor mixtures

Abstract: Nonparametric mixture models based on the Pitman-Yor process represent a flexible tool for density estimation and clustering. Natural generalization of the popular class of Dirichlet process mixture models, they allow for more robust inference on the number of components characterizing the distribution of the data. We propose a new sampling strategy for such models, named importance conditional sampling (ICS), which combines appealing properties of existing methods, including easy interpretability and a within-iteration parallelizable structure. An extensive simulation study highlights the efficiency of the proposed method which, unlike other conditional samplers, shows stable performances for different specifications of the parameters characterizing the Pitman-Yor process. We further show that the ICS approach can be naturally extended to other classes of computationally demanding models, such as nonparametric mixture models for partially exchangeable data.

• The paper introduces ICS, a novel method that integrates marginal and conditional sampling techniques for Pitman-Yor mixtures.
• It demonstrates superior computational efficiency and stability across various discount parameter settings through detailed simulation studies.
• Practical application to perinatal data showcases ICS's ability to model complex dependencies in heterogeneous real-world scenarios.

Importance Conditional Sampling for Pitman-Yor Mixtures

Introduction

The paper "Importance Conditional Sampling for Pitman-Yor Mixtures" (1906.08147) introduces a novel sampling methodology for nonparametric mixture models based on the Pitman-Yor process (PY). The PY process is a generalization of the Dirichlet process (DP) that offers enhanced flexibility and robustness in density estimation and clustering tasks. The proposed method, termed Importance Conditional Sampling (ICS), seeks to amalgamate the advantages of existing marginal and conditional sampling techniques while addressing their inherent limitations. Specifically, ICS provides stable performance across varied parameter settings and facilitates computational efficiency through its parallelizable structure.

Methodology

The ICS strategy leverages the representation of the PY process's posterior distribution derived by Pitman, wherein the full conditional distribution of the PY can be decomposed into a mixture involving both discrete and continuous components. This decomposition permits the seamless integration of a sampling-importance resampling mechanism to approximate posterior distributions without explicitly realizing the infinite-dimensional PY measure. Through this method, ICS achieves efficient sampling by focusing on the finite-dimensional summary necessary for updating latent parameters, while the inherent parallelizability of the conditional sampling steps ensures high computational efficiency.

ICS's mechanism is reminiscent of the Blackwell-MacQueen urn scheme and allows the sampler to preserve within-iteration tractability. The finite-dimensional realization is informed by the Dirichlet distribution of process weights and augmented via exchangeable auxiliary variables, facilitating a practical implementation of the model across diverse specifications.

Figure 1: Boxplots for empirical distributions of $M_n$ with $n=100$, demonstrating the influence of the discount parameter $\sigma$ on the sampler's performance and computational constraints.

Simulation Studies

A comprehensive simulation study was conducted to assess ICS's performance relative to established marginal and slice sampling methodologies. The study explored various PY parameter configurations and sample sizes, evaluating the effective sample size (ESS) and computational efficiency through the time/ESS ratio.

ICS consistently exhibited superior computational tractability, particularly at larger discount parameter values, where slice samplers encounter prohibitive computational overheads. Despite slice samplers' competitiveness in scenarios with small $\sigma$, ICS demonstrated robustness across all tested values of $\sigma$, emphasizing its applicability in complex models where parameters significantly impact clustering behavior.

(Figures 2 and 3)

Figures: Simulated data reflecting the mixing and runtime efficiency of the ICS algorithm across different parameter settings, outperforming marginal and slice samplers, especially for larger $\sigma$.

Practical Application

The ICS methodology was applied to analyze data from the Collaborative Perinatal Project (CPP), focusing on gestational age and pollutant exposure across varying hospital settings. The model incorporated dependent Dirichlet processes to accommodate cross-hospital heterogeneity and partial exchangeability, illustrating ICS's flexibility in real-world Bayesian nonparametric models.

ICS facilitated the estimation of joint densities and conditional probabilities of premature birth, with results indicating significant differences between smoker and non-smoker cohorts in pollutant impact. Additionally, the analysis extended to multi-hospital data, employing the ICS framework in modeling the distribution of gestational ages while accounting for inter-hospital dependencies.

(Figures 5 and 6)

Figures: CPP data demonstrating ICS's capability in modeling complex dependencies across hospitals and heterogeneous cohorts effectively, providing insights into gestational age variation and pollutant effects.

Conclusion

The introduction of Importance Conditional Sampling for Pitman-Yor mixtures presents a significant advancement in the nonparametric Bayesian modeling landscape. ICS reconciles the strengths of conditional and marginal methods, and its efficiency at handling diverse parameter configurations positions it as a versatile tool for complex clustering and density estimation problems. Its broad applicability, as demonstrated in both simulation and real-world scenarios, underscores its potential to catalyze further developments in models necessitating robust and scalable inference mechanisms. The paper concludes with an acknowledgment of ongoing research to extend ICS's applicability across other computational demanding models.