Covariate-Dependent Product Partition Models

• Covariate-Dependent PPMx is a Bayesian nonparametric framework that embeds covariate similarity into partition models to promote adaptive clustering.
• It enhances traditional product partition models with a similarity function that increases grouping probability based on similar covariate profiles.
• Applications include adaptive regression, spatial data analysis, and personalized treatment selection, leveraging flexible prior specifications.

A Covariate-Dependent Product Partition Model (PPMx) is a flexible Bayesian nonparametric modeling framework that enriches traditional random partition models by explicitly embedding covariate information into the prior over data partitions. In a PPMx, the probability of grouping two or more observations into the same cluster increases when their covariate profiles are similar according to a user-defined similarity measure. This mechanism enables data-adaptive clustering and improved modeling of heterogeneity in regression, density estimation, clustering, and related inferential tasks.

1. Formal Structure and Mechanism

The PPMx modifies the standard product partition model (PPM), which assigns probability to partitions via a product of "cohesion" functions $c(S_j)$ over clusters, by introducing a covariate-informed similarity term:

$$p(\rho) \propto \prod_{j=1}^K c(S_j) \, g(\mathbf{x}_j^*)$$

where:

• $\rho = \{ S_1, S_2, \ldots, S_K \}$ is a partition of the data indices,
• $c(S_j)$ is the cluster cohesion, typically a function of the cluster size,
• $g(\mathbf{x}_j^*)$ is a cluster-level similarity function, with $\mathbf{x}_j^*$ the set of covariate vectors for cluster $S_j$.

The similarity function $g(\cdot)$ is designed so that it is maximized when the covariates in a cluster are "compact" or similar according to an application-determined criterion. Common approaches include marginalizing out a parametric auxiliary model for the covariates within each cluster or selecting deterministic cluster compactness functions.

Formally, if the covariates include continuous and/or categorical variables, $g$ may be constructed as a product over covariate types, each possibly induced from a latent variable model or directly via a distance function.

2. Partition Priors and Covariate Similarity

In the PPMx family, cluster cohesion can be tailored by adopting completely random measures such as the Dirichlet process, normalized generalized gamma process, or other exchangeable partition priors. The similarity component can take multiple forms:

• Auxiliary model approach: For cluster $S_j$ and covariate $x_{id}$,

$$g_d(\mathbf{x}_j^*) = \int \left[ \prod_{i \in S_j} q(x_{id}|\zeta_j) \right] q(\zeta_j) d\zeta_j$$

with $q(\cdot)$ a "working" likelihood for the covariate and $q(\zeta_j)$ its prior.

• Pairwise similarity: For units $u, u' \in S_j$,

$$s(\mathbf{x}_u, \mathbf{x}_{u'}) = \frac{1}{\Xi_{uu'}} \sum_{d=1}^D \xi_d \, s_d(x_{ud}, x_{u'd})$$

Finally, $g(\mathbf{x}_j^*)$ is typically the average similarity across pairs in $S_j$.

• Deterministic compactness penalty: For clustering individuals where cluster $A_j$ has centroid $c_{A_j}$,

$$\mathcal{D}_{A_j} = \sum_{i\in A_j} d(\mathbf{x}_i, c_{A_j}) \qquad g(\mathbf{x}_j^*) = \exp[ -\lambda \mathcal{D}_{A_j} \log (1 + \lambda \mathcal{D}_{A_j}) ]$$

This construction allows researchers to encode domain knowledge regarding which aspects of the covariate space are most relevant to clustering.

3. Random Partitioning and Covariance Structure

The prior on partitions in a PPMx is not only informed by covariate similarity but may also include spatial, temporal, or network structure through modification of the cohesion $c(S_j)$. For example:

• Spatial Product Partition Models (SPPM): The cohesion $c(S_j, s_j^*)$ incorporates spatial coordinates, penalizing spatially dispersed clusters to favor spatially localized groupings.
• Markov Random Field-constrained PPM: An MRF cost function is included to favor clusters with many geographically contiguous or graph-adjacent members (Pan et al., 2020).

Dependent partition-valued processes, including those utilizing latent Gaussian processes, extend the PPMx framework by allowing partitions themselves to depend smoothly on continuous covariates (e.g., space, time), as in (Karabatsos et al., 2012) and (Palla et al., 2013).

4. Inference, Variable Selection, and Computation

Posterior inference in PPMx models is typically performed via Markov chain Monte Carlo (MCMC) algorithms. Gibbs sampling and partially-collapsed Gibbs or auxiliary variable samplers are standard choices due to the non-conjugate and high-dimensional nature of the parameter space. Through explicit modeling of the covariates, PPMx supports covariate selection either by incorporating variable selection indicators directly in the similarity function (e.g., latent $\gamma_{jd}^*$ controlling the impact of covariate $d$ in cluster $j$) or using spike-and-slab priors on regression coefficients when joint response-covariate models are adopted (Barcella et al., 2015, Barcella et al., 2015).

For high-dimensional covariates, computational tractability is achieved by selecting "working" likelihoods for similarity functions. Extensions to allow for missing data are handled by dropping missing entries in the similarity computation (Page et al., 2019).

5. Applications and Model Extensions

PPMx models have seen application in diverse areas, including:

• Bayesian nonparametric regression: Adaptive modeling of the conditional density $p(y|x)$, capturing multimodality and nonlinearity by allowing partitions of $y$ to depend on $x$.
• Variable selection in regression: Cluster-specific variable selection via spike-and-slab base measures (Barcella et al., 2015).
• Personalized treatment selection: PPMx-driven clustering of patients according to molecular or prognostic markers, with predictive inference for optimal treatment arms using flexible cohesion functions (e.g., Normalized Generalized Gamma Process) (Pedone et al., 2022).
• Spatial data analysis: Explicit spatial clusters for areal/geostatistical data, enabling decomposition of spatial correlation into local (cluster-specific) and global (e.g., Gaussian process) components (Page et al., 2015).
• Aggregate monitoring across studies: AE rate estimation and real-time safety monitoring where data come from multiple trials with variable covariate granularity, using pairwise similarity-driven PPMx clustering (Yuan et al., 8 Sep 2025).
• Clustering functional or ranking data: Generalization to ranking or preference data using Mallows models informed by PPMx-type similarity priors (Eliseussen et al., 2023).

These extensions also demonstrate compatibility with normalized completely random measures (e.g., NGG process), spatial and network constraints, or hierarchical modeling.

6. Advantages and Theoretical Properties

Covariate-Dependent Product Partition Models possess several key advantages:

• Data-driven clustering: Observations with similar covariates are a priori favored to co-cluster, supporting adaptive estimation of latent structure.
• Flexible prior specification: The separation of cohesion (size/structure of clusters) and similarity (covariate effect) allows precise control over the clustering mechanism and prior beliefs regarding cluster homogeneity.
• Improved predictive performance: By borrowing strength adaptively across similar units, especially in regimes of sparse data (e.g., small subgroups, rare events), PPMx structures can substantially reduce posterior uncertainty and improve out-of-sample predictions.
• Handling of missing data: Rather than imputation, missingness is accommodated directly in the covariate similarity construction (Page et al., 2019).
• Consistency and theoretical support: Under suitable regularity conditions, Bayesian PPMx procedures concentrate posterior mass around the true (or best) partition as the sample size increases (Pan et al., 2020).

7. Comparative Perspective with Alternative Models

PPMx differs fundamentally from other Bayesian nonparametric methods for covariate-dependent clustering, such as:

• Dependent Dirichlet process mixtures and stick-breaking models: These allow the mixture weights (and sometimes atoms) to be direct functions of covariates, but do not generally use a prior directly penalizing within-cluster covariate heterogeneity; the partition prior in PPMx provides a distinct mechanism (Fujimoto et al., 2012, Barcella et al., 2015).
• Partition construction via latent Gaussian processes: This approach, as in (Karabatsos et al., 2012, Palla et al., 2013), induces correlation in partitions through a global latent process, facilitating both local smoothing and nonparametric partitioning based, not on pairwise similarity alone, but on continuous covariate maps.
• Profile Regression and joint models: These are often equivalent to PPMx under joint modeling of responses and covariates, but the PPMx framework is more transparent in separating the roles of clustering and covariate similarity.
• Spatial or network-constrained extensions: Spatial PPMx (Page et al., 2015) and Markov-random-field PPM (Pan et al., 2020) further demonstrate the modularity with which additional dependence structures may be incorporated.

This comparative landscape highlights the PPMx as a highly adaptable tool for modeling structured heterogeneity, variable selection, and flexible Bayesian inference across a wide range of applied problems in statistics and machine learning.