Lower-dimensional posterior density and cluster summaries for overparameterized Bayesian models

Abstract: The usefulness of Bayesian models for density and cluster estimation is well established across multiple literatures. However, there is still a known tension between the use of simpler, more interpretable models and more flexible, complex ones. In this paper, we propose a novel method that integrates these two approaches by projecting the fit of a flexible, over-parameterized model onto a lower-dimensional parametric summary, which serves as a surrogate. This process increases interpretability while preserving most of the fit of the original model. Our approach involves three main steps. First, we fit the data using nonparametric or over-parameterized models. Second, we project the posterior predictive distribution of the original model onto a sequence of parametric summary estimates using a decision-theoretic approach. Finally, given the lower parametric dimension of the summary estimate that best approximates the original model learned in the second step, we construct uncertainty quantification for the summary by projecting the original full posterior distribution. We demonstrate the effectiveness of our method in summarizing a variety of nonparametric and overparameterized models, providing uncertainty quantification for both density and cluster summaries on synthetic and real datasets.