A multiscale Bayesian nonparametric framework for partial hierarchical clustering

Abstract: In recent years, there has been a growing demand to discern clusters of subjects in datasets characterized by a large set of features. Often, these clusters may be highly variable in size and present partial hierarchical structures. In this context, model-based clustering approaches with nonparametric priors are gaining attention in the literature due to their flexibility and adaptability to new data. However, current approaches still face challenges in recognizing hierarchical cluster structures and in managing tiny clusters or singletons. To address these limitations, we propose a novel infinite mixture model with kernels organized within a multiscale structure. Leveraging a careful specification of the kernel parameters, our method allows the inclusion of additional information guiding possible hierarchies among clusters while maintaining flexibility. We provide theoretical support and an elegant, parsimonious formulation based on infinite factorization that allows efficient inference via Gibbs sampler.