Learning Mixtures of Nonparametric and Convolutional Measures on Effectively Low-dimensional Affine Spaces

Abstract: In this paper, we develop a finite mixture of convolutional distributions, a statistical model to analyze continuous data distributed approximately on a mixture of low-dimensional affine subspaces. The observations are assumed independent and identically distributed from the mixture of distributions, where each component arises from a convolution of a distribution supported on a low-dimensional subspace with a suitable noise kernel. We discuss theoretical properties of such class of models, including identifiability under very general conditions - in particular, showing that the minimal representation for such mixtures is uniquely identifiable in a semi-parametric setting. We further study the posterior contraction rates for the parameters for a parametrized class of such models where the supports of the component mixing measures are assumed to be convex polytopes under a suitable well-specified Bayesian regime. This still requires developing novel inverse bounds for problems involving a nested mixture structure, where the mixture kernel is itself another continuous mixture. Our approach for both the identifiability theory and posterior contraction rates is to exploit the geometric structure of the underlying support of the latent measures. Apart from applications in end-member analysis, spectral unmixing and topic models, this study provides a grounded framework for subspace clustering with the goal of exploring conditions for learning multiple latent low-dimensional structures. We illustrate our findings through careful simulation study, which also includes developing new algorithms for such class of models

• The paper introduces a flexible semiparametric mixture model that unifies nonparametric subspace structures with convolutional noise, enhancing identifiability through distinct affine hulls.
• The paper develops novel algorithms and theoretical guarantees, including inverse bounds and a posterior contraction rate of O(√((log n)/n)) for consistent parameter recovery.
• The paper demonstrates practical efficacy via simulations, illustrating robust recovery of polytopes, mixture weights, and noise parameters even with intersecting component supports.

Learning Mixtures of Nonparametric and Convolutional Measures on Effectively Low-dimensional Affine Spaces

Model Formulation and Geometric Underpinnings

This work introduces a semiparametric mixture model for data residing approximately on a union of low-dimensional affine spaces, each subject to full-dimensional convolutional noise. For a sample $X \in \mathbb{R}^D$, the data-generating model is a finite mixture:

$p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$

where each $p_k$ is a convolution $G_k \star \phi_k$, with $G_k$ a probability measure supported (often nonparametrically) on a low-dimensional affine subset of $\mathbb{R}^D$ and $\phi_k$ a component-specific noise kernel. This includes, as special cases, models such as mixture of probabilistic PCA (MPPCA), mixture of factor analyzers (MFA), topic models, and archetypal analysis.

The model allows for highly flexible subspace structure:

• $G_k$ can have support on arbitrary polytopes and even sets with complex topology or nontrivial intersections.
• Supports $S_k$ of $G_k$ may overlap as long as their affine spans are distinct (or under a weaker connected-disjointness condition).
• Component noise kernels $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$0 can differ in variance and even be non-Gaussian.

The latent variable generative process is hierarchical, with a discrete subpopulation indicator $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$1, then a latent low-dimensional variable $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$2 (for $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$3), and finally $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$4 with $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$5.

Figure 1: Visualization in $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$6 with three components; two 2D supports (planes intersecting along a line) and one 1D support (disjoint union of segments). The left shows the clean latent structure; the right shows observed data after convolutional noise.

The model subsumes numerous existing latent structure models, unifying them under a common geometric-convolutional framework while allowing arbitrary nonparametric distributions supported on complex subspaces.

Identifiability: Geometric Criteria and Deconvolutional Arguments

A central theoretical challenge is identifiability in such flexible semiparametric models, especially since classical nonparametric mixture models are notoriously non-identifiable in the absence of structural assumptions. The paper introduces a geometric separation criterion ("Assumption A"):

• Distinct Affine Hulls: The supports $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$7 of the $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$8 are such that, for any $p(x) = \sum_{k=1}^K \pi_k \, p_k(x)$9, either their affine spans differ, or their intersection is strictly lower-dimensional.

This condition is purely geometric—no separation or disjointness is required beyond affine span distinguishability. Under this condition, the authors establish:

• The minimal mixture representation is unique up to label permutation (number of components, support dimensions, and nonparametric laws).
• The result generalizes to a weaker "connected component" criterion (Assumption B), effectively partitioning the union of supports into connected subregions.

When convolutional noise is added, identifiability is achieved via analysis in characteristic function space:

• If $p_k$0 is from an appropriate family (e.g., Gaussian, isotropic stable), and minimal noise variance exists, component-wise deconvolution is possible.
• A recursive "peeling" strategy isolates singular (low-dimensional) parts in Fourier space, matches latent $p_k$1 via geometric criteria, then repeats on the residual.

Polytope-supported Components and Natural Parametrizations

To enable tractable estimation and theoretical rates, the paper proposes a natural parametric form for components: $p_k$2 is the pushforward of a distribution (e.g., Dirichlet) on the simplex through a map defined by $p_k$3 vertices (endmembers), resulting in a polytope-supported, absolutely continuous measure on a $p_k$4-dimensional affine set.

Figure 2: Mixture in $p_k$5 with two components, each on a 2D triangle; displays latent noiseless polytopes and noisy observed samples. The component mixing distribution's effect is visible in the color shading.

This enables direct connections to:

• Archetypal analysis: endmember identification, convex geometry.
• Topic models: convex hulls of topic vectors, Dirichlet mixing over word proportions.
• Subspace clustering: interpretable, geometric clustering beyond elliptical models.

Identifiability for such polytope mixtures, modulo non-minimal representations, is formalized using the notion of "exposed" vertices—those that are simultaneously extreme points for the mixture polytope and uniquely associated to a single component.

Parameter Estimation: Posterior Contraction and Inverse Bounds

A technical milestone is the derivation of inverse bounds—lower bounding statistical distance between two models by the metric in parameter space—critical for converting density estimation rates to parameter estimation/convergence rates.

The authors introduce a robust metric $p_k$6 on model parameters (up to permutation), and show via intricate geometric and analytic arguments that, under total exposure of the component polytopes, there exists $p_k$7 such that:

$p_k$8

locally around the true parameter $p_k$9. This facilitates translation of posterior contraction for densities (in Hellinger or KL metric) to parametric contraction of the underlying structural parameters.

Through careful entropy and prior mass calculations (for suitably regular priors), they prove that the posterior for model parameters concentrates at the optimal rate $G_k \star \phi_k$0 in the well-specified case of fixed $G_k \star \phi_k$1, $G_k \star \phi_k$2, and Dirichlet mixing.

Algorithms and Simulation Studies

The work develops new inference algorithms for these mixture models:

• For single components (no mixture), extensions of spectral moment methods for Dirichlet and Gaussian kernels, as well as Bayesian MCMC methods via latent variable augmentation.
• For mixtures, both EM-type algorithms via deterministic or Monte Carlo approximations, and Grouped Independent Metropolis-Hastings MCMC using a pseudo-marginal likelihood strategy.

Extensive simulation studies validate:

• Consistent recovery of polytopes, mixture weights, and noise parameters under diverse settings (e.g., varying $G_k \star \phi_k$3, $G_k \star \phi_k$4, support geometry, symmetric and asymmetric Dirichlet mixing).
• Empirical estimation rates consistent with theory, except in challenging high-dimensional or highly mis-specified settings.
• Success of model selection via BIC-type criteria for joint estimation of $G_k \star \phi_k$5 and $G_k \star \phi_k$6.
• Robustness of inference to mis-specification of the mixing family.

  Figure 3: Simulation results (single component): Parametric convergence rates are observed for simplex-type supports; polytope cases show slightly slower but still favorable behavior.

  

  Figure 4: General mixture settings: EM and Gaussian approximation-based methods achieve strong estimation accuracy; MCMC performs less efficiently in challenging settings due to high-dimensionality and complex local likelihood landscape.

  

  Figure 5: Model selection performance: BIC reliably recovers ground-truth $G_k \star \phi_k$7 and $G_k \star \phi_k$8 across sample sizes.

Theoretical and Practical Implications

The results demonstrate that, given weak geometric structure in mixture supports, a broad class of semiparametric mixtures (encompassing numerous latent structure models) is generically identifiable. This allows rigorous parameter recovery (up to permutation) even with potentially intersecting component supports and flexible nonparametric forms.

The derivation of inverse bounds for nested mixture-convolutional structures is of practical importance for the study of continuous and hierarchical latent variable models. This broadens the class of tractable models for endmember analysis, topic modeling, spectral unmixing, and interpretable probabilistic clustering.

From a methodological perspective, the introduced algorithms demonstrate practical feasibility for moderate $G_k \star \phi_k$9, $G_k$0 even in high-dimensional data. The proposed framework resolves strong identifiability limitations of general nonparametric mixture models by leveraging geometric and convolutional structure.

Future Directions

Open problems and promising directions highlighted in the paper include:

• Generalization to fully nonparametric mixing distributions $G_k$1 (e.g., Dirichlet processes) and the associated rates of contraction.
• Extension to high-dimensional regimes (growing $G_k$2) and adaptive selection of $G_k$3, $G_k$4, and support geometry.
• Development of scalable, gradient-based MCMC and stochastic-EM algorithms able to handle intractable likelihoods in real large-scale applications.
• Further exploration of links to modern clustering, manifold learning, and interpretable AI through the lens of geometric mixture decompositions.

Conclusion

This work advances both theoretical and computational understanding of mixtures of low-dimensional, nonparametric, and convolutional measures, providing a unified framework for many problems in geometric and latent variable modeling. Through minimal geometric conditions, rigorous identifiability is established, with efficient estimation procedures substantiated both theoretically and empirically. This enables principled recovery of latent low-dimensional subpopulations in mixed and noisy data environments, with broad applicability in statistical machine learning and beyond.