Barycenter Kernel MMD

• Barycenter Kernel MMD is a framework that summarizes probability measures by aligning their reproducing kernel Hilbert space (RKHS) mean embeddings.
• It employs discrete approximations and optimal weight assignments to minimize the MMD, facilitating clustering, quantization, and kernel quadrature.
• The approach integrates gradient flows, particle algorithms, and multi-kernel aggregations while addressing convergence and higher-order structure challenges.

Barycenter Kernel Maximum Mean Discrepancy (MMD) denotes a class of constructions in which probability measures are compared, summarized, or optimized through their reproducing kernel Hilbert space (RKHS) mean embeddings, and barycentric structure is expressed in that feature-space geometry rather than through transport in the original sample space. In the single-target setting, a discrete weighted measure $\nu_{Y,w}=\sum_{i=1}^M w_i\delta_{y_i}$ is sought so that its mean embedding is close to that of a target $\pi$, yielding a “kernel MMD barycenter approximation” of $\pi$. In the multi-distribution setting, one minimizes weighted sums of squared MMDs, and the RKHS barycenter is the weighted sum of the constituent mean embeddings (Belhadji et al., 14 Feb 2025). This viewpoint places barycenters, quantization, quadrature, and kernel two-sample methodology inside a common Hilbertian framework, while the mathematical meaning of the induced geometry depends critically on whether the chosen kernel makes MMD a genuine metric that metrizes weak convergence (Simon-Gabriel et al., 2020).

1. RKHS formulation of barycentric MMD

For a positive definite kernel $k$ with RKHS $\mathcal H_k$, the kernel mean embedding of a probability measure $\mu$ is

$m_\mu = \int k(\cdot,x)\,d\mu(x),$

and the maximum mean discrepancy is the RKHS norm distance

$$\mathrm{MMD}(\mu,\nu)=\|m_\mu-m_\nu\|_{\mathcal H_k}.$$

Equivalently,

$$\mathrm{MMD}_k^2(\mu,\nu) = \mathbb E_{X,X'\sim\mu}[k(X,X')] -2\mathbb E_{X\sim\mu,Y\sim\nu}[k(X,Y)] +\mathbb E_{Y,Y'\sim\nu}[k(Y,Y')].$$

These identities make barycentric formulations immediate: a probability measure is represented by a single element of $\mathcal H_k$, and barycenters are defined by Hilbert-space averaging of such representations (Belhadji et al., 14 Feb 2025).

In the discrete approximation setting, the target $\pi$0 is approximated by

$\pi$1

with feature-space representation

$\pi$2

Seeking $\pi$3 such that $\pi$4 is small is therefore exactly the problem of finding a finite barycentric summary whose mean embedding is close to $\pi$5 (Belhadji et al., 14 Feb 2025).

For several input measures $\pi$6 with weights $\pi$7, $\pi$8, the natural objective is

$\pi$9

The objective is minimized when

$\pi$0

so the barycenter in RKHS is simply the weighted sum of mean embeddings (Belhadji et al., 14 Feb 2025). In that sense, “barycenter kernel MMD” is fundamentally a feature-space barycenter construction.

2. Discrete barycenters, weighted quantization, and support constraints

A central finite-dimensional formulation is

$\pi$1

With $\pi$2 and $\pi$3, where

$\pi$4

the squared discrepancy satisfies

$\pi$5

For fixed support $\pi$6, the optimal weights are

$\pi$7

and the reduced objective becomes

$\pi$8

(Belhadji et al., 14 Feb 2025).

This formulation differs from classical $\pi$9-quantization in two ways emphasized in the literature. First, the discrepancy is MMD rather than Wasserstein distance. Second, the weights are not constrained to the simplex; they are allowed to lie in $k$0, which is a relaxation of classical Wasserstein quantization (Belhadji et al., 14 Feb 2025). The resulting discrete measure is simultaneously a quantizer, a clustering-type representation, and an RKHS barycenter approximation.

The same structure appears in kernel quadrature and Bayesian quadrature. For fixed nodes $k$1, kernel quadrature chooses weights by minimizing

$k$2

and when the kernel matrix is invertible the posterior variance of Bayesian quadrature is exactly the squared MMD between the quadrature measure and $k$3 (Naslidnyk, 25 Feb 2026). In this restricted-support sense, kernel quadrature is a constrained barycenter problem in MMD geometry.

3. Metric meaning and topological conditions

The usefulness of barycenter constructions depends on whether MMD defines a meaningful geometry on probability measures. For bounded continuous kernels, kernel mean embeddings

$k$4

exist for finite regular Borel measures, and

$k$5

is the induced MMD distance (Simon-Gabriel et al., 2020).

On compact Hausdorff spaces, a bounded Borel measurable kernel metrizes weak convergence of probability measures if and only if it is continuous and characteristic to $k$6. On locally compact, non-compact Hausdorff spaces, assuming $k$7, the correct condition is stronger: $k$8 metrizes weak convergence if and only if it is continuous and integrally strictly positive definite over all signed, finite, regular Borel measures (Simon-Gabriel et al., 2020). These conditions are foundational for barycenter problems in MMD space because they ensure that convergence in MMD coincides with the standard probabilistic notion of weak convergence.

This distinction matters in non-compact spaces. If the kernel fails the relevant condition, one can have sequences that converge in MMD while mass escapes to infinity, which undermines barycentric interpretations at the level of distributions (Simon-Gabriel et al., 2020). By contrast, for standard kernels such as Gaussian, Laplace, Matérn, and inverse multiquadratic on $k$9, the condition $\mathcal H_k$0 is satisfied, and the kernels commonly used in practice fall into the regime where MMD can serve as a probabilistically meaningful geometry (Simon-Gabriel et al., 2020).

A complementary observation is that MMD barycenters are trivial in RKHS but not necessarily trivial as measures. The embedding barycenter is the weighted mean of embeddings; realizing that point as an actual probability measure, or as a discrete approximation with prescribed support size, is the nontrivial part of the problem (Belhadji et al., 14 Feb 2025, Naslidnyk, 25 Feb 2026).

4. Gradient flows and particle algorithms

One line of work studies barycentric MMD optimization dynamically through gradient flows. For

$\mathcal H_k$1

the Wasserstein gradient flow of MMD satisfies the continuity equation

$\mathcal H_k$2

where the first variation is the witness function. Along this flow,

$\mathcal H_k$3

so the MMD energy is non-increasing (Arbel et al., 2019).

For weighted discrete barycenters, a more suitable geometry is Wasserstein–Fisher–Rao (WFR), because WFR allows mass creation and destruction. The associated PDE is

$\mathcal H_k$4

where the reaction term enables weights to adapt rather than remain simplex-constrained (Belhadji et al., 14 Feb 2025). Under a discrete ansatz

$\mathcal H_k$5

this yields the interacting ODE system

$\mathcal H_k$6

$\mathcal H_k$7

The location dynamics decrease MMD through kernel gradients, while the weight dynamics increase or decrease particle mass according to local over- or under-representation in feature space (Belhadji et al., 14 Feb 2025).

Stationary conditions for this WFR system lead to the fixed-point map called Mean Shift Interacting Particles (MSIP),

$\mathcal H_k$8

which extends classical mean shift from $\mathcal H_k$9 to interacting multi-particle barycenters. Under the kernel-gradient structure $\mu$0, the gradient of the reduced objective satisfies

$\mu$1

so MSIP is exactly a preconditioned gradient descent method. The paper further identifies MSIP as an exact MMD-side analog of Lloyd’s algorithm (Belhadji et al., 14 Feb 2025).

A distinct numerical realization is EVI-MMD, which interprets MMD as a free energy and applies an implicit Euler discretization. Each time step solves a proximal problem

$\mu$2

typically by L-BFGS, and uses an adaptive Gaussian bandwidth

$\mu$3

to interpolate between coarse exploration and fine exploitation (Chen et al., 2021).

5. Kernel-side aggregation and related constructions

Barycentric structure also appears on the kernel side. Because MMD is linear in the kernel,

$\mu$4

when $\mu$5. This makes multi-scale Gaussian mixtures a natural way to define composite MMD geometries, although scale normalization can be important because raw MMD values vary substantially with bandwidth, dimension, and batch size (Rustamov, 2019).

A more general aggregation strategy is to combine several single-kernel MMD estimates through their Mahalanobis distance. For a kernel family $\mu$6, let

$\mu$7

Then the Mahalanobis-aggregated statistic is

$\mu$8

where $\mu$9 estimates the joint null covariance of the MMD vector. This aggregation is universally consistent, has non-trivial asymptotic Pitman efficiency under local alternatives, and is more powerful in finite samples across a range of alternatives than single-kernel tests in the reported experiments (Chatterjee et al., 2023). Although this is a testing construction rather than a distributional barycenter, it is another instance of covariance-aware barycentric combination in MMD space.

Random-feature kernel learning yields a related perspective. A kernel can be written as

$m_\mu = \int k(\cdot,x)\,d\mu(x),$0

so learning the distribution $m_\mu = \int k(\cdot,x)\,d\mu(x),$1 over random features amounts to learning a mixture, or empirical barycenter, of rank-one base kernels $m_\mu = \int k(\cdot,x)\,d\mu(x),$2. In the mean-field limit, the empirical distribution of the feature particles follows a Wasserstein gradient flow on the space of feature distributions (Khuzani et al., 2019).

6. Applications, limitations, and broader landscape

Barycenter kernel MMD constructions appear in several applied forms. In weighted quantization they provide clustering- and quadrature-type summaries of a target distribution with adaptive locations and weights (Belhadji et al., 14 Feb 2025). In simulation-based inference, optimally weighted MMD estimators use Bayesian quadrature weights to approximate $m_\mu = \int k(\cdot,x)\,d\mu(x),$3 more efficiently than standard empirical estimators under smoothness assumptions (Naslidnyk, 25 Feb 2026). In deterministic sampling and generative modeling, particle systems are evolved so that the empirical measure minimizes MMD to a target distribution or to empirical data (Chen et al., 2021). In Wasserstein auto-encoders, Gaussian-kernel MMD admits closed forms against the standard normal prior, and multi-kernel mixtures can be standardized componentwise to avoid scale domination (Rustamov, 2019).

A persistent misconception is to identify MMD barycenters with Wasserstein barycenters. The two geometries are different. Under MMD, each distribution is represented by a point in an RKHS, and the barycenter is the Euclidean barycenter of those points in feature space. Under Wasserstein $m_\mu = \int k(\cdot,x)\,d\mu(x),$4, the geometry is based on transporting mass in the original space, and the barycenter minimizes average transport cost (Rustamov, 2019). The analogy between MSIP and Lloyd’s algorithm is structural, not geometric: both are preconditioned fixed-point methods, but one operates in RKHS-induced MMD geometry and the other in Euclidean transport geometry (Belhadji et al., 14 Feb 2025).

Several open questions remain explicit. For MSIP, the analytical status of non-degeneracy, descent, local minima versus saddle points, and convergence rates is unresolved (Belhadji et al., 14 Feb 2025). More broadly, recent work on kernel quantile discrepancies argues that MMD’s reliance on mean embeddings can miss higher-order structure, and proposes quantile-based kernel discrepancies that recover sliced Wasserstein and max-sliced Wasserstein as special cases while retaining near-linear algorithms (Naslidnyk, 25 Feb 2026). This suggests that barycenter methodology in kernel geometry may continue to bifurcate: one branch centered on RKHS means and MMD, the other on kernelized quantile and transport-like structures.

In its most compact form, barycenter kernel MMD is the principle that a probability measure can be summarized or optimized by matching kernel mean embeddings, and that barycentric combination is therefore linear in RKHS even when the realization of the resulting object as a concrete distribution, a sparse particle system, or a computationally stable algorithm is highly nontrivial (Belhadji et al., 14 Feb 2025).