---
title: Barycenter Kernel MMD
url: https://www.emergentmind.com/topics/barycenter-kernel-maximum-mean-discrepancy-mmd
type: topic
---

# Barycenter Kernel MMD

Barycenter Kernel Maximum Mean Discrepancy (MMD) denotes a class of constructions in which probability measures are compared, summarized, or optimized through their reproducing kernel Hilbert space (RKHS) mean embeddings, and barycentric structure is expressed in that feature-space geometry rather than through transport in the original sample space. In the single-target setting, a discrete weighted measure \(\nu_{Y,w}=\sum_{i=1}^M w_i\delta_{y_i}\) is sought so that its mean embedding is close to that of a target \(\pi\), yielding a “kernel MMD barycenter approximation” of \(\pi\). In the multi-distribution setting, one minimizes weighted sums of squared MMDs, and the RKHS barycenter is the weighted sum of the constituent mean embeddings [2502.10600]. This viewpoint places barycenters, quantization, quadrature, and kernel two-sample methodology inside a common Hilbertian framework, while the mathematical meaning of the induced geometry depends critically on whether the chosen kernel makes MMD a genuine metric that metrizes weak convergence [2006.09268].

## 1. RKHS formulation of barycentric MMD

For a positive definite kernel \(k\) with RKHS \(\mathcal H_k\), the kernel mean embedding of a probability measure \(\mu\) is
\[
m_\mu = \int k(\cdot,x)\,d\mu(x),
\]
and the maximum mean discrepancy is the RKHS norm distance
\[
\mathrm{MMD}(\mu,\nu)=\|m_\mu-m_\nu\|_{\mathcal H_k}.
\]
Equivalently,
\[
\mathrm{MMD}_k^2(\mu,\nu)
=
\mathbb E_{X,X'\sim\mu}[k(X,X')]
-2\mathbb E_{X\sim\mu,Y\sim\nu}[k(X,Y)]
+\mathbb E_{Y,Y'\sim\nu}[k(Y,Y')].
\]
These identities make barycentric formulations immediate: a probability measure is represented by a single element of \(\mathcal H_k\), and barycenters are defined by Hilbert-space averaging of such representations [2502.10600].

In the discrete approximation setting, the target \(\pi\) is approximated by
\[
\nu_{Y,w}=\sum_{i=1}^M w_i\delta_{y_i},
\]
with feature-space representation
\[
m_{\nu_{Y,w}}=\sum_{i=1}^M w_i\,k(\cdot,y_i).
\]
Seeking \(\nu_{Y,w}\) such that \(\mathrm{MMD}(\pi,\nu_{Y,w})\) is small is therefore exactly the problem of finding a finite barycentric summary whose mean embedding is close to \(m_\pi\) [2502.10600].

For several input measures \(\pi_1,\dots,\pi_K\) with weights \(\lambda_k\ge 0\), \(\sum_k\lambda_k=1\), the natural objective is
\[
\mu^\star \in \arg\min_{\mu\in\mathcal M_+(\mathcal X)}
\sum_{k=1}^K \lambda_k\,\mathrm{MMD}^2(\mu,\pi_k).
\]
The objective is minimized when
\[
m_\mu=\sum_{k=1}^K \lambda_k m_{\pi_k},
\]
so the barycenter in RKHS is simply the weighted sum of mean embeddings [2502.10600]. In that sense, “barycenter kernel MMD” is fundamentally a feature-space barycenter construction.

## 2. Discrete barycenters, weighted quantization, and support constraints

A central finite-dimensional formulation is
\[
F(\mu):=\tfrac12\,\mathrm{MMD}^2(\mu,\pi),
\qquad
F_M(Y):=
\inf_{w\in\mathbb R^M}
F\!\left[\sum_{i=1}^M w_i\delta_{y_i}\right].
\]
With \(\bm K(Y)=(k(y_i,y_j))_{ij}\) and \(\bm v(Y)=(v_0(y_i))_i\), where
\[
v_0(y)=\int k(x,y)\,d\pi(x),
\]
the squared discrepancy satisfies
\[
\mathrm{MMD}^2(\pi,\nu_{Y,w})
=
C_\pi-2\bm w^\top\bm v(Y)+\bm w^\top\bm K(Y)\bm w.
\]
For fixed support \(Y\), the optimal weights are
\[
\hat{\bm w}(Y)=\bm K(Y)^{-1}\bm v(Y),
\]
and the reduced objective becomes
\[
F_M(Y)
=
\tfrac12\left(C_\pi-\bm v(Y)^\top \bm K(Y)^{-1}\bm v(Y)\right)
\]
[2502.10600].

This formulation differs from classical \(W_2\)-quantization in two ways emphasized in the literature. First, the discrepancy is MMD rather than Wasserstein distance. Second, the weights are not constrained to the simplex; they are allowed to lie in \(\mathbb R^M\), which is a relaxation of classical Wasserstein quantization [2502.10600]. The resulting discrete measure is simultaneously a quantizer, a clustering-type representation, and an RKHS barycenter approximation.

The same structure appears in kernel quadrature and Bayesian quadrature. For fixed nodes \(x_1,\dots,x_N\), kernel quadrature chooses weights by minimizing
\[
\mathrm{MMD}_k\!\left(\sum_{n=1}^N w_n\delta_{x_n},P\right),
\]
and when the kernel matrix is invertible the posterior variance of Bayesian quadrature is exactly the squared MMD between the quadrature measure and \(P\) [2602.21846]. In this restricted-support sense, kernel quadrature is a constrained barycenter problem in MMD geometry.

## 3. Metric meaning and topological conditions

The usefulness of barycenter constructions depends on whether MMD defines a meaningful geometry on probability measures. For bounded continuous kernels, kernel mean embeddings
\[
f_\mu(\cdot)=\int_X k(\cdot,x)\,d\mu(x)
\]
exist for finite regular Borel measures, and
\[
d_k(\mu,\nu)=\|f_\mu-f_\nu\|_{\mathcal H_k}
\]
is the induced MMD distance [2006.09268].

On compact Hausdorff spaces, a bounded Borel measurable kernel metrizes weak convergence of probability measures if and only if it is continuous and characteristic to \(\mathcal P\). On locally compact, non-compact Hausdorff spaces, assuming \(\mathcal H_k\subset C_0(X)\), the correct condition is stronger: \(k\) metrizes weak convergence if and only if it is continuous and integrally strictly positive definite over all signed, finite, regular Borel measures [2006.09268]. These conditions are foundational for barycenter problems in MMD space because they ensure that convergence in MMD coincides with the standard probabilistic notion of weak convergence.

This distinction matters in non-compact spaces. If the kernel fails the relevant condition, one can have sequences that converge in MMD while mass escapes to infinity, which undermines barycentric interpretations at the level of distributions [2006.09268]. By contrast, for standard kernels such as Gaussian, Laplace, Matérn, and inverse multiquadratic on \(\mathbb R^d\), the condition \(\mathcal H_k\subset C_0(X)\) is satisfied, and the kernels commonly used in practice fall into the regime where MMD can serve as a probabilistically meaningful geometry [2006.09268].

A complementary observation is that MMD barycenters are trivial in RKHS but not necessarily trivial as measures. The embedding barycenter is the weighted mean of embeddings; realizing that point as an actual probability measure, or as a discrete approximation with prescribed support size, is the nontrivial part of the problem [2502.10600; 2602.21846].

## 4. Gradient flows and particle algorithms

One line of work studies barycentric MMD optimization dynamically through gradient flows. For
\[
F(\mu)=\tfrac12\,\mathrm{MMD}^2(\mu,\pi),
\]
the Wasserstein gradient flow of MMD satisfies the continuity equation
\[
\partial_t \nu_t = \mathrm{div}\!\left(\nu_t \nabla f_{\mu,\nu_t}\right),
\]
where the first variation is the witness function. Along this flow,
\[
\frac{d}{dt}\mathcal F(\nu_t)
=
-\int \|\nabla f_{\mu,\nu_t}(x)\|^2\,d\nu_t(x)
\le 0,
\]
so the MMD energy is non-increasing [1906.04370].

For weighted discrete barycenters, a more suitable geometry is Wasserstein–Fisher–Rao (WFR), because WFR allows mass creation and destruction. The associated PDE is
\[
\partial_t \mu_t
=
\alpha\,\mathrm{div}(\mu_t\nabla[\mu_t])-[\mu_t]\mu_t,
\]
where the reaction term enables weights to adapt rather than remain simplex-constrained [2502.10600]. Under a discrete ansatz
\[
\mu_t=\sum_{i=1}^M w_i(t)\delta_{y_i(t)},
\]
this yields the interacting ODE system
\[
\dot y_i(t)
=
-\alpha\Big(\sum_{m=1}^M w_m(t)\nabla_2 k(y_m(t),y_i(t))-\nabla v_0(y_i(t))\Big),
\]
\[
\dot w_i(t)
=
-w_i(t)\Big(\sum_{m=1}^M w_m(t)k(y_m(t),y_i(t))-v_0(y_i(t))\Big).
\]
The location dynamics decrease MMD through kernel gradients, while the weight dynamics increase or decrease particle mass according to local over- or under-representation in feature space [2502.10600].

Stationary conditions for this WFR system lead to the fixed-point map called Mean Shift Interacting Particles (MSIP),
\[
Y=\Psi_{\mathrm{MSIP}}(Y),
\]
which extends classical mean shift from \(M=1\) to interacting multi-particle barycenters. Under the kernel-gradient structure \(\nabla_2 k(x,y)=(x-y)\bar\kappa(x,y)\), the gradient of the reduced objective satisfies
\[
\nabla F_M(Y)
=
\bm W(Y)\bm{\bar K}(Y)\bm W(Y)\big(Y-\Psi_{\mathrm{MSIP}}(Y)\big),
\]
so MSIP is exactly a preconditioned gradient descent method. The paper further identifies MSIP as an exact MMD-side analog of Lloyd’s algorithm [2502.10600].

A distinct numerical realization is EVI-MMD, which interprets MMD as a free energy and applies an implicit Euler discretization. Each time step solves a proximal problem
\[
\{x_i^{(n+1)}\}
=
\arg\min_{\{x_i\}}
\left[
\frac{1}{2\tau^\ast N}\sum_{i=1}^N \|x_i-x_i^{(n)}\|_2^2
+
\mathcal F_{h_n}^\ast(\{x_i\})
\right],
\]
typically by L-BFGS, and uses an adaptive Gaussian bandwidth
\[
h_n=\frac{a}{n^c}+b
\]
to interpolate between coarse exploration and fine exploitation [2111.10722].

## 5. Kernel-side aggregation and related constructions

Barycentric structure also appears on the kernel side. Because MMD is linear in the kernel,
\[
\mathrm{MMD}^2_k(P,Q)=\sum_{\ell} w_\ell\,\mathrm{MMD}^2_{k_\ell}(P,Q)
\]
when \(k=\sum_\ell w_\ell k_\ell\). This makes multi-scale Gaussian mixtures a natural way to define composite MMD geometries, although scale normalization can be important because raw MMD values vary substantially with bandwidth, dimension, and batch size [1901.03227].

A more general aggregation strategy is to combine several single-kernel MMD estimates through their Mahalanobis distance. For a kernel family \(K=\{k_1,\dots,k_r\}\), let
\[
\mathrm{MMD}^2[K,X_m,Y_n]
=
\big(\widehat{\mathrm{MMD}^2}_{k_1},\dots,\widehat{\mathrm{MMD}^2}_{k_r}\big)^\top.
\]
Then the Mahalanobis-aggregated statistic is
\[
T_{m,n}
=
\mathrm{MMD}^2[K,X_m,Y_n]^\top
\widehat\Sigma^{-1}
\mathrm{MMD}^2[K,X_m,Y_n],
\]
where \(\widehat\Sigma\) estimates the joint null covariance of the MMD vector. This aggregation is universally consistent, has non-trivial asymptotic Pitman efficiency under local alternatives, and is more powerful in finite samples across a range of alternatives than single-kernel tests in the reported experiments [2302.10687]. Although this is a testing construction rather than a distributional barycenter, it is another instance of covariance-aware barycentric combination in MMD space.

Random-feature kernel learning yields a related perspective. A kernel can be written as
\[
K_\mu(x,y)
=
\int \varphi(x;\xi)\varphi(y;\xi)\,\mu(d\xi),
\]
so learning the distribution \(\mu\) over random features amounts to learning a mixture, or empirical barycenter, of rank-one base kernels \(\varphi(x;\xi)\varphi(y;\xi)\). In the mean-field limit, the empirical distribution of the feature particles follows a Wasserstein gradient flow on the space of feature distributions [1909.11820].

## 6. Applications, limitations, and broader landscape

Barycenter kernel MMD constructions appear in several applied forms. In weighted quantization they provide clustering- and quadrature-type summaries of a target distribution with adaptive locations and weights [2502.10600]. In simulation-based inference, optimally weighted MMD estimators use Bayesian quadrature weights to approximate \(\mathrm{MMD}_k(P_\theta,Q)\) more efficiently than standard empirical estimators under smoothness assumptions [2602.21846]. In deterministic sampling and generative modeling, particle systems are evolved so that the empirical measure minimizes MMD to a target distribution or to empirical data [2111.10722]. In Wasserstein auto-encoders, Gaussian-kernel MMD admits closed forms against the standard normal prior, and multi-kernel mixtures can be standardized componentwise to avoid scale domination [1901.03227].

A persistent misconception is to identify MMD barycenters with Wasserstein barycenters. The two geometries are different. Under MMD, each distribution is represented by a point in an RKHS, and the barycenter is the Euclidean barycenter of those points in feature space. Under Wasserstein \(W_2\), the geometry is based on transporting mass in the original space, and the barycenter minimizes average transport cost [1901.03227]. The analogy between MSIP and Lloyd’s algorithm is structural, not geometric: both are preconditioned fixed-point methods, but one operates in RKHS-induced MMD geometry and the other in Euclidean transport geometry [2502.10600].

Several open questions remain explicit. For MSIP, the analytical status of non-degeneracy, descent, local minima versus saddle points, and convergence rates is unresolved [2502.10600]. More broadly, recent work on kernel quantile discrepancies argues that MMD’s reliance on mean embeddings can miss higher-order structure, and proposes quantile-based kernel discrepancies that recover sliced Wasserstein and max-sliced Wasserstein as special cases while retaining near-linear algorithms [2602.21846]. This suggests that barycenter methodology in kernel geometry may continue to bifurcate: one branch centered on RKHS means and MMD, the other on kernelized quantile and transport-like structures.

In its most compact form, barycenter kernel MMD is the principle that a probability measure can be summarized or optimized by matching kernel mean embeddings, and that barycentric combination is therefore linear in RKHS even when the realization of the resulting object as a concrete distribution, a sparse particle system, or a computationally stable algorithm is highly nontrivial [2502.10600].

Source: https://www.emergentmind.com/topics/barycenter-kernel-maximum-mean-discrepancy-mmd