Measure-Based Diffusion Kernels

• Measure-based diffusion kernels are a generalization of classical diffusion maps that incorporate probability measures to robustly capture both geometric and statistical properties.
• They enable efficient computation of diffusion distances and embeddings, handling challenges such as non-uniform sampling, sparse data, and complex PDEs with memory effects.
• Leveraging Gaussian mixture models and optimal transport techniques, these kernels provide scalable and precise embedding methods across various data domains.

Measure-based diffusion kernels generalize the classical diffusion map framework by incorporating probability measures or densities into the kernel construction, enabling analysis on data domains where classical manifold assumptions are inadequate. Rather than relying solely on pointwise notions of proximity or on uniformly sampled manifolds, these kernels encode similarities by integrating over reference measures or sampling densities, yielding kernels and associated Markov operators that capture both geometric and statistical structure. This measure-centric perspective enables robust handling of non-uniform sampling, sparse data, scalable computation, and even PDEs with memory effects. The approach has a precise mathematical characterization and computational procedures, with guarantees on diffusion distance preservation, scale invariance, and efficient embedding.

1. Foundations: Definition and Decomposition

Let $X\subset\mathbb{R}^m$ be a data domain, and $M\subset\mathbb{R}^m$ a reference domain equipped with probability density $q(r)$. The canonical measure-based Gaussian-correlation (MGC) kernel is

$$k_\varepsilon(x,y) = \int_M g_m(r; x, \tfrac{\varepsilon}{2}I_m)\;g_m(r; y, \tfrac{\varepsilon}{2}I_m)\;q(r)\,dr,$$

where $g_m(r;\theta,\Sigma)$ is the multivariate Gaussian density at $r$ with mean $\theta$ and covariance $\Sigma$. This construction integrates (in both measure and geometric sense) the pointwise affinity over the distribution $q$.

For densities $q$ which admit Gaussian mixture expansions,

$$q(r) = \sum_{j=1}^n a_j\,g_m(r; \theta_j, \Sigma_j),$$

the kernel admits a closed decomposition ("measure/data factorization"),

$$k_\varepsilon(x, y) = \sum_{j=1}^n a_j\,g_m(x; \theta_j, \tfrac{1}{2}I_m+\Sigma_j)\,g_m(y; c_j(x), D_j),$$

with explicit formulas for $c_j(x)$ and $D_j$. This separation renders the dependence on the measure and point explicit and tractable (Salhov et al., 2015).

The Markov operator is then normalized via

$$p_\varepsilon(x,y) = \frac{k_\varepsilon(x,y)}{\nu_\varepsilon(x)}, \quad \nu_\varepsilon(x) = \int_X k_\varepsilon(x, y)\,dy,$$

where $\nu_\varepsilon$ serves as the stationary "local volume".

2. Associated Diffusion Distances and Invariance

The nonnegative, symmetric measure-based kernels yield Markov chains whose $t$-step diffusion distances are

$$d_\varepsilon^{(t)}(x, z) = \|\;p_\varepsilon^{(t)}(x, \cdot) - p_\varepsilon^{(t)}(z, \cdot)\|_{L^2(X)}$$

and, importantly for $t=1$,

$$d_\varepsilon^{(1)}(x, z) = \sqrt{ \frac{W_{x,x}}{\nu_\varepsilon(x)^2} + \frac{W_{z,z}}{\nu_\varepsilon(z)^2} - 2\frac{W_{x,z}}{\nu_\varepsilon(x)\nu_\varepsilon(z)} }$$

with $$W_{x,z} = \int_X k_\varepsilon(x, y) k_\varepsilon(z, y)\,dy$$. These quantities depend only on marginalizations over $q$ and parameter tuples of the Gaussian mixture, not the raw dataset. The diffusion distance is independent of dataset cardinality and, with appropriate scaling, invariant to rescalings of $X$ (Salhov et al., 2015).

3. Closed-Form Embedding and Computational Complexity

Lemma 4.1 in (Salhov et al., 2015) establishes that the $L^2$ diffusion distance admits an exact embedding as the Euclidean distance between infinite-dimensional feature maps (composed of Gaussian/Taylor stacks constructed from mixture parameters). This embedding is

$\|f(x) - f(z)\|_2 = d_\varepsilon^{(1)}(x, z),$

which obviates the need for explicit eigen-decomposition and enables direct computation. For GMM order $n$ and $l$-term Taylor truncation, embedding a new point incurs $O(n\,m^l)$ complexity, independent of the dataset size $|X|$. No large $N\times N$ kernel matrices are formed: all computations scale with the number of components, not samples (Salhov et al., 2015).

4. Measures, Normalizations, and Theoretical Extensions

Bi-Stochastic and Optimal Transport Normalizations

A pivotal line of development generalizes the measure-based kernel construction via bi-stochastic normalization: $$K_\varepsilon(x, y) = \frac{k_\varepsilon(x, y)}{d(x)\,d(y)},$$ where $d$ is chosen so $K_\varepsilon$ is bi-stochastic with respect to a specified measure $\nu$, i.e.,

$\int_M K_\varepsilon(x, y)\,d\nu(y) = 1.$

The existence and smoothness of such $d$ is guaranteed for $k_\varepsilon$ of the required class (Marshall et al., 2017). The diffusion generator in the $\varepsilon \to 0$ limit becomes a weighted Laplacian whose structure depends on both the sampling measure and the choice of reference $\nu$.

Recent work employs symmetric Sinkhorn normalization from optimal transport to scale general positive symmetric kernels (and measures) to ensure mass preservation, symmetricity, spectrum in $[0,1]$, and other "diffusion-like" properties, even when standard geometric assumptions fail. The resulting normalized operator $Q=\Lambda K M \Lambda$ converges to a continuous kernel operator under mesh refinement, and the construction is validated across point clouds, meshes, voxel grids, and Gaussian mixtures (Kessler et al., 8 Jul 2025).

Variable-Bandwidth and Adaptive Kernels

Variable-bandwidth kernels select the bandwidth function $\rho(x)\approx q(x)^\beta$ to control local scaling according to density, yielding

$$K^S_\varepsilon(x, y) = h\Bigl( \frac{ \|x - y\|^2 }{ \varepsilon\,\rho(x)\,\rho(y) } \Bigr ),$$

with rigorous asymptotic expansions for the limiting (local) generator, error bounds, and Monte Carlo variance (Berry et al., 2014). For $\beta<0$, errors remain uniformly bounded even in regions of vanishing density—a property that fixed-bandwidth kernels lack on noncompact domains or with highly nonuniform sampling.

5. Measure-based Kernels Beyond Geometry: Applications and Extensions

Compression and Efficient Embeddings

The MGC kernel can support region-based compression. Aggregating the kernel over moderately large data partitions enables a compressed Markov walk between "regions", which can then be used for region-level embeddings, and fine-grained pointwise embeddings recovered via interpolation. This enables manifold/fiber geometry recovery for very large-scale datasets with significant gains in efficiency and negligible geometric distortion, provided regions are "local" in the diffusion geometry (Gigante et al., 2019).

Data-Driven Inference and PDEs with Memory

In mean-field limits and inverse problems, the measure-based (mean-field) kernel framework enables parametric identification of interaction/diffusion kernels from empirical trajectory data. Regression is performed against empirical measures so that kernel identification is stable with respect to sampling and ergodic in long-time averages (Albi et al., 16 Mar 2026).

In PDE contexts, measure-valued kernels $\mu$ allow encoding of memory and delay dynamics: $$\partial_t u(t) + A_0 u(t) + (\mathcal{K}_\mu u)(t) = f(t),$$ where $$(\mathcal{K}_\mu u)(t) = \int_{(0,t]} A_1 u(t-s)\,d\mu(s)$$ may include distributed, discrete, or mixed memory/delay effects. Existence, uniqueness, stability, and refined energy inequalities are proved under coercivity and positivity conditions on $A_0, A_1$ and $\mu$. For completely monotone $\mu$, internal-variable representations and energy dissipation mechanisms are derived (Ishizaka, 22 Feb 2026).

6. Empirical Illustration and Practical Implementation

Measure-based diffusion kernels have been validated:

• On synthetic multimodal mixtures, with GMM parameterization, closed-form measure-based kernels reproduce stationary distributions and diffusion distances to small $\lt1\%$ error (Salhov et al., 2015).
• In large-scale cytometry, compressed (region-based) kernels yield manifold embeddings visually and quantitatively indistinguishable from full diffusion maps, but at order-of-magnitude lower computation (Gigante et al., 2019).
• In spectral geometry tasks, normalized operators preserve Laplacian eigenstructure across unstructured, irregular, or mixture domains, both in spectrum and eigenfunctions (Kessler et al., 8 Jul 2025).
• In variable-bandwidth graph Laplacians, choice of $\beta$ and normalization achieves robust error bounds and insensitivity to kernel scale even under noncompactness or non-uniformity (Berry et al., 2014).
• In particle-to-mean-field systems, both random-batch and mean-field regression strategies are used to learn $\mathcal{D}$ from time-series data, achieving kernel errors $10^{-2}-10^{-1}$ even for unobserved pairs (Albi et al., 16 Mar 2026).

7. Connections, Generalizations, and Current Research Directions

Measure-based diffusion kernels unify kernelized geometry, optimal transport, sparsity-refined regression, graph Laplacians, weighted manifold learning, and even model PDEs with historical/heterogeneous structure. They admit:

• Robust transfer to discrete, non-Euclidean, and composite data domains.
• Spectrally consistent normalization via Sinkhorn and related scaling for both theoretical and computational tractability.
• Extensions to reproducing kernel Hilbert space (RKHS) settings, where Laplacian and diffusion operators are estimated in nonparametric functional spaces with non-asymptotic, dimension-free error rates (Pillaud-Vivien et al., 2023).

Ongoing research explores scalable stochastic and region-based approximations, RKHS-based Laplacian learning for high-dimensional manifold and clustering tasks, integration with optimal transport for regularization, and novel applications in functional connectivity and stochastic PDEs with memory (Salhov et al., 2015, Kessler et al., 8 Jul 2025, Gigante et al., 2019, Berry et al., 2014, Marshall et al., 2017, Pillaud-Vivien et al., 2023, Ishizaka, 22 Feb 2026, Tung et al., 2021, Albi et al., 16 Mar 2026).

---

Key references: "Diffusion Representations" (Salhov et al., 2015), "Compressed Diffusion" (Gigante et al., 2019), "Normalizing Diffusion Kernels with Optimal Transport" (Kessler et al., 8 Jul 2025), "Variable Bandwidth Diffusion Kernels" (Berry et al., 2014), "Manifold learning with bi-stochastic kernels" (Marshall et al., 2017), "A unified theory for diffusion with memory and delay via measure-valued kernels" (Ishizaka, 22 Feb 2026), "Discovery of interaction and diffusion kernels in particle-to-mean-field multi-agent systems" (Albi et al., 16 Mar 2026), "Kernelized Diffusion Maps" (Pillaud-Vivien et al., 2023).