Sliced Wasserstein Kernels

• Sliced Wasserstein Kernels are positive-definite kernels derived by averaging one-dimensional Wasserstein distances over projections, capturing the geometric structure of distributions.
• They are constructed using a distance-substitution approach with Gaussian or Laplacian functions on Hilbertian metrics, ensuring both computational efficiency and universality.
• Empirical results show these kernels excel in tasks like image histogram classification, graph learning, and audio captioning, offering improved accuracy and robustness.

The Sliced Wasserstein kernel is a family of positive-definite kernels constructed from the Sliced Wasserstein (SW) distance—a measure derived from optimal transport that computes the average one-dimensional Wasserstein distance between projections (“slices”) of high-dimensional probability measures. Sliced Wasserstein kernels inherit geometric sensitivity from the Wasserstein distance, computational efficiency via closed-form 1D transport, and universality on the space of probability distributions, enabling effective learning over distributional inputs in kernel-based machine learning frameworks.

1. Definition and Mathematical Foundations

Let $\mu,\nu$ be probability measures on a compact domain $\Omega\subset\mathbb{R}^d$. For each direction $\theta\in S^{d-1}$ (the unit sphere), define the push-forward projections $\mu_\theta = \theta^*_{\sharp}\mu$, where $\theta^*(x) = \langle\theta, x\rangle$. The one-dimensional $p$-Wasserstein distance between $\mu_\theta$ and $\nu_\theta$ admits the explicit formula: $$W_p(\mu_\theta,\nu_\theta) = \|F_{\mu_\theta}^{[-1]} - F_{\nu_\theta}^{[-1]}\|_{L^p([0,1])}$$ where $F_\mu^{[-1]}$ is the generalized inverse CDF.

The Sliced Wasserstein distance of order $\Omega\subset\mathbb{R}^d$0 is then

$\Omega\subset\mathbb{R}^d$1

for the uniform measure $\Omega\subset\mathbb{R}^d$2 on $\Omega\subset\mathbb{R}^d$3.

For $\Omega\subset\mathbb{R}^d$4, the feature map

$\Omega\subset\mathbb{R}^d$5

exhibits the Hilbertian structure: $\Omega\subset\mathbb{R}^d$6 Thus, $\Omega\subset\mathbb{R}^d$7 is a genuine metric metrizing weak convergence on $\Omega\subset\mathbb{R}^d$8 and allows for the construction of positive-definite (p.d.) kernels (Meunier et al., 2022, Kolouri et al., 2015).

2. Construction and Universality of Sliced Wasserstein Kernels

A distance-substitution kernel is constructed using a monotonically decreasing function $\Omega\subset\mathbb{R}^d$9 (e.g., Gaussian/Laplacian RBF) and a Hilbertian distance $\theta\in S^{d-1}$0: $\theta\in S^{d-1}$1 For SW, this leads to the Gaussian and Laplacian-style kernels: $\theta\in S^{d-1}$2 Schoenberg’s theorem guarantees positive definiteness when $\theta\in S^{d-1}$3 is (conditionally) negative-definite. By the Hilbertian property of $\theta\in S^{d-1}$4, the resultant kernels are p.d.

When the underlying space $\theta\in S^{d-1}$5 is compact and $\theta\in S^{d-1}$6 metrizes the weak convergence, the corresponding Gaussian-type kernel is universal: its RKHS is dense in $\theta\in S^{d-1}$7, enabling universal consistency in regression and classification (Meunier et al., 2022).

3. Algorithmic Implementation and Computational Considerations

Approximating $\theta\in S^{d-1}$8 involves:

• Sampling $\theta\in S^{d-1}$9 directions $\mu_\theta = \theta^*_{\sharp}\mu$0
• Sampling $\mu_\theta = \theta^*_{\sharp}\mu$1 grid points $\mu_\theta = \theta^*_{\sharp}\mu$2

For a measure $\mu_\theta = \theta^*_{\sharp}\mu$3, the feature vector in $\mu_\theta = \theta^*_{\sharp}\mu$4 is: $\mu_\theta = \theta^*_{\sharp}\mu$5 Computing SW kernel matrix entries involves sorting projected samples (cost $\mu_\theta = \theta^*_{\sharp}\mu$6 per direction, for empirical measures with $\mu_\theta = \theta^*_{\sharp}\mu$7 points), feature evaluation ($\mu_\theta = \theta^*_{\sharp}\mu$8), and matrix evaluation ($\mu_\theta = \theta^*_{\sharp}\mu$9 for $\theta^*(x) = \langle\theta, x\rangle$0 distributions).

Empirically, setting $\theta^*(x) = \langle\theta, x\rangle$1 yields robust estimates. The kernel bandwidth $\theta^*(x) = \langle\theta, x\rangle$2 is typically tuned via cross-validation (Meunier et al., 2022, Kolouri et al., 2015, Luong et al., 8 Feb 2025).

Monte-Carlo integration is necessary; the kernel approximation error decreases as $\theta^*(x) = \langle\theta, x\rangle$3. Unbiased variants further reduce estimation bias by averaging exponentials, enabling direct unbiased stochastic optimization (Luong et al., 8 Feb 2025).

4. Theoretical Guarantees: Consistency and Excess Risk

For kernel ridge regression (KRR) with SW kernels on empirical measures $\theta^*(x) = \langle\theta, x\rangle$4 and associated responses $\theta^*(x) = \langle\theta, x\rangle$5: $\theta^*(x) = \langle\theta, x\rangle$6 the estimator interpolates via the Gram matrix $\theta^*(x) = \langle\theta, x\rangle$7 with entries $\theta^*(x) = \langle\theta, x\rangle$8.

Under the assumption that $\theta^*(x) = \langle\theta, x\rangle$9 is universal and Hölder-continuous with respect to $p$0, the excess risk admits a bound that separates

• Stage-1 sampling complexity ($p$1 bags)
• Stage-2 sampling complexity ($p$2 samples/bag)
• The effective kernel dimension, and
• The regularization parameter $p$3

Concretely, if $p$4 and the kernel/Lipschitz constants are suitably chosen,

$p$5

Universal consistency follows as $p$6 (Meunier et al., 2022).

5. Empirical Performance and Use Cases

SW kernels outperform MMD-based and standard RBF kernels in synthetic and real-world tasks where geometric structure is salient:

• Mode-counting (synthetic Gaussians): SW kernels yield 20–30% lower RMSE than MMD on predicting cluster count from samples.
• Image histogram classification: On MNIST and Fashion-MNIST (raw and perturbed), $p$7 achieves $p$8 accuracy vs. $p$9 (MMD) and $\mu_\theta$0 (RBF); under transformations, SW kernels maintain higher accuracy and robustness (Meunier et al., 2022).
• Graph learning: Sliced Wasserstein Weisfeiler-Lehman (SWWL) graph kernels allow for positive definite, scalable, and accurate graph similarity measures, handling datasets with $\mu_\theta$1–$\mu_\theta$2 nodes efficiently (Perez et al., 2024).
• Audio captioning: Unbiased SW-RBF kernels with temporal augmentations improve alignment and generation fidelity under stochastic sampling (Luong et al., 8 Feb 2025).

SW kernels are especially effective when mass-transport geometry correlates with the learning target, e.g., mode structure, geometric invariance, and robustness to perturbations.

6. Variants and Extensions

Max-Sliced and Tree-Sliced Wasserstein Kernels

• Max-sliced Wasserstein replaces the averaging over directions by maximization—optimizing a (potentially nonlinear) projection to maximize 1D Wasserstein between projected measures. While achieving sharper discrimination and dimension-free statistical rates, the projection optimization is NP-hard for $\mu_\theta$3 but admits tight semidefinite relaxations (Wang et al., 2024).
• Tree-sliced Wasserstein kernels generalize the slice to arbitrary random tree metrics rather than lines (chains), averaging per-tree closed-form transports. This allows for flexible embedding of geometric structure and scalable, positive-definite kernels (Le et al., 2019).

Unbiased and Monte-Carlo Approximations

• The construction of unbiased estimators for the kernel value via Monte Carlo is crucial for stochastic gradient optimization and reducing estimator variance, with convergence rates $\mu_\theta$4 with $\mu_\theta$5 samples (Luong et al., 8 Feb 2025).

7. Practical Guidelines and Limitations

When to use SW kernels:

• For regression or classification where output depends on underlying geometry of distributions or empirical measures.
• For problems requiring robustness to support mismatch or geometric transformations (e.g., histogram classification, graph regression, multi-modality).
• When computational efficiency is critical: 1D transport is closed-form and scalable.

Limitations:

• MC estimation introduces additional stochastic error (i.e., three-stage sampling in distribution regression: bag, within-bag, and MC).
• Convergence rates for distribution regression under SW are slower in worst-case theory ($\mu_\theta$6) compared to MMD ($\mu_\theta$7), but empirical geometry often compensates.
• Exact SW kernel evaluation is infeasible in very high-dimensional settings unless $\mu_\theta$8, $\mu_\theta$9 are chosen adaptively.

Parameter tuning:

• Choose $\nu_\theta$0 (projections/quantiles) to balance accuracy and efficiency.
• Kernel bandwidths (e.g., $\nu_\theta$1) via cross-validation or median heuristics.

Sliced Wasserstein kernels fundamentally bridge geometric optimal transport and positive-definite kernel-based learning, providing a computationally tractable class of universal kernels well-suited for distributional data (Meunier et al., 2022, Kolouri et al., 2015, Perez et al., 2024, Luong et al., 8 Feb 2025, Le et al., 2019).