Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows (2504.07820v1)

Abstract: Negative distance kernels $K(x,y) := - |x-y|$ were used in the definition of maximum mean discrepancies (MMDs) in statistics and lead to favorable numerical results in various applications. In particular, so-called slicing techniques for handling high-dimensional kernel summations profit from the simple parameter-free structure of the distance kernel. However, due to its non-smoothness in $x=y$, most of the classical theoretical results, e.g. on Wasserstein gradient flows of the corresponding MMD functional do not longer hold true. In this paper, we propose a new kernel which keeps the favorable properties of the negative distance kernel as being conditionally positive definite of order one with a nearly linear increase towards infinity and a simple slicing structure, but is Lipschitz differentiable now. Our construction is based on a simple 1D smoothing procedure of the absolute value function followed by a Riemann-Liouville fractional integral transform. Numerical results demonstrate that the new kernel performs similarly well as the negative distance kernel in gradient descent methods, but now with theoretical guarantees.

Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows

This paper introduces a new approach to smoothing distance kernels for use in maximum mean discrepancies (MMDs) and Wasserstein gradient flows. The authors propose a kernel that retains the computational efficiency of negative distance kernels but also offers theoretical advantages such as Lipschitz differentiability and positive definiteness.

Motivation and Background

In machine learning and statistics, kernels are often used in methods like MMDs for comparing probability distributions. Negative distance kernels, defined as $K(x, y) = -\|x - y\|$, have shown promising numerical results due to their simplicity and parameter-free structure. However, traditional negative distance kernels lack smoothness at zero (i.e., when $x = y$), complicating their use in Wasserstein gradient flows, where smoothness is desirable for theoretical consistency and convergence guarantees.

Proposed Method

The authors construct a smoothed kernel that preserves the attributes of a negative distance kernel, notably being conditionally positive definite of order one. Their approach is based on smoothing the absolute value function, traditionally part of the negative distance representation, through convolution with a suitable filter and further applying the Riemann-Liouville fractional integral transform.

1. Smoothing the Absolute Value Function: The paper introduces convolution with functions from $\mathcal{U}^n(\mathbb{R})$, a class of functions defined for better computational and analytical handling. These smoothed absolute value functions maintain beneficial properties but gain differentiability, giving rise to the smoothness necessary for stable gradient flows.
2. Prospective Application with Gradient Flows: In Wasserstein gradient flows, the inability of negative distance kernels to differentiate properly at zero limits their theoretical integration into certain algorithms. The new smoothed kernel overcomes this by presenting a function $F(x)$ that is Lipschitz differentiable, thereby enabling consistent gradient flow computations and maintaining positive definiteness, crucial for maintaining the RKHS structure in the resulting functional space.

Theoretical Contributions

• Lipschitz Differentiability: The new kernel aids in defining a well-behaved landscape for Wasserstein gradient flows. Such differentiability ensures continuity and boundedness of derivatives, properties vital for the convergence of numerical optimization schemes like gradient descent.
• Conditional Positive Definiteness: By ensuring the smoothed kernel retains conditional positive definiteness, the paper solidifies that the kernel can be used effectively for MMD, guaranteeing that the associated RKHS properties hold, and thus the MMD retains its distance-like properties.

Numerical Experiments and Applications

Numerical demonstrations exhibit the capability of the smoothed kernels to achieve comparable numerical performance to traditional negative distance kernels in Wasserstein gradient flows. The experiments further showcase that traditional problems such as oscillations and lack of convergence in explicit schemes are mitigated.

Implications and Future Work

The introduction of smoothed distance kernels opens new avenues for utilizing MMDs and Wasserstein gradient flows more reliably, particularly in high-dimensional applications such as generative modeling and data comparisons. These kernels reduce the gap between computational efficiency and theoretical soundness, potentially prompting further research into expanding these concepts to broader classes of machine learning problems.

Future developments could explore parameter tuning, automated selection of smoothing parameters, and the integration of these kernels into deep learning frameworks. Investigations might also consider the extension to more complex data structures and the implications of such kernels in Bayesian inference methodologies, such as Stein variational gradient descent, where traditional kernels face limitations.

Overall, this paper contributes a theoretically robust method that enhances the practical deployment of MMDs, offering significant computational and theoretical benefits.