---
title: Kernel Gradient Flow
url: https://www.emergentmind.com/topics/kernel-gradient-flow
type: topic
---

# Kernel Gradient Flow

Kernel gradient flow is a family of measure-valued or function-space gradient systems in which a positive-definite kernel determines either the objective, the metric, the preconditioner, or the approximation of a distributional velocity. The terminology is therefore not unique. In one usage, it denotes a Wasserstein gradient flow of a kernel discrepancy such as squared MMD; in another, it denotes an RKHS-constrained functional gradient flow, as in Stein variational gradient descent (SVGD); in broader formulations, kernel operators modify Fisher–Rao, Wasserstein, or Wasserstein–Fisher–Rao geometries. Across these settings, the central objects are kernel mean embeddings, RKHSs, continuity equations, Onsager operators, and interacting-particle approximations.

## 1. Mathematical foundations and terminology

Let $\Omega\subseteq\mathbb R^d$, let $k:\Omega\times\Omega\to\mathbb R$ be a positive semidefinite kernel, and let $\mathcal H$ be its RKHS. The reproducing property is
\[
g(x)=\langle g,k(x,\cdot)\rangle_{\mathcal H}.
\]
For a probability measure $\nu$, its kernel mean embedding is
\[
\phi_\nu=\int k(\cdot,x)\,d\nu(x)\in\mathcal H.
\]
A kernel is characteristic when $\phi_\nu=\phi_\mu$ implies $\nu=\mu$. In that case,
\[
\operatorname{MMD}(\mu,\nu)=\|\phi_\mu-\phi_\nu\|_{\mathcal H}
\]
is a metric on probability measures and an integral probability metric over the unit ball of $\mathcal H$.

The phrase *kernel gradient flow* can refer to several geometrically distinct constructions:

- **Wasserstein gradient flow of a kernel energy**: the objective is a kernel discrepancy, while the state is a probability measure transported in $W_2$.
- **RKHS or Stein gradient flow**: the objective is commonly KL divergence, while the admissible velocity fields and their norm are determined by an RKHS.
- **Kernelized Fisher–Rao flow**: the objective may be a divergence, but a kernel operator modifies the reaction or mass-creation geometry.
- **Kernel gradient flow in regression**: a function evolves by gradient descent in an RKHS, with continuous-time and discrete-time versions acting as spectral regularization methods.
- **Kernel-induced identification of flows**: an RKHS is used to infer the energy or interaction kernel generating an observed Wasserstein flow.

These meanings should not be conflated. In particular, a Wasserstein flow of MMD is not ordinary gradient descent on the RKHS witness, and SVGD is not a Wasserstein flow with MMD as its energy.

For a general functional $\mathcal F$ on probability measures, a $W_2$ gradient flow has continuity-equation form
\[
\partial_t\nu_t+\nabla\cdot(\nu_t V_t)=0,
\qquad
V_t=-\nabla\frac{\delta\mathcal F}{\delta\nu}(\nu_t).
\]
Its energy dissipation is
\[
\frac{d}{dt}\mathcal F(\nu_t)
=
-\int \|V_t(x)\|^2\,d\nu_t(x).
\]
The kernel enters through either the first variation $\delta\mathcal F/\delta\nu$, the metric acting on that variation, or both.

## 2. Wasserstein flows of kernel discrepancies

For a fixed target measure $\mu$, define the squared MMD energy
\[
\mathcal F(\nu)=\frac12\operatorname{MMD}^2(\mu,\nu)
=\frac12\|\phi_\nu-\phi_\mu\|_{\mathcal H}^2.
\]
The associated witness function is
\[
f_{\mu,\nu}(z)=\phi_\nu(z)-\phi_\mu(z),
\]
up to the sign convention used for the two embeddings. Its first variation is
\[
\frac{\delta\mathcal F}{\delta\nu}(x)=f_{\mu,\nu}(x).
\]
Consequently, the Wasserstein velocity is
\[
V_t(x)=-\nabla f_{\mu,\nu_t}(x),
\]
and the flow equation is
\[
\boxed{
\partial_t\nu_t
=
\nabla\cdot\left(\nu_t\nabla f_{\mu,\nu_t}\right).
}
\]
Equivalently, if $X_t\sim\nu_t$,
\[
dX_t=-\nabla f_{\mu,\nu_t}(X_t)\,dt.
\]
This is a deterministic McKean–Vlasov process: each particle is transported by a field generated jointly by the target and the current law.

The witness gradient has the explicit form
\[
\nabla f_{\mu,\nu}(z)
=
\int\nabla_1 k(z,x)\,d\nu(x)
-
\int\nabla_1 k(z,x)\,d\mu(x).
\]
The energy-dissipation identity is
\[
\boxed{
\frac{d}{dt}\mathcal F(\nu_t)
=
-\int
\|\nabla f_{\mu,\nu_t}(x)\|^2\,d\nu_t(x)
\leq 0.
}
\]
Thus MMD is a Lyapunov functional. However, vanishing dissipation only requires the witness gradient to vanish on the support of $\nu_t$. It does not imply $f_{\mu,\nu_t}=0$ everywhere. Singular measures, including Dirac masses, can therefore be stationary without equaling the target.

The same functional admits a mean-field free-energy representation:
\[
\mathcal F(\nu)
=
\int V(x)\,d\nu(x)
+
\frac12\iint W(x,y)\,d\nu(x)d\nu(y)+C,
\]
where
\[
V(x)=-\int k(x,x')\,d\mu(x'),
\qquad
W(x,y)=k(x,y).
\]
The target generates the potential $V$, while $W$ describes particle interaction.

The MMD energy is convex along mixture paths,
\[
\mathcal F(\lambda\nu+(1-\lambda)\nu')
\leq
\lambda\mathcal F(\nu)+(1-\lambda)\mathcal F(\nu'),
\]
but is generally not displacement convex along $W_2$ geodesics. This prevents direct application of standard globally convergent convex Wasserstein-flow theory. A generalized second-variation analysis introduces a state-dependent curvature quantity $\Lambda(\rho,v)$; its sign need not be nonnegative. The resulting discrete-flow estimate has a barrier term:
\[
\mathcal F(\nu_n)
\leq
\frac{W_2^2(\nu_0,\mu)}{2\gamma n}-\overline K.
\]
When $\overline K\geq0$, this yields an $O(1/n)$ bound; when $\overline K<0$, it is compatible with convergence toward a non-global stationary point [1906.04370].

### Discretization and particles

Forward Euler gives
\[
\nu_{n+1}
=
\left(I-\gamma\nabla f_{\mu,\nu_n}\right)_{\#}\nu_n,
\]
or
\[
X_{n+1}=X_n-\gamma\nabla f_{\mu,\nu_n}(X_n).
\]
For empirical measures
\[
\widehat\nu_n=\frac1N\sum_{i=1}^N\delta_{X_n^i},
\qquad
\widehat\mu=\frac1M\sum_{m=1}^M\delta_{Y^m},
\]
the update is
\[
X_{n+1}^i
=
X_n^i-\gamma
\nabla f_{\widehat\mu,\widehat\nu_n}(X_n^i),
\]
with
\[
\nabla f_{\widehat\mu,\widehat\nu_n}(x)
=
\frac1N\sum_{j=1}^N\nabla_1k(x,X_n^j)
-
\frac1M\sum_{m=1}^M\nabla_1k(x,Y^m).
\]
The first term is particle interaction and the second is target attraction. The computational cost is $O((M+N)N)$ per iteration and memory is $O(M+N)$; random-feature approximations can make the cost essentially linear in $M+N$.

Under Lipschitz-gradient assumptions and sufficiently small step size, including $\gamma\leq2/(3L)$ for descent estimates, the Euler objective decreases. On finite time intervals, the Euler interpolation approximates the continuous flow in $W_2$ with error $O(\gamma)$. Empirical particle approximations satisfy finite-time propagation-of-chaos estimates with Monte Carlo contributions of order $N^{-1/2}$ and $M^{-1/2}$ [1906.04370].

### Negative-distance and Sobolev kernels

The negative-distance kernel
\[
K(x,y)=-\|x-y\|
\]
produces an energy-distance functional, which is a metric on $\mathcal P_1(\mathbb R^d)$. For empirical measures, the particle objective is
\[
F_p(x)
=
-\frac{1}{2N^2}\sum_{i,j}\|x_i-x_j\|
+
\frac{1}{MN}\sum_{i,m}\|x_i-p_m\|.
\]
Its gradient contains repulsive interparticle and attractive target terms. The associated particle flow is a discrete Wasserstein gradient flow on the finite-dimensional manifold of equal-weight empirical measures, rather than necessarily the unrestricted continuous Wasserstein flow.

The negative-distance kernel admits a projection identity reducing multidimensional computations to one-dimensional projected problems. Random projections, sorting, and one-dimensional gradients reduce the cost from approximately $O(N^2)$ to $O(N\log N)$ per projection. This construction has been applied to conditional posterior sampling and generative modeling, including inpainting, superresolution, and computed tomography. The conditional flow transports the signal coordinate while keeping the observation coordinate fixed; its Wasserstein interpretation is a gradient flow of a particle-constrained functional [2310.03054].

On the flat torus, periodic Riesz kernels yield
\[
\mathcal E_s^\nu(\mu)
=
\frac12\|\mu-\nu\|_{\dot H^{-s}}^2,
\qquad
(-\Delta)^sK_s=\delta_0-1.
\]
The Wasserstein velocity is
\[
v_t=-\nabla K_s*(\mu_t-\nu),
\]
and
\[
\frac{d}{dt}\mathcal E_s^\nu(\mu_t)
=
-\int
\left|\nabla(-\Delta)^{-s}(\mu_t-\nu)\right|^2\,d\mu_t.
\]
For $s=1$, global exponential convergence can hold under minimal assumptions. For $s>1$, high-frequency modes are weakly damped and convergence is generally local and polynomial, with rates determined by the Sobolev regularity of the initial and target measures [2603.01977].

## 3. Kernel geometry, SVGD, and functional gradient flow

A different use of kernel gradient flow arises in particle-based variational inference. Let $p_*$ be a target density and $p_t$ the evolving density. The KL functional is
\[
D_{\mathrm{KL}}(p_t\|p_*)
=
\int p_t(x)\log\frac{p_t(x)}{p_*(x)}\,dx.
\]
The ordinary Wasserstein velocity is
\[
g_{\mathrm W}(t,x)
=
\nabla\log\frac{p_*(x)}{p_t(x)},
\]
but this requires the unknown score $\nabla\log p_t$. Restricting the velocity to a function class $\mathcal F$ gives the variational problem
\[
g(t,\cdot)
=
\arg\min_{f\in\mathcal F}
\left[
-\mathbb E_{p_t}
\bigl[\nabla\cdot f+f^\top\nabla\log p_*\bigr]
+
Q(f)
\right].
\]
The regularizer $Q$ defines the functional geometry.

For a vector-valued RKHS $\mathcal H^d$ with
\[
Q(f)=\frac12\|f\|_{\mathcal H^d}^2,
\]
the minimizing velocity is
\[
g(t,x)
=
\int p_t(y)
\left[
\nabla_y k(y,x)
+
k(y,x)\nabla\log p_*(y)
\right]dy.
\]
This is the continuous-time SVGD direction. Its particle discretization is
\[
g(x_i)
\approx
\frac1n\sum_{j=1}^n
\left[
\nabla_{x_j}k(x_j,x_i)
+
k(x_j,x_i)\nabla\log p_*(x_j)
\right].
\]
Thus SVGD is an RKHS-regularized or RKHS-projected functional gradient flow of KL divergence. The kernel determines the tangent-space norm and therefore the notion of steepest descent, rather than merely supplying a numerical smoothing device [2211.13954].

The mean-field Stein flow can also be written
\[
\partial_t\rho_t
=
\nabla\cdot\left[
\rho_t
K_s*\left(\rho_t\nabla\log\frac{\rho_t}{\pi}\right)
\right].
\]
Its Lyapunov functional is relative entropy, while its dissipation is a kernel Stein discrepancy. For Riesz kernels, the Fourier multiplier is $|k|^{-2s}$, so the linearized dissipation rate of mode $k$ is $|k|^{2-2s}$. The Coulomb case $s=1$ yields exponential convergence; for $s>1$, high-frequency modes decay slowly and strong-norm convergence is polynomial. Under smoothness, positivity of the target, and small initial entropy, explicit local polynomial rates hold in entropy and Sobolev norms [2605.09456].

### General functional classes and preconditioning

Replacing the RKHS by a general function class, including neural networks, yields preconditioned functional gradient flow. With a quadratic pointwise regularizer
\[
Q(f)=\frac12\int p_t(x)\|f(x)\|_H^2\,dx,
\]
the ideal velocity is
\[
g_H^*(t,x)
=
H^{-1}\nabla\log\frac{p_*(x)}{p_t(x)}.
\]
The matrix $H$ changes the geometry explicitly. Choosing $H$ to approximate target curvature, such as $-\nabla^2\log p_*$, produces a Newton-like or natural-gradient preconditioning effect. Neural-network fields avoid explicit kernel matrices and can scale linearly in the particle count for fixed network size. The tradeoff is the introduction of nonconvex inner optimization and approximation error [2211.13954].

Semi-Implicit Functional Gradient Flow (SIFG) replaces the empirical particle law $\mu_t$ by its Gaussian convolution
\[
\widehat\mu_t=\mu_t*\mathcal N(0,\sigma^2I_d)
\]
and minimizes
\[
\operatorname{KL}(\widehat\mu_t\|\pi).
\]
The exact velocity is
\[
v_t^*(z)
=
\mathbb E_{x\sim\mathcal N(z,\sigma^2I_d)}
\left[
\nabla\log\pi(x)-\nabla\log\widehat\mu_t(x)
\right].
\]
The smoothed score is learned by denoising score matching rather than divergence estimation. The method is a Wasserstein flow of a smoothed objective, not Langevin dynamics: randomness enters through auxiliary Gaussian perturbations, while the latent particle state evolves deterministically [2410.17935].

### Acceleration and regularization

Accelerated SVGD introduces a momentum potential and a second-order Hamiltonian flow in the Stein metric. The effective dynamics contain a kernel-dependent geometric correction because the Stein metric depends on the evolving distribution. Practical discretizations use momentum, damping, restarts, and a regularization $K+\varepsilon I$ to stabilize kernel-matrix inversion. The regularization corresponds geometrically to adding a Wasserstein component to the Stein metric. General convergence guarantees are not established, and the numerical evidence is primarily low-dimensional [2503.23462].

## 4. Kernelization of measure geometries

A unified formulation treats a gradient system as a state space, an energy, and a dissipation potential:
\[
(X,F,\mathcal R).
\]
For a quadratic dissipation,
\[
\mathcal R(u,\dot u)
=
\frac12\langle\mathbb G(u)\dot u,\dot u\rangle,
\qquad
\mathcal R^*(u,\xi)
=
\frac12\langle\xi,\mathbb K(u)\xi\rangle,
\]
where $\mathbb G$ is the metric tensor and $\mathbb K$ the Onsager operator. The flow is
\[
\dot u=-\mathbb K(u)DF(u),
\]
with energy dissipation
\[
\frac{d}{dt}F(u_t)
=
-\langle DF(u_t),\mathbb K(u_t)DF(u_t)\rangle.
\]

### Kernelized Fisher–Rao flow

For Fisher–Rao geometry,
\[
\mathbb K_{\mathrm{FR}(\mu)\xi=\mu\xi.
\]
Kernelizing the force gives
\[
\mathbb K_{\mathrm{FRk}(\rho)\xi
=
\rho\,\mathcal K_\rho\xi,
\]
and the reaction equation
\[
\boxed{
\dot\mu
=
-\mu\,\mathcal K_\mu\frac{\delta F}{\delta\mu}.
}
\]
The dissipation is
\[
-\frac{d}{dt}F(\mu_t)
=
\left\langle
\frac{\delta F}{\delta\mu},
\mathcal K_{\mu_t}
\frac{\delta F}{\delta\mu}
\right\rangle_{L^2(\mu_t)}.
\]
For inclusive KL, this dissipation is exactly an MMD squared; for ordinary KL, it is a kernelized logarithmic discrepancy. In this sense, MMD can appear as an entropy-dissipation functional rather than merely as a chosen objective.

Because $\mathcal K_\mu$ is compact, inversion is unstable. Adding ordinary Fisher–Rao dissipation yields a regularized operator
\[
(\mathcal K_\mu+\lambda I)^{-1}\mathcal K_\mu.
\]
The corresponding force is the kernel ridge-regression estimator
\[
r_t
=
\arg\min_f
\left\{
\left\|f-\frac{\delta F}{\delta\mu_t}\right\|_{L^2(\mu_t)}^2
+
\lambda\|f\|_{\mathcal H}^2
\right\}.
\]
The flow is
\[
\dot\mu_t
=
-\mu_t
(\mathcal K_{\mu_t}+\lambda I)^{-1}
\mathcal K_{\mu_t}
\frac{\delta F}{\delta\mu_t}.
\]
This is a genuine regularized gradient system, not merely a pointwise regression approximation [2410.20622].

### MMD geometry and Stein geometry

The MMD geometry itself has state-independent dissipation
\[
\mathcal R_{\mathrm{MMD}}(\dot\mu)
=
\frac12\|\mathcal K\dot\mu\|_{\mathcal H}^2,
\]
with Onsager operator $\mathcal K^{-1}$. Its geodesics are mixture interpolations,
\[
\mu_t=(1-t)\mu+t\nu.
\]
This is fundamentally different from the Wasserstein flow of MMD, whose geodesics transport mass spatially.

For $W_2$ geometry,
\[
\mathbb K_{W_2}(\mu)\xi
=
-\nabla\cdot(\mu\nabla\xi).
\]
Velocity-kernelization produces the Stein operator
\[
\mathbb K_{\mathrm{Stein}}(\mu)\xi
=
-\nabla\cdot(\mu\mathcal K_\mu\nabla\xi),
\]
and the PDE
\[
\boxed{
\partial_t\mu
=
\nabla\cdot
\left(
\mu\mathcal K_\mu\nabla\frac{\delta F}{\delta\mu}
\right).
}
\]
SVGD is the particle discretization of this flow. Force-kernelized Fisher–Rao dynamics are reactive and may change mass; velocity-kernelized Wasserstein dynamics are transported and mass-preserving. The distinction concerns the Onsager operator, not merely the notation used for the kernel.

### Sobolev-regularized MMD

Sobolev-regularized MMD modifies the witness through
\[
f_{\mu,\pi}
=
(S_\mu+\lambda I)^{-1}(m_\mu-m_\pi),
\]
where
\[
S_\mu=D_\mu^*D_\mu,
\qquad
D_\mu f=\nabla f.
\]
The velocity is
\[
v_{\mathrm{SrMMD}}[\mu](x)
=
-\nabla f_{\mu,\pi}(x).
\]
This flow is not the ordinary Wasserstein gradient flow of the Sobolev-regularized MMD objective, because differentiating the distribution-dependent inverse operator is not performed. Rather, it is the steepest descent of ordinary squared MMD under the modified metric
\[
g_\mu^\lambda(v,w)
=
\langle D_\mu^*v,D_\mu^*w\rangle_{\mathcal H}
+
\lambda\langle v,w\rangle_{L_2^d(\mu)}.
\]
The preconditioner suppresses directions poorly identified through gradient observations. Under a range condition on the kernel mean-embedding difference, continuous and discrete flows have global convergence guarantees in MMD [2605.11884].

## 5. Applications to neural networks, generative modeling, and inverse problems

Kernel gradient-flow constructions connect naturally to mean-field neural-network training. Consider
\[
\Psi(z,\nu)=\int\psi(z,x)\,d\nu(x),
\]
with finite-width approximation
\[
\Psi_m(z)=\frac1m\sum_{i=1}^m\psi(z,x_i).
\]
For a well-specified regression model, define
\[
k(x,x')
=
\mathbb E_z[\psi(z,x)^\top\psi(z,x')].
\]
Then the excess population regression loss is an MMD functional:
\[
\mathcal F(\nu)
=
\frac12
\int
\|\Psi(z,\nu)-\Psi(z,\mu)\|^2\,dp(z).
\]
Finite particles correspond to finite-width networks; the population limit gives a measure-valued parameter flow; and minibatch or random-feature approximations yield stochastic versions. Thus the Wasserstein MMD flow can represent infinite-width neural-network optimization [1906.04370].

For recurrent networks, the effective kernel is operator-valued and acts on hidden-state trajectories. If $\mathcal P$ is the linearized dynamical propagator and $\mathcal K$ the parameter Gram operator, the training-induced trajectory variation factors as
\[
\boxed{
\delta z=-\mathcal P\mathcal K\mathcal P^*\operatorname{Err}.
}
\]
The operator $\mathcal K$ determines which trajectory directions are accessible through parameter updates, while $\mathcal P$ determines how perturbations propagate through the recurrent dynamics. The effective recurrent kernel is
\[
\Theta_{\mathrm{recurrent}}=\mathcal P\mathcal K\mathcal P^*.
\]
Low effective rank of $\mathcal K$ can create a parameter bottleneck and is associated with low-dimensional latent dynamics, attractor sharing, and dynamical collapse. This is a finite-width, trajectory-space analogue of NTK structure [2507.06381].

### Conditional generative modeling

Negative-distance MMD flows can be conditioned on observations. Given joint samples $(p_i,q_i)$ and latent variables $z_i$, the dynamics transport only the signal coordinates:
\[
\gamma_{N,q}(t)
=
\frac1N\sum_i\delta_{(u_i(t),q_i)}.
\]
The observation coordinates remain fixed, and the resulting curve is a Wasserstein gradient flow of a constrained empirical functional. Neural networks can be trained to approximate blocks of Euler steps, producing a conditional map
\[
T(z,y)
\]
whose output approximates posterior samples. The theory provides a dimension-dependent joint-to-conditional stability estimate under compact-support and Hölder-regularity assumptions, but does not provide a complete end-to-end decomposition of particle, discretization, slicing, momentum, and neural approximation errors [2310.03054].

Kernel-gradient drifting uses a related construction for one-step generative models. For positive differentiable $k$, define
\[
G_p^+(x)
=
\frac{\mathbb E_p[\nabla_xk(x,y)]}{\mathbb E_p[k(x,y)]},
\qquad
G_q^-(x)
=
\frac{\mathbb E_q[\nabla_xk(x,x')]}{\mathbb E_q[k(x,x')]}.
\]
The balanced drift is
\[
V^\nabla_{p,q}(x)
=
G_p^+(x)-G_q^-(x)
=
\nabla\log\widehat p_k(x)-\nabla\log\widehat q_k(x),
\]
where $\widehat p_k$ and $\widehat q_k$ are kernel-smoothed distributions. For characteristic kernels, vanishing drift implies $p=q$. On manifolds, Euclidean gradients are replaced by Riemannian gradients and updates use the exponential map. The resulting neural training procedure uses stop-gradient regression toward transported samples; convergence of the full neural optimization procedure is not established [2605.10727].

### Learning the energy that generates a flow

Kernel methods can also identify unknown potentials and interaction kernels from observed density trajectories. A Wasserstein free energy of the form
\[
\mathcal F(\rho)
=
\int U(\rho)
+
\int V\rho
+
\frac12\iint W(x-y)\rho(x)\rho(y)
\]
generates
\[
\partial_t\rho
=
\nabla\cdot
\left[
\rho\nabla\left(U'(\rho)+V+W*\rho\right)
\right].
\]
The inverse problem learns $V$ and $W$ through the structure-preserving operator
\[
A(V,W)
=
\Delta_{\rho_t}(V+W*\rho_t).
\]
RKHS regularization yields closed-form population and empirical estimators, representer formulas, and error decompositions involving approximation, mesh, and finite-difference errors. The method is a kernelized inverse solver for a Wasserstein flow, not an RKHS time integrator for $V$ and $W$ [2511.06655].

## 6. Function-space kernel gradient flow and generalization

Kernel gradient flow also denotes continuous-time optimization in an RKHS. Given data $(x_i,y_i)$ and empirical loss
\[
\widehat E(f)=\frac12\sum_{j=1}^n(f(x_j)-y_j)^2,
\]
continuous-time kernel gradient flow satisfies
\[
\frac{d}{dt}\widehat f_t(x)
=
-\sum_{j=1}^n
k(x,x_j)(\widehat f_t(x_j)-y_j).
\]
Its spectral representation is
\[
\widehat f_t
=
\varphi_t(T_X)\widehat g,
\qquad
\varphi_t(z)=\frac{1-e^{-tz}}{z},
\]
with remainder filter
\[
\psi_t(z)=e^{-tz}.
\]
Discrete gradient flow replaces the exponential by
\[
\psi_t^{\mathrm{dis}}(z)
=
(1-\eta z)^{t/\eta},
\]
under the stability condition
\[
0<\eta<\frac1{2\kappa^2}.
\]

The estimator admits a bias–variance decomposition. Under a source condition $f^*\in[H]^s$, an embedding index $\alpha_0=1/\beta$, and $s>\alpha_0$, the supremum-norm error has the structure
\[
\|\widehat f_t-f^*\|_\infty
\lesssim
t^{-(s-\alpha)/2}
+
\sqrt{\frac{t^\alpha\log n}{n}},
\]
where $\alpha=\alpha_0+\varepsilon$. Balancing the terms gives
\[
t_{\mathrm{opt}}\asymp n^{1/s},
\]
and the resulting rate is minimax-optimal up to an arbitrary $\varepsilon$-loss:
\[
\|\widehat f_{t_{\mathrm{opt}}}-f^*\|_\infty
=
O_{\mathbb P}
\left(
n^{-\frac{s\beta-1}{2s\beta}+\varepsilon}
\right).
\]
The same framework provides functional Bahadur representations, Gaussian approximations, and multiplier-bootstrap simultaneous confidence bands. Inference requires a later, variance-dominated stopping time $t=n^\theta$ with $1/s<\theta<\beta$, so that the regularization bias is asymptotically negligible [2605.05768].

A related trajectory-dependent perspective defines the loss path kernel
\[
K_T(z,z';\mathcal S)
=
\int_0^T
\langle\nabla_w\ell(w(t),z),\nabla_w\ell(w(t),z')\rangle\,dt.
\]
This is positive semidefinite because it is an integrated feature inner product. Continuous-time training satisfies
\[
\ell(w_T,z)
=
\ell(w_0,z)
-
\frac1n\sum_{i=1}^nK_T(z,z_i;\mathcal S).
\]
The resulting generalization bound contains the trajectory-dependent complexity
\[
\Gamma
=
\frac{2}{n^2}
\sqrt{\sum_{i,j}K_T(z_i,z_j;\mathcal S)}
\sqrt{\sum_iK_T(z_i,z_i;\mathcal S)}.
\]
In the NTK regime, this reduces to static-kernel behavior; beyond that regime, it incorporates feature learning and the entire optimization trajectory [2506.11357].

## 7. Convergence, regularization, and open problems

The convergence behavior of kernel gradient flows depends on the interaction of kernel spectrum, geometry, target regularity, and initialization.

For Wasserstein MMD flows, monotone decrease of MMD does not ensure convergence to the target because singular non-global equilibria can exist. Conditional global convergence follows under a bounded negative-Sobolev distance:
\[
\|\mu-\nu_t\|_{\dot H^{-1}(\nu_t)}^2\leq C.
\]
Then the continuous flow satisfies
\[
\mathcal F(\nu_t)
\leq
\frac{C}{C\mathcal F(\nu_0)^{-1}+4t},
\]
and the Euler scheme has an analogous $O(1/n)$ bound. Noise-injected MMD updates evaluate the gradient at a Gaussian-perturbed point,
\[
X_{n+1}
=
X_n-\gamma\nabla f_{\mu,\nu_n}(X_n+\beta_nU_n).
\]
Under a control condition on the blurred gradient energy and if
\[
\sum_{n=0}^\infty\beta_n^2=\infty,
\]
the objective converges to zero. The mechanism is geometric exploration away from the current support, not entropy-producing diffusion [1906.04370].

Sobolev regularization, KRR preconditioning, and semi-implicit Gaussian smoothing address related degeneracies through different mechanisms. Sobolev MMD modifies the transport metric; KRR regularizes compact-kernel inversion; SIFG smooths the particle law and learns its score by denoising score matching. These procedures generally alter the geometry or the optimized functional, and their guarantees must be interpreted accordingly.

Several general limitations remain:

- **Nonconvexity**: kernel energies may be mixture-convex but not displacement-convex.
- **Support degeneracy**: dissipation evaluated only on the current support can miss errors away from collapsed particles.
- **Kernel dependence**: bandwidth, spectral decay, locality, and characteristicness strongly affect dynamics.
- **Finite-particle error**: continuum well-posedness does not automatically imply uniform particle convergence.
- **Neural approximation**: learned velocity or score fields introduce optimization and approximation errors absent from idealized functional theories.
- **Regularization bias**: Gaussian smoothing, Sobolev penalties, KRR, and Wasserstein metric regularization each modify the limiting system.
- **High-dimensional behavior**: pairwise kernel methods scale quadratically, while overly smooth kernels can weakly dissipate high-frequency errors.
- **Identifiability**: inverse flow learning may determine only the action of an energy through a null-space quotient rather than uniquely recover its components.
- **Incomplete variational convergence**: evolutionary $\Gamma$-convergence has been established in selected Fisher–Rao approximation settings only at the liminf level; corresponding Wasserstein and Wasserstein–Fisher–Rao results remain open [2410.20622].

The modern theory increasingly treats kernel gradient flow as a unifying language for distributional optimization. Kernel mean discrepancies provide quadratic energies; RKHSs specify admissible function classes; kernel operators define Onsager geometries and preconditioners; Wasserstein, Stein, and Fisher–Rao structures determine whether mass is transported or reweighted; and neural or particle discretizations provide computational realizations. The precise mathematical object is therefore not a single algorithm but a hierarchy of kernel-modified gradient systems whose behavior is governed by the interaction between kernel-induced geometry and the regularity of the evolving distribution.

Source: https://www.emergentmind.com/topics/kernel-gradient-flow