Gradient Flow: Dynamics and Geometry

• Gradient flow is a continuous-time steepest descent process that adapts its evolution based on the underlying geometry and functional constraints.
• It unifies various formulations—Euclidean, Riemannian, Wasserstein, and others—to rigorously analyze convergence, dissipation, and structural properties.
• It is widely applied in optimization, gauge theories, and neural network training, offering practical insights into system dynamics and regularization effects.

Gradient flow is a continuous-time steepest-descent evolution for an objective functional, but the meaning of “steepest” depends on the ambient geometry. In its simplest Euclidean form it is the ODE $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t))$, while in other settings it becomes a Riemannian flow on manifolds of positive definite forms, a metric flow on spaces of measures, a diffusion-type evolution for gauge fields, or a preconditioned stochastic dynamics on Hilbert space (Wadayama et al., 2024, Li et al., 10 Jun 2025, Erbar et al., 2024, Kitazawa et al., 2014, Pillai et al., 2011). Across these formulations, gradient flow serves as both an analytic object—used to prove existence, dissipation, convergence, and structural properties—and a computational paradigm for optimization, sampling, PDEs, lattice field theory, coding, and neural-network training (Maury et al., 2010, Kim et al., 2022, Shou et al., 26 May 2025).

1. Euclidean, preconditioned, and stochastic formulations

In finite-dimensional optimization, the generic gradient-flow dynamics is

$\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$

and the trajectory is the continuous-time steepest descent of the potential $f$ (Wadayama et al., 2024). This viewpoint is explicitly used in optimization-based decoding, where a continuous state $\bm x(t)\in\mathbb R^n$ evolves toward equilibria associated with bipolar LDPC codewords, and in latent-variable inference, where the latent code $z$ is obtained by solving a reconstruction-loss flow rather than by applying a learned encoder (Wadayama et al., 2024, Flouris et al., 2021).

A more general formulation appears in infinite-dimensional probability and sampling. For a target measure

$$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$

the diffusion limit of a pCN Metropolis chain is the SPDE

$dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$

which can be written as

$$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$

Here the deterministic part is a preconditioned gradient flow, and the additive noise is matched to the Gaussian covariance structure so that the dynamics is reversible with respect to the same target measure (Pillai et al., 2011).

This already illustrates a basic structural fact: gradient flow is not tied to the unpreconditioned Euclidean gradient. The operator multiplying the gradient may encode covariance, geometry, or constraints; in stochastic settings, the same operator often controls the invariant measure and reversibility properties of the limiting dynamics (Pillai et al., 2011).

2. Geometry as the defining ingredient

A central theme in modern uses of gradient flow is that the state space and metric are part of the model, not an auxiliary choice. In the kernel learning problem, the evolving variable is not only a function $F$, but a positive definite quadratic form $\Sigma$ indexing a family of RKHS norms. The effective objective is

$\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$0

and the flow is posed on the manifold $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$1 of positive definite symmetric bilinear forms. The chosen Riemannian metric $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$2 depends only on the covariance of $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$3, lifts to a Euclidean gradient flow in the $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$4-parameterization with $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$5, preserves the positive semidefinite cone and its rank stratification, and, in the presence of Gaussian noise variables, yields a continuous family of Lyapunov functionals with automatic denoising. In an orthonormal basis, the flow is

$\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$6

The paper stresses that this is not Euclidean gradient flow on $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$7, and not a Wasserstein flow; it is a Riemannian gradient flow for the specially chosen metric $\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$8 (Li et al., 10 Jun 2025).

A different non-Euclidean geometry arises in inner-product Gromov–Wasserstein geometry. There, the limiting continuity equation still has the form

$\frac{d\bm x(t)}{dt}=-\nabla f(\bm x(t)),$9

but the velocity is not the Wasserstein gradient itself. Instead,

$f$0

where $f$1 and

$f$2

The corresponding metric tensor is

$f$3

so the IGW gradient is formally

$f$4

This turns steepest descent into a globally aligned motion rather than a purely local transport field (Zhang et al., 2024).

These constructions show that “gradient flow” is a geometric concept before it is an algorithmic one: the same functional can induce radically different dynamics depending on the metric tensor or mobility operator used to define descent (Li et al., 10 Jun 2025, Zhang et al., 2024).

3. Metric gradient flows in spaces of measures

In Wasserstein and Wasserstein-type settings, gradient flow is often constructed variationally rather than by writing an explicit vector field from the outset. For crowd motion with hard congestion, if the desired velocity is of gradient type,

$f$5

the evolution is interpreted as the Wasserstein gradient flow of

$f$6

where $f$7 encodes the density constraint $f$8 and possible concentration on the exit. The actual velocity is the projection $f$9, and the flow is constructed by the minimizing movement scheme

$\bm x(t)\in\mathbb R^n$0

A central difficulty is that the functional is neither finitely valued nor geodesically convex, and with an exit the admissible set is not geodesically convex in Wasserstein space, so the convergence theory requires ad hoc compactness and projection arguments (Maury et al., 2010).

For nonlinear diffusion with constant Dirichlet boundary data, the classical mass-preserving Wasserstein geometry is no longer appropriate because the boundary acts as a reservoir. The relevant distance is the Figalli–Gigli modified Wasserstein distance $\bm x(t)\in\mathbb R^n$1, defined through couplings whose marginals are prescribed only in the interior, so mass may disappear into or emerge from $\bm x(t)\in\mathbb R^n$2 (Erbar et al., 2024). In the $\bm x(t)\in\mathbb R^n$3 case, the internal energy

$\bm x(t)\in\mathbb R^n$4

generates the diffusion

$\bm x(t)\in\mathbb R^n$5

and the Dirichlet condition $\bm x(t)\in\mathbb R^n$6 is encoded through the finite-slope condition $\bm x(t)\in\mathbb R^n$7, where

$\bm x(t)\in\mathbb R^n$8

The analysis is carried out in the sense of curves of maximal slope rather than EVI, because the functional is generally not semiconvex along $\bm x(t)\in\mathbb R^n$9-geodesics (Erbar et al., 2024).

The porous medium equation with nonnegative constant Dirichlet boundary condition,

$z$0

admits an analogous minimizing-movement formulation with the same boundary-reservoir distance $z$1. The energy is shifted so that $z$2 is the minimizer,

$z$3

and the discrete scheme

$z$4

produces weak solutions with an energy dissipation inequality. The paper explicitly states that a full curve-of-maximal-slope characterization in the abstract AGS sense remains open because the local slope is not explicitly identified and geodesic convexity is unavailable (Kim et al., 2022).

Taken together, these works show that metric gradient flow extends well beyond the classical $z$5–JKO setting. Hard constraints, open boundaries, and non-geodesically-convex energies do not eliminate the gradient-flow structure, but they do change the underlying metric space, the notion of admissible tangent motion, and the form of the convergence theory (Maury et al., 2010, Erbar et al., 2024, Kim et al., 2022).

4. Field-theoretic gradient flow, renormalization, and RG

In gauge theory, gradient flow is a flow in field configuration space rather than in a finite-dimensional parameter space. For Yang–Mills theory, the flowed gauge field satisfies

$z$6

and at tree level the equation is diffusion-like. The flow smooths the gauge field over a radius $z$7, and composite operators built from flowed fields at positive flow time are UV finite (Kitazawa et al., 2014). This finiteness is the basis of the small flow-time expansion used to define the renormalized energy-momentum tensor on the lattice: $z$8 In pure $z$9 gauge theory, this construction yields thermodynamic observables consistent with the conventional integral method while requiring less statistics (Kitazawa et al., 2014).

The background-field method provides a gauge-covariant perturbative reformulation of the Yang–Mills flow. The modified gauge-fixed equation

$$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$0

preserves covariance under background gauge transformations and leads to compact one-loop small flow-time expansions, including the coefficients needed for the lattice energy-momentum tensor (Suzuki, 2015).

Flowed gauge fields also modify fermionic spectral observables. In the microscopic Dirac spectrum, the paper on chiral perturbation theory for gradient flow shows that the repulsion of low Dirac eigenvalues from the dynamical quark mass decreases with increasing flow time, and that sufficiently large microscopic flow time drives the low-lying spectral observables toward their quenched form. The scale of this decorrelation is governed by a new low-energy constant $$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$1 (Christensen et al., 2014).

Outside gauge theory, extending gradient flow to scalar field theory is nontrivial. In four-dimensional $$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$2 theory, the naive interacting flow produces uncancelled divergences, so the paper replaces it by

$$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$3

The resulting kernel $$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$4 behaves like a higher-derivative regulator, and the paper establishes finiteness to all orders in perturbation theory without introducing new counterterms beyond those of the underlying renormalized $$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$5 theory (Fujikawa, 2016).

Gradient flow also admits an RG interpretation. In scalar theory, with Gaussian cutoff $$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$6, the diffused field

$$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$7

can be related exactly to a Wilsonian RG flow with

$$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$8

and this structure motivates a manifestly gauge-invariant exact renormalization-group equation for Yang–Mills theory built from flowed gauge fields (Sonoda et al., 2020). Closely related work interprets expectation values of finite flowed operators as coordinates on coupling space, so that flow time acts as RG time and flowed observables can display the approach to infrared fixed points such as Banks–Zaks or Wilson–Fisher (Makino et al., 2018).

5. Learning, coding, and neural-network dynamics

In machine learning, gradient flow is the zero-step-size limit of gradient descent, but the relevant state variable may be weights, latent codes, decoder coordinates, or structured geometric parameters. For deep linear neural networks with quadratic loss, the adjacency-matrix representation

$$\frac{d\pi^\tau}{d\pi_0^\tau}(x)\propto \exp\!\left(-\frac{\Psi(x)}{\tau}\right), \qquad \pi_0^\tau=N(0,\tau C),$$9

converts the layerwise flow into the polynomial matrix ODE

$dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$0

In this form the system is nilpotent, polynomial, isospectral, and endowed with conservation laws

$dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$1

The loss is a positive semidefinite Lyapunov function, every trajectory converges pointwise to a critical point, the landscape contains infinitely many global minima and saddle points but no local minima or maxima, and the critical values encode which singular values of the input-output data have been learned along the trajectory (Wendin et al., 13 Nov 2025).

A different use of gradient flow appears in decoder-only representation learning. Given a decoder $dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$2 and sample $dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$3, gradient flow encoding defines the latent code as the endpoint of

$dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$4

thereby replacing the encoder by direct latent optimization (Flouris et al., 2021). The paper also considers the second-order ODE

$dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$5

as a continuous-time approximation of Nesterov acceleration, and proposes an adaptive minimize-distance solver that chooses step sizes by requiring per-step decrease of the reconstruction loss rather than by controlling local truncation error (Flouris et al., 2021).

Gradient flow has also been used to learn optimizer-induced training dynamics. Gradient Flow Matching models neural-network training as a continuous-time system

$dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$6

in the SGD case, but learns a surrogate vector field $dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$7 from observed weight trajectories. In the practical formulation, the target field is estimated from finite differences $dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$8, and the method uses conditional flow matching together with a terminal prediction penalty to forecast converged weights from short weight prefixes. The paper interprets this as learning an effective continuous-time flow for SGD, Adam, RMSprop, and related optimizers (Shou et al., 26 May 2025).

In coding theory, gradient flow decoding relaxes LDPC decoding from the discrete code $dz=-(z+C\nabla\Psi(z))\,dt+\sqrt{2\tau}\,dW,$9 to $$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$0 and uses the ODE

$$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$1

for AWGN channels, or more generally

$$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$2

for arbitrary channel likelihoods. The method is continuous-time, tensor-computable, and well suited to analog circuit implementation; in reported experiments it is comparable to multibit gradient descent bit-flipping on AWGN and competitive with MMSE + BP in LDPC-coded MIMO settings (Wadayama et al., 2024).

These applications share a common pattern: gradient flow is used when the inference or training variable is best understood as the trajectory of a dynamical system rather than as the output of a one-shot map. The appeal is often structural—continuous-time interpretation, geometric transparency, or compatibility with analog and ODE-based computation—rather than merely notational (Flouris et al., 2021, Shou et al., 26 May 2025, Wadayama et al., 2024).

6. Dissipation, regimes, and recurring distinctions

A unifying feature of gradient-flow systems is dissipation, but the corresponding Lyapunov structure can be highly geometry-dependent. In kernel learning, the dissipation identity is

$$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$3

and under suitable moment and regularity assumptions the flow exists for all $$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$4, extends to the positive semidefinite cone, preserves rank strata, and converges subsequentially to stationary points on those strata (Li et al., 10 Jun 2025). In deep linear networks, the analogous identity is

$$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$5

but the stationary set is highly nongeneric: each critical value corresponds to an unbounded invariant fiber of factorizations rather than to an isolated point (Wendin et al., 13 Nov 2025).

For measure transport based on kernel discrepancies, the geometry again changes the form of descent. Sobolev-regularized MMD flow replaces the plain witness $$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$6 by

$$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$7

and evolves particles by

$$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$8

The paper explicitly states that this is not the Wasserstein gradient flow of $$dz=-C\nabla J(z)\,dt+\sqrt{2\tau}\,dW, \qquad J(x)=\frac12\|C^{-1/2}x\|^2+\Psi(x).$$9; instead, it is a Sobolev-regularized variant of MMD flow that can be interpreted as steepest descent of the MMD functional under a modified metric $F$0. Under a source condition

$F$1

the flow enjoys global convergence guarantees in MMD/KSD in both continuous and discrete time, and the analysis does not rely on isoperimetric assumptions on the target distribution (Tian et al., 12 May 2026).

At a more formal asymptotic level, gradient flow can itself have multiple scaling regimes. For CP tensor learning, the loss trajectory under gradient flow admits a formal time expansion whose coefficients are encoded by diagrams akin to Feynman diagrams. In large-$F$2 limits, the dominant diagram classes classify distinct regimes—free evolution, NTK/lazy, and under- and over-parameterized mean-field—and in several cases the resulting formal series can be summed by converting coefficient recurrences into PDEs, often first-order and solvable by characteristics (Yarotsky et al., 4 Feb 2026).

Several common identifications are therefore misleading. Gradient flow is not synonymous with Euclidean gradient descent, because the descent direction may be preconditioned, covariant, or constrained (Pillai et al., 2011, Li et al., 10 Jun 2025). It is not synonymous with Wasserstein flow, because IGW and boundary-reservoir geometries produce different tangent structures and mobility operators (Zhang et al., 2024, Erbar et al., 2024). Nor is it always best understood as direct optimization of a static finite-dimensional objective: in gauge theory it is also a smoothing and renormalization device whose flow time can act as an RG scale (Kitazawa et al., 2014, Sonoda et al., 2020).

In this broader sense, gradient flow is less a single equation than a class of descent evolutions indexed by geometry, state space, and functional calculus. What remains invariant across its many realizations is the idea that dynamics should be generated by the steepest available descent once the relevant notion of distance, admissible variation, and symmetry has been fixed.