---
title: Entropic Gradient Flows
url: https://www.emergentmind.com/topics/entropic-gradient-flow
type: topic
---

# Entropic Gradient Flows

Entropic gradient flow designates a class of gradient flows in which the central driving functional is (a variant of) entropy—typically the Boltzmann–Shannon (Kullback–Leibler) entropy or its relatives—evolved in a geometric structure reflecting underlying transport, reaction, or probabilistic features. Its archetype is the realization of the heat (or Fokker–Planck) equation as the Wasserstein steepest descent of the entropy, but the concept encompasses a variety of systems, including those governed by entropic-regularized optimal transport, mean-field models, reaction/transport equations in unbalanced spaces, discrete Markov processes, and more. Numerical schemes leveraging entropic regularization have revolutionized optimal transport computation and enabled fast, stable implementations on large-scale problems. Entropic gradient flows enjoy robust variational and geometric foundations, admit large-deviations interpretations, and unify stochastic, geometric, and information-theoretic perspectives.

## 1. Entropic Regularized Optimal Transport and the Entropic JKO Scheme

Let $\mu,\nu$ be probability measures in $\mathcal P_2(\mathbb R^d)$ with cost $c(x,y)=|x-y|^2$. The standard 2-Wasserstein distance is defined via the Monge–Kantorovich problem, while the entropic regularized optimal transport (also known as the Schrödinger problem) minimizes
\[
W_{2,\varepsilon}^2(\mu,\nu) = \min_{\pi\in\Pi(\mu,\nu)} \left\{ \int |x-y|^2\,d\pi(x,y) + \varepsilon\,H(\pi|\mu\otimes\nu) \right\}
\]
with $H(\pi|\mu\otimes\nu)$ the Kullback–Leibler divergence. This regularization renders the problem strictly convex, smooth, and numerically amenable via Sinkhorn's algorithm through iterative scaling of the dual potentials ($\varphi$, $\psi$) [1512.02783].

The classical Jordan–Kinderlehrer–Otto (JKO) scheme realizes gradient flows by implicit Euler steps in the Wasserstein space,
\[
\rho^{k+1} = \arg\min_{\rho\in \mathcal{P}_2(\mathbb{R}^d)} \left\{ \frac{1}{2\tau}W_2^2(\rho,\rho^k) + F(\rho) \right\}
\]
where $F$ is a suitable convex functional. The entropic JKO scheme replaces $W_2$ with $W_{2,\varepsilon}$:
\[
\rho^{k+1}_{\varepsilon,\tau} = \arg\min_{\rho} \left\{ \frac{1}{2\tau}W_{2,\varepsilon}^2(\rho,\rho^k_{\varepsilon,\tau}) + F(\rho) \right\}
\]
This strictly convex, smooth problem can be efficiently solved with iterative Sinkhorn projections and is highly parallelizable [1512.02783, 1502.06216].

Convergence of the entropic flow—as $\varepsilon, \tau \to 0$ with $\varepsilon|\log\varepsilon| = O(\tau^2)$—to the original (unregularized) gradient flow is established via evolutionary $\Gamma$-convergence. The entropy barrier improves stability, enables massive parallelism, and allows JKO steps that are robust even on complex computational domains [1512.02783, 2008.10962].

## 2. Geometry, Variational Structure, and Large-Deviation Principles

Entropic gradient flows possess a natural variational structure grounded in generalized Riemannian geometry on the space of probability measures. The Otto calculus formalizes the geometric setting, allowing the identification of the 2-Wasserstein metric with a Riemannian structure whose "energy" is the entropy [1201.4601, 2008.09220]. The prototypical equation,
\[
\partial_t\rho = \nabla\cdot(\rho\,\nabla \delta F/\delta\rho)
\]
emerges as the steepest descent of $F$ under the Wasserstein metric.

A profound link exists between entropic gradient flows and large-deviation theory. The discrete-time JKO minimization is asymptotically equivalent to the large-deviation rate functional for many-particle systems (e.g., independent Brownian motions), tightly coupling microscopic stochastic fluctuations and macroscopic deterministic evolution [1004.4076, 1201.4601]. Thus, gradient-flow dynamics are identified as zero-action (most probable) curves in the path-space large-deviation principle, unifying probabilistic and variational perspectives.

## 3. Extensions: Sinkhorn Geometry, Discrete and Nonlocal Settings

Several entropic geometries have been developed to address settings beyond classical transport:

- **Sinkhorn Divergences:** The Sinkhorn divergence (debiased entropic OT) defines a geometry metrizing weak convergence and enables gradient flows via modified JKO schemes [2511.14278]. The induced geometry can be characterized via a reproducing kernel Hilbert space embedding, and admits efficient minimizing-movement schemes retaining well-posedness, contractivity, and global convergence to minimizers.

- **Discrete Markov Chains:** On finite spaces, a Benamou–Brenier-type nonlocal metric $W$ is constructed so that the law of a reversible Markov chain is the gradient flow of entropy with respect to $W$ [1102.5238]. The discrete chain thus exactly parallels the Wasserstein-entropy gradient flow of diffusion.

- **Porous Medium and Reaction Equations:** Both in discrete spaces and for nonlinear Markov semigroups, non-Euclidean metrics and suitably chosen convex entropies yield porous-medium-type and reaction-diffusion equations as entropic gradient flows [1212.1129, 1706.08957].

- **Boltzmann and Landau Kinetics:** For the spatially homogeneous Boltzmann or Landau equation, a nonlocal Onsager operator or metric is constructed so that the kinetic PDE is the entropy gradient flow in a "collision geometry" or through a nonlocal action [1603.00540, 2007.08591].

## 4. Stochastic Processes, Schrödinger Problems, and Interpolations

Entropic gradient flows connect deeply with stochastic processes:

- **Schrödinger Bridge & Entropic Interpolation:** The Schrödinger problem seeks the most likely (minimum-entropy) path ensemble connecting two marginals under a reference diffusion. The resulting path-space minimizer (Schrödinger bridge) interpolates between marginals via entropic transport, converging in the small-noise limit to classical OT and in the large-horizon limit to heat flow—the entropy gradient flow [2007.07594, 1601.04891].

- **Projected Langevin Dynamics:** For entropy-regularized OT between prescribed marginals, the projected Langevin SDE provides a stochastic process constrained to the coupling manifold $\Pi(\mu,\nu)$ whose law evolves via the gradient flow of the entropic cost. Exponential convergence to optimal couplings is achieved under log-Sobolev inequalities [2309.08598].

- **Entropy Dissipation & Inequalities:** Along entropic gradient flows in both regular and projected OT geometries, entropy decays at rates governed by projected Fisher information and log-Sobolev constants, offering quantitative convergence guarantees [2008.09220, 2309.08598].

## 5. Numerical and Algorithmic Implications

Algorithmically, the principal benefit of entropic regularization is the reduction of high-dimensional, non-smooth transport problems to strictly convex, smooth optimization solvable by matrix scaling (Sinkhorn) or Dykstra's iterative projections [1512.02783, 1502.06216]. Each JKO/entropic step reduces to alternating KL projections, massively accelerating computations on grids, meshes, or graphs and allowing for GPU parallelization. For regular meshes and Gibbs kernels (Gaussian heat kernels), each scaling step costs $O(N)$ [1502.06216].

Eulerian implementations of the Sinkhorn-JKO scheme enjoy energy dissipation, global contractivity, and "teleportation" effects for mass transfer across barriers, unattainable in traditional $W_2$-gradient flows [2511.14278]. The convergence of SJKO schemes to continuous flows can be established via monotonicity and maximal monotone operator theory in RKHS embeddings.

In discrete Markov and network settings, analogues of the JKO scheme can be devised using nonlocal transport metrics tailored to the generator, achieving convergence to continuum diffusivities under mesh refinement [1212.1129, 2008.10962].

## 6. Generalizations, Function Spaces, and Future Directions

The entropic gradient flow paradigm generalizes to:

- **Fisher–Rao Geometry:** In mean-field games and min-max optimization with entropic regularization, the Fisher–Rao metric underlies birth–death flows that guarantee exponential convergence and admit explicit Lyapunov functions [2405.15834].
- **Hellinger–Kantorovich Geometry:** Nonlinear Fokker–Planck and reaction-diffusion equations are entropic gradient flows in an unbalanced OT metric, crucial for models involving creation/annihilation of mass [1706.08957].
- **Curves, Image Processing, Quantum Gravity:** Entropic action functionals founded on quantum or information-theoretic relative entropy give rise to gradient flows in geometric settings, such as flows of planar curves or the anisotropic diffusion (Perona–Malik) in image processing [2503.14048, 2101.11219].

Future research encompasses further abstraction to generalized dissipation potentials (beyond quadratic), connection with control theory, stochastic processes with constraints, and non-equilibrium thermodynamics, including recent formulations identifying entropy as a Noether charge—invariant under continuous symmetries in the Hamiltonian action, unifying reversibility and macroscopic conservation [2406.16819].

## 7. Summary Table: Principal Entropic Gradient Flows and Geometries

| Setting / PDE                                  | Metric / Geometry          | Driving Functional      | Reference         |
|------------------------------------------------|---------------------------|------------------------|-------------------|
| Heat / Fokker–Planck ($\partial_t\rho = \Delta\rho$) | $W_2$ (Wasserstein)        | Entropy $H(\rho)$      | [1512.02783], [1201.4601], [2008.09220] |
| Entropic OT / Sinkhorn-JKO                     | Sinkhorn divergence        | $E(\mu) + D_\varepsilon$ | [2511.14278], [1502.06216] |
| Discrete Markov chains                         | Discrete transport metric  | Entropy H              | [1102.5238]       |
| Boltzmann / Landau equations                   | Collision/Landau distance  | Boltzmann entropy      | [1603.00540], [2007.08591] |
| Hellinger–Kantorovich reaction-diffusion       | HK metric (unbalanced OT)  | Free energy $\mathcal E$| [1706.08957]     |
| Fisher–Rao min-max games                       | Fisher–Rao (information)   | Relative entropy $D_{KL}$ | [2405.15834]   |
| Perona–Malik, quantum gravity                  | Info-geometry (operator)   | Quantum rel. entropy   | [2503.14048]      |


The unifying principle of entropic gradient flow is the identification of a suitable entropy-related functional as the gradient flow generator in a Riemannian or sub-Riemannian metric structure determined by the relevant physical, probabilistic, or geometric constraints. This enables a general framework for evolution equations, variational approximation schemes, and efficient algorithms with strong theoretical guarantees for convergence and stability across diverse settings [1512.02783, 2511.14278, 2008.10962, 1004.4076, 1201.4601, 2406.16819].

Source: https://www.emergentmind.com/topics/entropic-gradient-flow