---
title: Gradient Flow and Particle Models
url: https://www.emergentmind.com/topics/gradient-flow-and-particle-models
type: topic
---

# Gradient Flow and Particle Models

Gradient flows describe the dynamical evolution of probability densities as steepest descent in the space of measures endowed with a chosen metric. Particle models are finite-dimensional approximations that represent such flows by the trajectories of interacting particles, enabling both analysis and high-dimensional computation. This article synthesizes foundational formulations, Lagrangian and Eulerian perspectives, central particle schemes (notably those based on the Jordan–Kinderlehrer–Otto (JKO) and generalized flows), convergence properties, and advanced neural-network-empowered and scalable methodologies for high-dimensional problems.

## 1. Variational Gradient Flow Formulations

The prototypical mathematical framework consists of a free energy functional on densities, for instance
\[
F(\rho) = \int_{\mathbb{R}^d} U(\rho(x))\,dx + \int_{\mathbb{R}^d} V(x) \rho(x)\,dx + \frac12 \int_{\mathbb{R}^d\times\mathbb{R}^d} W(x,y)\rho(x)\rho(y)\,dx\,dy,
\]
where $U$ encodes internal (e.g., entropic or porous media) energy, $V$ is a potential, and $W$ describes interaction.

The archetype of implicit variational time-discretization is the JKO scheme:
\[
\rho_{k+1} = \arg\min_{\rho \in \mathcal{P}(\mathbb{R}^d)} \left\{ \frac{1}{2\tau} W_2^2(\rho, \rho_k) + F(\rho) \right\},
\]
where $W_2$ is the $2$-Wasserstein distance, and $\tau$ is the time step [2311.06700].

Gradient flows on $\mathcal{P}_p(\mathbb{R})$ for $p>1$ are governed by doubly nonlinear diffusion (for $H$ convex, superlinear), yielding PDEs such as
\[
\partial_t \rho = \partial_x \Big(|\partial_x H'(\rho)|^{q-2} \partial_x H'(\rho)\Big),
\]
with $q$ the dual exponent [2501.03459].

Particle models are constructed to discretize these flows while preserving variational structure at the discrete level.

## 2. Lagrangian Reformulation and Particle Models

Instead of directly approximating densities, one pushes forward a set of particles along a transport induced by a velocity field. In the Lagrangian picture, let the transport map $T(t, x)$ evolve under
\[
\frac{d}{dt} T(t, x) = v(t, T(t, x)), \quad T(0,x) = x,
\]
with the evolved density given by the pushforward $\rho(t) = T(t, \cdot)_\# \rho_k$ [2311.06700].

The evolution of the determinant of the Jacobian relates to the divergence:
\[
\frac{d}{dt} \log\det \nabla_x T(t, x) = \operatorname{div} v\big(t, T(t, x)\big).
\]
For practical reasons, the velocity $v(t, x)$ is often parameterized directly or as the gradient of a neural-network-parameterized potential $\phi_\theta$, so that $v_\theta = \nabla_x \phi_\theta(t,x)$.

Particle models also arise from geometric discretizations, such as nonoverlapping balls or Laguerre (weighted Voronoi) cells, which grant explicit expressions for the induced discrete energy and its gradients [2501.03459, 2304.05069]. For non-overlapping balls in 1D:
\[
E_N(x_1, \ldots, x_N) = \sum_{i=1}^N |B_i| H\left(\frac{1}{N |B_i|}\right),
\]
with $B_i$ the cell associated to $x_i$.

Gradient flows for these energies translate into ODEs on particle positions:
\[
\dot{x}_i = -\partial_{x_i} E_N(x_1,\ldots,x_N),
\]
or, more generally, via the duality mapping for general $p$-geometry [2501.03459].

## 3. Algorithmic Realizations and Loss Functions

A significant class of contemporary particle methods adopt neural network-based parameterizations and neural ODE frameworks [2311.06700, 2310.16516, 2211.13954, 2410.17935]. For implicit-in-time JKO steps:
- Draw $N$ samples from $\rho_k$.
- Evolve these under forward Euler or higher-order ODE solvers, simultaneously updating density using instantaneous Jacobian determinants.
- The loss function (for linear mobility) is
\[
\mathcal{L}(\theta) = \frac{1}{N} \sum_{j=1}^N \left[ \sum_{k=0}^{N_t-1} \Delta t_{\rm in} |\nabla_x \phi_\theta(t_k, z_j^k)|^2 + 2\tau \left( \frac{U(\rho_{k+1}(z_j^{N_t}))}{\rho_{k+1}(z_j^{N_t})} + V(z_j^{N_t}) \right) + \frac{2\tau}{N}\sum_{l=1}^N W(z_j^{N_t}, z_l^{N_t}) \right].
\]
- Update $\theta$ (neural network parameters) via stochastic gradient descent or Adam.

For generalized Wasserstein geometries [2310.16516], the gradient flow of the Kullback–Leibler divergence in a $p$-norm produces:
\[
dx_t = \nabla g^*(\nabla \log \pi(x_t) - \nabla \log \rho_t(x_t))\,dt,
\]
where $g^*$ is the Legendre conjugate of the regularizer, allowing for adaptive and structural choices in the induced geometry.

Particle approximations represent the density by $\{x_i^k\}$, and velocity fields are learned via neural nets or kernelized function classes (as in SVGD, PFG, and Radon–Wasserstein flows) [2602.05227, 2211.13954, 2310.16516, 1704.07520].

## 4. Analytical Properties and Convergence Results

Particle models can be shown to converge to continuum flows under suitable scaling and regularity conditions:
- Gamma-convergence of the discrete energy (for nonoverlapping balls or Voronoi volumes) to its continuum counterpart [2501.03459].
- Serfaty's framework for convergence of gradient flows in metric spaces is key for rigorously passing from particle ODEs to the Wasserstein gradient flow PDE [2501.03459].
- Energy dissipation is preserved at the particle level [2311.06700, 1512.03029].
- For strong convexity or log-concavity scenarios, explicit exponential rates can be established [2403.02004, 2204.12965].

Limitations include extension of convexity-based proofs to higher dimensions, the requirement of well-prepared initial data (e.g., uniformity in spacing), and the lack of sharp, quantitative rates in most nontrivial settings.

## 5. Nonlinear Mobility, Regularization, and High-Dimensional Flows

Gradient flows with nonlinear mobility $M(\rho)$ or in non-Euclidean geometries expand the applicability of particle models:
\[
\partial_t \rho = \nabla \cdot \left( M(\rho) \nabla \frac{\delta E}{\delta\rho} \right ),
\]
leading to modified variational costs and weighted particle motion in Lagrangian frameworks [2311.06700, 1512.03029].

Alternative regularizations, such as preconditioned or functional flows, enable adaptation to ill-conditioning and high-dimensional state spaces. The incorporation of neural architectures in velocity parameterizations or score networks is critical to achieving scalability and approximation power, avoiding kernel methods' curse of dimensionality [2211.13954, 2311.06700, 2602.05227].

Radon–Wasserstein gradient flows impose a geometry where velocities depend only on 1D projections, yielding algorithms with $O(nd)$ complexity per step rather than $O(n^2 d)$ (as in SVGD), thus enabling tractable high-dimensional sampling [2602.05227].

## 6. Applications and Extensions

Gradient flow particle models have been applied and benchmarked in:
- Fokker–Planck and aggregation-diffusion equations (e.g., porous medium, Kalman–Wasserstein flows),
- Nonlocal interaction models (including noisy nonlocal aggregation and biological/collective systems) [2602.03654],
- Variational inference (ParVI, GWG, SIFG, PFG),
- Bayesian inverse problems, Bayesian neural networks, and latent variable EM [2311.06700, 2310.16516, 2410.17935, 2204.12965],
- Generative modeling (score-based diffusion, GANs as particle flows) [2305.16150].

Particle schemes reliably capture mass-preservation, energy decay, metastability (including Arrhenius and Eyring–Kramers rates), entropy-regularized effects (degenerate LSI/PLI), and formation of singularities or clusters under sticky dynamics [2510.16506, 2403.19020].

A practical summary of methods and their core features:

| Method                    | Geometry / Regularizer         | Scalability         | Neuralization | Convergence Guarantee          |
|---------------------------|-------------------------------|---------------------|--------------|-------------------------------|
| Deep JKO [2311.06700]     | Wasserstein-2/Nonlinear mob.  | $O(N d^2)$          | Yes          | Yes, unconditional energy decay  |
| SVGD [1704.07520]         | RKHS (Stein operator)         | $O(N^2 d)$          | Yes          | KL descent, weak convergence  |
| GWG [2310.16516]          | General $p$-Wasserstein       | $O(N d)$            | Yes          | Strong, rate-matched to Langevin  |
| PFG [2211.13954]          | Data-adaptive preconditioning | $O(N d)$            | Yes          | Linear KL decay, ill-conditioned  |
| Radon-Wass. [2602.05227]  | 1D-projection-based           | $O(N d)$            | No           | Well-posed, mean-field limit  |
| SIFG [2410.17935]         | Semi-implicit Gaussian family | $O(N d)$            | Yes          | Non-asymptotic, adaptive noise    |

Extensions and active research questions include quantitative rates in multi-dimensional setups, structure-preserving discretization for broader families, extension of convergence frameworks to flows on manifolds or non-Euclidean spaces, and scalable implementations for large particle numbers and high dimensionality [2311.06700, 2501.03459, 2602.05227].

Source: https://www.emergentmind.com/topics/gradient-flow-and-particle-models