Papers
Topics
Authors
Recent
Search
2000 character limit reached

Probability Flow ODE

Updated 1 November 2026
  • "Probability Flow ODEs are deterministic ordinary differential equations whose time-marginal probability distributions align with a corresponding diffusion process, to create time-evolving data distributions for practical generative modeling."
  • Probability Flow ODEs are fundamental in generative models like score-based generative modeling, deterministic diffusion samplers, and continuous normalizing flows.
  • Applications of Probability Flow ODEs range from deterministic sampling and likelihood evaluation to invertible mappings in data analysis using exact diffusion trajectories.

A probability flow ODE is a deterministic ordinary differential equation whose velocity field is constructed so that its time-marginal probability distributions coincide with those of a corresponding diffusion process. For a diffusion

dXt=f(Xt,t) dt+σ(t) dWt,dX_t=f(X_t,t)\,dt+\sigma(t)\,dW_t,

with density ptp_t, the associated probability-flow velocity is

vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),

and the deterministic dynamics are

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).

Under appropriate well-posedness conditions, Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t for every time tt, although the diffusion and ODE have different path laws: diffusion trajectories are stochastic, whereas ODE trajectories are deterministic conditional on their initial state. The construction is central to score-based generative modeling, deterministic diffusion samplers, continuous normalizing flows, density estimation, and transport-based analysis.

1. Mathematical construction

The marginal density ptp_t of the diffusion satisfies the Fokker–Planck equation

∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.

Using

∇pt=pt∇log⁡pt\nabla p_t=p_t\nabla\log p_t

and

Δpt=∇⋅(pt∇log⁡pt),\Delta p_t=\nabla\cdot\left(p_t\nabla\log p_t\right),

the equation can be rewritten as

ptp_t0

This is a continuity equation with velocity

ptp_t1

The characteristic equation is therefore

ptp_t2

The factor ptp_t3 distinguishes the probability-flow ODE from the reverse-time SDE. The reverse SDE contains the full score correction and stochastic forcing, whereas the probability-flow ODE contains half the score correction and no Brownian term. Both processes have the same one-time marginals when the exact score and exact dynamics are used.

For a spatially homogeneous diffusion matrix ptp_t4, the same construction is

ptp_t5

For a state-dependent diffusion tensor ptp_t6, the probability-flow velocity becomes

ptp_t7

where

ptp_t8

The additional ptp_t9 term is essential when the diffusion tensor varies spatially (Wu et al., 22 Dec 2025).

The continuity-equation representation also gives the pushforward relation

vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),0

whenever the ODE generates a suitable flow. Thus, if vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),1, then vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),2 has the same law as vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),3 at every time. The equality concerns one-time marginals, not pathwise trajectories.

2. Score-based diffusion modeling

In score-based generative modeling, the forward diffusion transforms a data distribution into an approximately Gaussian distribution. A reverse-time SDE uses a learned score

vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),4

to transport Gaussian noise toward data. The probability-flow ODE replaces this stochastic reverse process by deterministic dynamics. For the Ornstein–Uhlenbeck process

vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),5

the reverse-time probability-flow ODE is

vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),6

where vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),7 denotes the forward marginal. With a learned score, the implemented field is

vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),8

For a variance-preserving diffusion

vt(x)=f(x,t)−12σ(t)2∇xlog⁡pt(x),v_t(x)=f(x,t)-\frac{1}{2}\sigma(t)^2\nabla_x\log p_t(x),9

the reverse probability-flow ODE has the form

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).0

For a variance-exploding diffusion with dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).1, the forward-oriented probability-flow equation is

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).2

which is integrated backward from a high-noise state.

The probability-flow formulation supports deterministic sampling, likelihood evaluation, and invertible data-to-latent mappings. A conventional score model learns the infinitesimal vector field and obtains finite-time transitions by numerical integration. A trajectory model can instead learn the finite-time solution operator directly. Consistency Trajectory Models define a transition map

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).3

representing the state obtained by following the probability-flow trajectory from time dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).4 to time dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).5. The map satisfies

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).6

which enables arbitrary time-to-time trajectory jumps. Such models encompass score-based sampling and ordinary consistency models as special cases (Kim et al., 2023).

3. Deterministic transport, likelihoods, and invertibility

The deterministic flow induces an instantaneous change-of-variables identity. If

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).7

then along a trajectory,

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).8

Consequently, for a data point transported to a terminal latent state,

dYtdt=vt(Yt).\frac{dY_t}{dt}=v_t(Y_t).9

with the sign depending on the chosen time orientation. In high dimensions, the divergence is commonly estimated with Hutchinson’s estimator: Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t0 where Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t1 is Gaussian or Rademacher noise.

This makes the probability-flow ODE a deterministic continuous normalizing flow. It can map data to a simple prior and reverse the map for generation. The implementation is exact only in the idealized setting of an exact score, exact divergence evaluation, exact integration, and a correctly specified terminal prior. Practical likelihoods additionally contain score-model, trace-estimation, and numerical-integration errors (Arvinte et al., 2023).

The deterministic character also permits the definition of shared-noise distributional distances. If Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t2 and Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t3 are the probability-flow noise-to-data maps associated with two distributions, Probability Flow Distance is defined by

Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t4

The coupling uses the same Gaussian initial noise for both maps. With the identity feature map, the resulting transport cost upper-bounds the Wasserstein distance: Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t5 Probability Flow Distance has been used to define generalization and memorization errors for diffusion models and to study scaling, early learning, double descent, and bias–variance behavior (Zhang et al., 26 May 2025).

A deterministic PF-ODE can also serve as an encoder. If the forward map Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t6 is injective and has a measurable inverse on its range, then

Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t7

This property enables change detection by mapping pre-change observations to a Gaussian latent space and applying MMD against the Gaussian null. A stochastic reverse diffusion sampler does not provide a single-valued encoding because fresh noise is injected during sampling (Kraevskiy et al., 24 Aug 2026).

4. Numerical discretization and convergence theory

Probability-flow sampling requires approximating both the score and the ODE solution. Exponential integrators treat the linear drift analytically. For a general variance-preserving field,

Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t8

an exponential Runge–Kutta method integrates the first term exactly and approximates the score-dependent term. This yields first-, second-, and higher-order schemes with global numerical errors of order Law⁡(Yt)=pt\operatorname{Law}(Y_t)=p_t9 for a tt0-th order method (Huang et al., 16 Jun 2025).

Several convergence regimes have been established.

  • Under strong log-concavity and smoothness, exponential-integrator samplers admit non-asymptotic tt1 bounds. For variance-preserving schedules, the analyzed iteration complexity is tt2, while the analyzed variance-exploding schedules have worse dependence on dimension and accuracy (Gao et al., 2024).
  • Under compact support, integrated tt3 score error, and regularity assumptions on the learned score, Runge–Kutta samplers admit total-variation bounds whose principal terms have the form

tt4

where tt5 is an integrated score error and tt6 is the step size (Huang et al., 2024).

  • A sharper deterministic analysis for a particular DDIM-type discretization obtains, up to logarithmic factors,

tt7

The resulting iteration complexity is tt8 under a specialized noise schedule. The analysis does not assume smoothness, log-concavity, or a Lipschitz target score, but it requires control of both score and Jacobian errors (Li et al., 2024).

  • For target distributions with low metric entropy or intrinsic dimension tt9, a deterministic DDIM/probability-flow sampler can achieve a discretization rate of order

ptp_t0

up to logarithmic factors and score-estimation terms. The result interprets “adaptivity” as statistical adaptation to low-dimensional support rather than online adaptive step-size selection (Tang et al., 31 Jan 2025).

  • For subgaussian targets with ptp_t1-Hölder densities, ptp_t2, a smoothly regularized kernel score estimator simultaneously controls score and mean-Jacobian errors. The resulting probability-flow sampler achieves the density-estimation minimax rate

ptp_t3

in total variation, up to logarithmic factors (Cai et al., 12 Mar 2025).

A recurring distinction is between score error and numerical error. Refining the ODE solver reduces discretization error but cannot remove score-estimation error. High-order methods are therefore useful only until numerical error becomes smaller than the error induced by the learned score.

5. Well-posedness, endpoint singularities, and score stability

The formal rewriting of the Fokker–Planck equation as a continuity equation does not by itself guarantee that the probability-flow ODE generates a well-defined deterministic flow. Eulerian well-posedness of the density equation and Lagrangian well-posedness of the characteristic ODE are distinct.

Under weak assumptions on the drift, the Fokker–Planck equation can have a unique density flow in an energy class. To obtain a deterministic flow, stronger conditions are required, such as Sobolev or bounded-variation regularity of the velocity, linear growth, and one-sided bounds on its divergence. The DiPerna–Lions–Ambrosio theory then yields a regular Lagrangian flow ptp_t4 satisfying

ptp_t5

The compressibility bound depends on the negative part of the divergence: ptp_t6

Reverse-time sampling and invertibility require two-sided divergence control. Forward marginal matching under a one-sided bound does not automatically imply reverse well-posedness, global invertibility, or exact change-of-variables formulas (Cont, 28 Jul 2026).

Endpoint singularities arise because diffusion smooths the data distribution at every positive time but the score may diverge as the noise level approaches zero. For compactly supported or manifold-supported data, score derivatives can scale as inverse powers of the diffusion variance. Early stopping at a positive diffusion time ptp_t7 avoids the singular endpoint and restores regularity on ptp_t8. The resulting approximation must then account separately for the terminal smoothing bias.

The mismatch between density-weighted score training and deterministic-flow stability is fundamental. Score matching commonly controls

ptp_t9

whereas trajectory stability can depend on an unweighted velocity error. Errors in low-density regions receive little weight during score training but may still affect the deterministic flow. Under Sobolev regularity, stability estimates are generally logarithmic in the velocity error; under stronger Lipschitz assumptions, linear trajectory estimates are available (Cont, 28 Jul 2026).

For deterministic samplers, ∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.0 score accuracy alone is insufficient for total-variation control. Small oscillatory score perturbations can produce large changes in the density of a deterministic pushforward, including singular outputs. Jacobian control is therefore essential. This explains the appearance of mean-Jacobian error in total-variation convergence theorems (Li et al., 2024).

6. Extensions and applications

The probability-flow construction extends beyond finite-dimensional Euclidean diffusion models.

In infinite-dimensional Hilbert spaces, ordinary Lebesgue densities and pointwise scores generally do not exist. The appropriate replacement is the Fomin logarithmic gradient of a measure along its Cameron–Martin space. For a Hilbert-space SDE

∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.1

with ∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.2, the infinite-dimensional probability-flow ODE is

∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.3

A weak integration-by-parts identity shows that this deterministic flow has the same marginal laws as the stochastic process under appropriate assumptions. Fourier neural operators can approximate the functional score, and experiments have applied the construction to function generation, diffusion–reaction equations, and heat-equation trajectories (Na et al., 13 Mar 2025).

For Fokker–Planck equations with state-dependent diffusion, the probability-flow velocity includes the diffusion-divergence correction. Self-Consistent Probability Flow methods parameterize the velocity directly and minimize a residual of the first-order continuity equation generated by the same continuous normalizing flow. Hutchinson trace estimation avoids explicit second-order derivatives, while adaptive sampling concentrates collocation points on the evolving probability mass (Wu et al., 22 Dec 2025).

The framework has also been used for deterministic particle methods. Rewriting overdamped Langevin dynamics as a continuity equation gives

∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.4

Replacing ∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.5 by a kernel density estimator yields an interacting particle flow with an attractive target-score term and a repulsive KDE-score term. This connects probability-flow transport with variational inference, kernel mean embeddings, quadrature, and sequential Monte Carlo, although global convergence of the finite-particle system is not established in general (Klebanov, 2024).

The deterministic flow also provides a useful setting for studying memorization. In idealized minimum-norm shallow ReLU denoisers, probability-flow trajectories can converge to training points, sums of training points, or intermediate points on an implicit hyperbox-like manifold. The diffusion-time schedule acts as a dynamical early-stopping mechanism: sufficiently long low-noise evolution resolves trajectories toward vertices, whereas shorter evolution can terminate on edges or faces (Zeno et al., 23 Jun 2025).

Finally, stochastic interpolants provide a broader probability-flow construction. If

∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.6

with ∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.7 Gaussian and ∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.8 distributed according to a target ∂tpt=−∇⋅(fpt)+12σ(t)2Δpt.\partial_t p_t = -\nabla\cdot(f p_t) +\frac{1}{2}\sigma(t)^2\Delta p_t.9, then

∇pt=pt∇log⁡pt\nabla p_t=p_t\nabla\log p_t0

defines a deterministic flow with the same marginals as the interpolant. Annealed Langevin Monte Carlo can estimate the conditional expectations through importance-weighted particles and Jarzynski-type reweighting, providing a route to probability-flow sampling for multimodal and unnormalized targets (Huang, 21 Apr 2026).

The central limitation across these settings is that the probability-flow ODE is not automatically valid merely because a Fokker–Planck equation can be algebraically rewritten as a continuity equation. A rigorous deterministic representation requires existence and stability of the associated flow, adequate regularity of the score or logarithmic gradient, control of divergence and growth, appropriate treatment of singular endpoints, and numerical methods whose discretization error is subordinate to score and density-estimation errors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (16)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Probability Flow ODE.