Probability Flow ODE
- "Probability Flow ODEs are deterministic ordinary differential equations whose time-marginal probability distributions align with a corresponding diffusion process, to create time-evolving data distributions for practical generative modeling."
- Probability Flow ODEs are fundamental in generative models like score-based generative modeling, deterministic diffusion samplers, and continuous normalizing flows.
- Applications of Probability Flow ODEs range from deterministic sampling and likelihood evaluation to invertible mappings in data analysis using exact diffusion trajectories.
A probability flow ODE is a deterministic ordinary differential equation whose velocity field is constructed so that its time-marginal probability distributions coincide with those of a corresponding diffusion process. For a diffusion
with density , the associated probability-flow velocity is
and the deterministic dynamics are
Under appropriate well-posedness conditions, for every time , although the diffusion and ODE have different path laws: diffusion trajectories are stochastic, whereas ODE trajectories are deterministic conditional on their initial state. The construction is central to score-based generative modeling, deterministic diffusion samplers, continuous normalizing flows, density estimation, and transport-based analysis.
1. Mathematical construction
The marginal density of the diffusion satisfies the Fokker–Planck equation
Using
and
the equation can be rewritten as
0
This is a continuity equation with velocity
1
The characteristic equation is therefore
2
The factor 3 distinguishes the probability-flow ODE from the reverse-time SDE. The reverse SDE contains the full score correction and stochastic forcing, whereas the probability-flow ODE contains half the score correction and no Brownian term. Both processes have the same one-time marginals when the exact score and exact dynamics are used.
For a spatially homogeneous diffusion matrix 4, the same construction is
5
For a state-dependent diffusion tensor 6, the probability-flow velocity becomes
7
where
8
The additional 9 term is essential when the diffusion tensor varies spatially (Wu et al., 22 Dec 2025).
The continuity-equation representation also gives the pushforward relation
0
whenever the ODE generates a suitable flow. Thus, if 1, then 2 has the same law as 3 at every time. The equality concerns one-time marginals, not pathwise trajectories.
2. Score-based diffusion modeling
In score-based generative modeling, the forward diffusion transforms a data distribution into an approximately Gaussian distribution. A reverse-time SDE uses a learned score
4
to transport Gaussian noise toward data. The probability-flow ODE replaces this stochastic reverse process by deterministic dynamics. For the Ornstein–Uhlenbeck process
5
the reverse-time probability-flow ODE is
6
where 7 denotes the forward marginal. With a learned score, the implemented field is
8
For a variance-preserving diffusion
9
the reverse probability-flow ODE has the form
0
For a variance-exploding diffusion with 1, the forward-oriented probability-flow equation is
2
which is integrated backward from a high-noise state.
The probability-flow formulation supports deterministic sampling, likelihood evaluation, and invertible data-to-latent mappings. A conventional score model learns the infinitesimal vector field and obtains finite-time transitions by numerical integration. A trajectory model can instead learn the finite-time solution operator directly. Consistency Trajectory Models define a transition map
3
representing the state obtained by following the probability-flow trajectory from time 4 to time 5. The map satisfies
6
which enables arbitrary time-to-time trajectory jumps. Such models encompass score-based sampling and ordinary consistency models as special cases (Kim et al., 2023).
3. Deterministic transport, likelihoods, and invertibility
The deterministic flow induces an instantaneous change-of-variables identity. If
7
then along a trajectory,
8
Consequently, for a data point transported to a terminal latent state,
9
with the sign depending on the chosen time orientation. In high dimensions, the divergence is commonly estimated with Hutchinson’s estimator: 0 where 1 is Gaussian or Rademacher noise.
This makes the probability-flow ODE a deterministic continuous normalizing flow. It can map data to a simple prior and reverse the map for generation. The implementation is exact only in the idealized setting of an exact score, exact divergence evaluation, exact integration, and a correctly specified terminal prior. Practical likelihoods additionally contain score-model, trace-estimation, and numerical-integration errors (Arvinte et al., 2023).
The deterministic character also permits the definition of shared-noise distributional distances. If 2 and 3 are the probability-flow noise-to-data maps associated with two distributions, Probability Flow Distance is defined by
4
The coupling uses the same Gaussian initial noise for both maps. With the identity feature map, the resulting transport cost upper-bounds the Wasserstein distance: 5 Probability Flow Distance has been used to define generalization and memorization errors for diffusion models and to study scaling, early learning, double descent, and bias–variance behavior (Zhang et al., 26 May 2025).
A deterministic PF-ODE can also serve as an encoder. If the forward map 6 is injective and has a measurable inverse on its range, then
7
This property enables change detection by mapping pre-change observations to a Gaussian latent space and applying MMD against the Gaussian null. A stochastic reverse diffusion sampler does not provide a single-valued encoding because fresh noise is injected during sampling (Kraevskiy et al., 24 Aug 2026).
4. Numerical discretization and convergence theory
Probability-flow sampling requires approximating both the score and the ODE solution. Exponential integrators treat the linear drift analytically. For a general variance-preserving field,
8
an exponential Runge–Kutta method integrates the first term exactly and approximates the score-dependent term. This yields first-, second-, and higher-order schemes with global numerical errors of order 9 for a 0-th order method (Huang et al., 16 Jun 2025).
Several convergence regimes have been established.
- Under strong log-concavity and smoothness, exponential-integrator samplers admit non-asymptotic 1 bounds. For variance-preserving schedules, the analyzed iteration complexity is 2, while the analyzed variance-exploding schedules have worse dependence on dimension and accuracy (Gao et al., 2024).
- Under compact support, integrated 3 score error, and regularity assumptions on the learned score, Runge–Kutta samplers admit total-variation bounds whose principal terms have the form
4
where 5 is an integrated score error and 6 is the step size (Huang et al., 2024).
- A sharper deterministic analysis for a particular DDIM-type discretization obtains, up to logarithmic factors,
7
The resulting iteration complexity is 8 under a specialized noise schedule. The analysis does not assume smoothness, log-concavity, or a Lipschitz target score, but it requires control of both score and Jacobian errors (Li et al., 2024).
- For target distributions with low metric entropy or intrinsic dimension 9, a deterministic DDIM/probability-flow sampler can achieve a discretization rate of order
0
up to logarithmic factors and score-estimation terms. The result interprets “adaptivity” as statistical adaptation to low-dimensional support rather than online adaptive step-size selection (Tang et al., 31 Jan 2025).
- For subgaussian targets with 1-Hölder densities, 2, a smoothly regularized kernel score estimator simultaneously controls score and mean-Jacobian errors. The resulting probability-flow sampler achieves the density-estimation minimax rate
3
in total variation, up to logarithmic factors (Cai et al., 12 Mar 2025).
A recurring distinction is between score error and numerical error. Refining the ODE solver reduces discretization error but cannot remove score-estimation error. High-order methods are therefore useful only until numerical error becomes smaller than the error induced by the learned score.
5. Well-posedness, endpoint singularities, and score stability
The formal rewriting of the Fokker–Planck equation as a continuity equation does not by itself guarantee that the probability-flow ODE generates a well-defined deterministic flow. Eulerian well-posedness of the density equation and Lagrangian well-posedness of the characteristic ODE are distinct.
Under weak assumptions on the drift, the Fokker–Planck equation can have a unique density flow in an energy class. To obtain a deterministic flow, stronger conditions are required, such as Sobolev or bounded-variation regularity of the velocity, linear growth, and one-sided bounds on its divergence. The DiPerna–Lions–Ambrosio theory then yields a regular Lagrangian flow 4 satisfying
5
The compressibility bound depends on the negative part of the divergence: 6
Reverse-time sampling and invertibility require two-sided divergence control. Forward marginal matching under a one-sided bound does not automatically imply reverse well-posedness, global invertibility, or exact change-of-variables formulas (Cont, 28 Jul 2026).
Endpoint singularities arise because diffusion smooths the data distribution at every positive time but the score may diverge as the noise level approaches zero. For compactly supported or manifold-supported data, score derivatives can scale as inverse powers of the diffusion variance. Early stopping at a positive diffusion time 7 avoids the singular endpoint and restores regularity on 8. The resulting approximation must then account separately for the terminal smoothing bias.
The mismatch between density-weighted score training and deterministic-flow stability is fundamental. Score matching commonly controls
9
whereas trajectory stability can depend on an unweighted velocity error. Errors in low-density regions receive little weight during score training but may still affect the deterministic flow. Under Sobolev regularity, stability estimates are generally logarithmic in the velocity error; under stronger Lipschitz assumptions, linear trajectory estimates are available (Cont, 28 Jul 2026).
For deterministic samplers, 0 score accuracy alone is insufficient for total-variation control. Small oscillatory score perturbations can produce large changes in the density of a deterministic pushforward, including singular outputs. Jacobian control is therefore essential. This explains the appearance of mean-Jacobian error in total-variation convergence theorems (Li et al., 2024).
6. Extensions and applications
The probability-flow construction extends beyond finite-dimensional Euclidean diffusion models.
In infinite-dimensional Hilbert spaces, ordinary Lebesgue densities and pointwise scores generally do not exist. The appropriate replacement is the Fomin logarithmic gradient of a measure along its Cameron–Martin space. For a Hilbert-space SDE
1
with 2, the infinite-dimensional probability-flow ODE is
3
A weak integration-by-parts identity shows that this deterministic flow has the same marginal laws as the stochastic process under appropriate assumptions. Fourier neural operators can approximate the functional score, and experiments have applied the construction to function generation, diffusion–reaction equations, and heat-equation trajectories (Na et al., 13 Mar 2025).
For Fokker–Planck equations with state-dependent diffusion, the probability-flow velocity includes the diffusion-divergence correction. Self-Consistent Probability Flow methods parameterize the velocity directly and minimize a residual of the first-order continuity equation generated by the same continuous normalizing flow. Hutchinson trace estimation avoids explicit second-order derivatives, while adaptive sampling concentrates collocation points on the evolving probability mass (Wu et al., 22 Dec 2025).
The framework has also been used for deterministic particle methods. Rewriting overdamped Langevin dynamics as a continuity equation gives
4
Replacing 5 by a kernel density estimator yields an interacting particle flow with an attractive target-score term and a repulsive KDE-score term. This connects probability-flow transport with variational inference, kernel mean embeddings, quadrature, and sequential Monte Carlo, although global convergence of the finite-particle system is not established in general (Klebanov, 2024).
The deterministic flow also provides a useful setting for studying memorization. In idealized minimum-norm shallow ReLU denoisers, probability-flow trajectories can converge to training points, sums of training points, or intermediate points on an implicit hyperbox-like manifold. The diffusion-time schedule acts as a dynamical early-stopping mechanism: sufficiently long low-noise evolution resolves trajectories toward vertices, whereas shorter evolution can terminate on edges or faces (Zeno et al., 23 Jun 2025).
Finally, stochastic interpolants provide a broader probability-flow construction. If
6
with 7 Gaussian and 8 distributed according to a target 9, then
0
defines a deterministic flow with the same marginals as the interpolant. Annealed Langevin Monte Carlo can estimate the conditional expectations through importance-weighted particles and Jarzynski-type reweighting, providing a route to probability-flow sampling for multimodal and unnormalized targets (Huang, 21 Apr 2026).
The central limitation across these settings is that the probability-flow ODE is not automatically valid merely because a Fokker–Planck equation can be algebraically rewritten as a continuity equation. A rigorous deterministic representation requires existence and stability of the associated flow, adequate regularity of the score or logarithmic gradient, control of divergence and growth, appropriate treatment of singular endpoints, and numerical methods whose discretization error is subordinate to score and density-estimation errors.