Flow-Matching Velocity Parameterizations

• Flow-matching velocity parameterizations are neural ODE vector fields that define continuous trajectories between noise and data distributions.
• Average and transition-based methods mitigate discretization errors and integration lag by leveraging interval-averaged velocities and semigroup consistency.
• Advanced strategies such as MeanCache, self-consistency, and Lagrangian approaches boost efficiency and accuracy in high-dimensional generative models.

Flow-Matching Velocity Parameterizations

Flow-matching velocity parameterizations define the class of neural vector fields used to specify, train, and exploit continuous-time ordinary differential equation (ODE) dynamics in generative modeling, notably within Flow Matching (FM) and related generative frameworks. These parameterizations dictate not only how learned ODEs interpolate between source and target distributions in data space, but also how one can leverage novel architectures, average or transition structures, and optimization principles to achieve accelerated, stable, or high-fidelity generative modeling.

1. Standard Instantaneous Velocity Parameterization

The classical flow-matching framework employs an instantaneous velocity field

$$v_\theta(x_t, t): \mathbb{R}^d \times [0,1] \to \mathbb{R}^d$$

learned to minimize the regression loss

$$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$

where $x_t = (1-t)x_0 + t x_1$ for a coupling between noise (source) and data (target) distributions. The instantaneous velocity at $x_t$ is the constant direction $x_1 - x_0$, corresponding to straight-line trajectories in state space (Lipman et al., 2024). At inference, sample generation proceeds by numerically integrating

$\frac{dx_t}{dt} = v_\theta(x_t, t)$

from $t=0$ (noise) to $t=1$ (data). Architectures include multilayer perceptrons (MLPs), U-Nets, and attention-based models, typically with sinusoidal or learned Fourier time embeddings (Lipman et al., 2024, He et al., 8 Jun 2026).

Limitations and Biases

This parameterization—prevalent in rectified flow models—induces two significant issues:

• Discretization Error: Single-step Euler integration is generally inadequate since true dynamics often deviate from the linear interpolant. High step counts are required for trajectory fidelity (Yang et al., 19 Sep 2025).
• Velocity Deficit: The MSE-based fit of $v_\theta$ to ambiguous targets (multiple pairs $(x_0, x_1)$ ending at the same $$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$0) leads to systematic underestimation in high dimensions, resulting in “integration lag” where samples fail to reach the data manifold (Li et al., 14 May 2026).

2. Average-Velocity and MeanFlow Parameterizations

To overcome these limitations, recent work advocates modeling average (interval) velocities rather than instantaneous direction. For times $$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$1, the interval-average velocity is defined as

$$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$2

and single-step propagation is realized by

$$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$3

Direct regression against this interval-average is employed in MeanFlow and “COSE” frameworks (Yang et al., 19 Sep 2025, Ma, 16 Mar 2026). Crucially, the mean velocity can be linked to endpoint displacements in the linear interpolant, providing tractable training targets.

JVP-Free Velocity Composition

Computing the target mean velocity naively requires expensive Jacobian–vector product (JVP) computations for derivatives with respect to time. “COSE” introduces a semigroup-based velocity-composition identity:

$$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$4

for $$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$5, enabling all training terms to be estimated by forward passes with no JVP, dramatically reducing memory and computational costs (Yang et al., 19 Sep 2025).

Empirical Impact

Average-velocity parameterizations enable exact one-step or few-step sampling, mitigate discretization error, and significantly accelerate inference (e.g., up to $$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$6 faster, $$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$7 lower memory in COSE for speech enhancement), without degradation in generative quality (Yang et al., 19 Sep 2025, Ma, 16 Mar 2026).

3. Transition and Terminal Velocity Parameterizations

Transition Flow Matching (TFM)

Rather than local or interval velocities, Transition Flow Matching directly models the global state-to-state map:

$$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$8

subject to the integral identity

$$\mathcal{L}_\text{FM}(\theta) = \mathbb{E}_{t \sim U[0,1],\, x_0, x_1} \| v_\theta(x_t, t) - (x_1 - x_0) \|^2$$9

The corresponding loss matches the “transition flow” (global endpoint) via regression, with the mean velocity recoverable as

$x_t = (1-t)x_0 + t x_1$0

TFM supports exact one-step and arbitrary-step sampling, achieving state-of-the-art fidelity (e.g., FID 2.77 on CIFAR-10 NFE=1; 3.02 on ImageNet-256), and strictly generalizes mean-velocity formulations (Ma, 16 Mar 2026).

Terminal Velocity Matching (TVM)

TVM introduces a two-time map $x_t = (1-t)x_0 + t x_1$1 representing the integrated flow from $x_t = (1-t)x_0 + t x_1$2 to $x_t = (1-t)x_0 + t x_1$3, and enforces via the “terminal velocity” identity

$x_t = (1-t)x_0 + t x_1$4

with displacement and regularization terms. TVM provides a provable upper bound on Wasserstein-2 error under Lipschitz smoothness and achieves top performance in one- and few-step generation (e.g., 3.29 FID on ImageNet 256x256 in a single function evaluation) (Zhou et al., 24 Nov 2025).

4. Advanced Extensions and Alternative Parameterizations

Blockwise and Reaction-Diffusion Velocity Fields

Blockwise Flow Matching partitions the time interval into $x_t = (1-t)x_0 + t x_1$5 disjoint blocks, each assigned a specialized velocity network $x_t = (1-t)x_0 + t x_1$6. This enhances inference efficiency and feature specialization and allows the integration of semantic guidance modules for class-conditional or feature-aligned flows (Park et al., 24 Oct 2025). In parallel, Graph Flow Matching (GFM) explcitly incorporates local batch or example context via a reaction–diffusion decomposition:

$x_t = (1-t)x_0 + t x_1$7

where $x_t = (1-t)x_0 + t x_1$8 is a lightweight graph neural network module aggregating over a dynamically built batch adjacency (Siddiqui et al., 30 May 2025).

Stochastic and Multi-Modal Velocity Parameterizations

Instead of deterministic, constant velocity fields, Discretized Rectified Flow (Discretized-RF) injects stepwise stochasticity into momentum fields along segmented subpaths, thus improving sample diversity and multi-scale noise handling. The neural velocity field $x_t = (1-t)x_0 + t x_1$9 is trained to match random subpath velocities, yielding a robust combination of efficiency and sample diversity (Ma et al., 10 Jun 2025).

Variational Rectified Flow Matching attaches latent variables $x_t$0 to each $x_t$1, parameterizing $x_t$2. The resulting Gaussian mixture over velocities captures flow ambiguity directly, overcoming the MSE-based mode collapse inherent to classic rectified flow, and yielding notable gains in both sample fidelity and controllable generation (Guo et al., 13 Feb 2025).

Lagrangian, Optimal-Transport, and Self-Consistency Approaches

Lagrangian Flow Matching generalizes the path and velocity design by positing a general least-action principle:

$x_t$3

subject to continuity, with $x_t$4 (kinetic, harmonic, potential, etc.) encoding trajectory geometry. Special cases recover straight-line velocities (kinetic), harmonic-geodesic velocities (variance-preserving diffusion), or anisotropic curved paths. This yields a unified simulation-free training procedure for new flow classes via direct regression onto closed-form geodesics and velocities (Du et al., 14 May 2026).

Self-Consistent Velocity Matching (SCVM) turns the mass-conserving PDE constraint into a velocity field fixed-point condition, implemented via an outer-iterative, biased-gradient scheme. Neural velocity maps $x_t$5 or invertible flows $x_t$6 are directly optimized to satisfy the self-consistency constraint pointwise along sample paths, enabling discretization-free, high-dimensional scalability (Li et al., 2023).

5. Bridging Parameterizations and Accelerated Inference

MeanCache and Average-Velocity Caching

Acceleration of inference in large-scale flow models is enabled by leveraging interval-average (rather than instantaneous) velocity parameterizations. The MeanCache framework constructs JVP-based average-velocity estimators over cached time intervals:

$x_t$7

with a trajectory-stability-driven cache scheduling strategy for dynamic edge weighting and peak-suppression, supporting $x_t$8 speed-ups and improved fidelity on large image/video models (Gao et al., 27 Jan 2026).

Diffusion Model ↔ Flow Matching Alignment

Diff2Flow presents a systematic mapping from pre-trained diffusion (score-based) models to FM-compatible velocity parameterizations via analytic transformation of trajectory and time variables, allowing seamless finetuning or LoRA adaptation. The analytic pushforward delivers closed-form, Jacobian-free FM velocity formulas, enabling parameter-efficient, training-compute-neutral transfer across the diffusion–flow-matching paradigm divide (Schusterbauer et al., 2 Jun 2025).

Early-Stage Energy Correction

Compensating for the intrinsic "velocity deficit" (underestimated kinetic energy) is critical for high-dimensional FM. Magnitude-Aware Flow Matching augments the training loss to supervise the $x_t$9 norm, while the Scale Schedule Corrector applies a simple time-varying rescaling at inference. These techniques dramatically enhance sampling efficiency, allowing 5$x_1 - x_0$0 reduction in step count without fidelity loss (e.g., FID drops from 13.68 to 7.58 on ImageNet-1k/256 with SSC) (Li et al., 14 May 2026).

6. Architectures, Conditioning, and Practical Recommendations

A broad spectrum of architectures is employed, from basic MLPs to U-Nets, transformers, and graph neural modules. Time-conditioning is typically injected via sinusoidal embeddings (Fourier), positional encoding, or learned features. Conditional variants concatenate class labels, global semantic features, or context vectors, with classifier-free guidance implemented by dual-output networks (Lipman et al., 2024, Park et al., 24 Oct 2025).

Empirical ablations stress that blockwise specialization, interval consistency objectives, and JVP-free average-velocity losses optimize the quality/speed Pareto frontier. For high-dimensional or time-critical applications, interval-average and transition-based parameterizations, possibly coupled with inference schedulers (e.g., MeanCache), are strongly favored.

7. Conceptual and Empirical Lessons

• Integral/average-velocity modeling is foundational for eliminating discretization error in one/few-step ODE-based generative modeling.
• Semigroup consistency (e.g., velocity composition) provides a robust alternative to derivative-based methods, with significant efficiency and portability advantages.
• Self-consistency across scales via random partitioning or global map learning encourages models to capture globally straightened or curved flows, dependent on Lagrangian design.
• Lipschitz continuity is essential for theoretical bounds and model stability, requiring modified normalization and regularization in deep backbones (Zhou et al., 24 Nov 2025).
• In high dimensions, controlling the magnitude and diversity of velocity fields is indispensable for avoiding integration lag, mode collapse, and fidelity loss.

Flow-matching velocity parameterizations have evolved rapidly, moving from pointwise, instantaneous fields to sophisticated, context-aware, interval-average, and transition-based formulations. These advances jointly underpin the leap in both efficiency (one/few-step sampling, inference caching) and sample quality now observed across imaging, speech, robotics, and scientific generation tasks (Yang et al., 19 Sep 2025, Ma, 16 Mar 2026, Zhou et al., 24 Nov 2025, Gao et al., 27 Jan 2026).