---
title: Velocity Matching in Flow Models
url: https://www.emergentmind.com/topics/velocity-matching-vm
type: topic
---

# Velocity Matching in Flow Models

Velocity Matching (VM) is a regression principle for learning time-dependent vector fields that transport probability mass between a source distribution and a target distribution along a prescribed probability path. In contemporary flow-based generative modeling, VM is typically instantiated inside Flow Matching (FM), Conditional Flow Matching (CFM), or rectified-flow formulations, where a neural field is trained to match an analytically defined velocity target along interpolants between noise and data. The learned field then defines a probability-flow ordinary differential equation (ODE), or related transition operator, whose integration generates samples. Across the recent literature, VM has become both a unifying formulation for continuous-time generative transport and a locus for several extensions addressing multi-modality, target variance, trajectory straightness, low-step generation, and domain-specific dynamics [2502.09616, 2602.05435, 2603.15689].

## 1. Formal definition and probabilistic setting

VM is defined on a continuous path of densities $\{p_t\}_{t\in[0,1]}$ connecting a simple source distribution to a target distribution. The state variable $X_t$ satisfies $X_t \sim p_t$, and the learned time-dependent field $v(x_t,t)$ drives deterministic dynamics through
$$
\frac{d x_t}{dt} = v(x_t,t).
$$
The corresponding density evolution satisfies the continuity equation
$$
\partial_t p_t(x_t) = -\,\nabla \cdot \big(p_t(x_t)\,v(x_t,t)\big).
$$
In conditional formulations, one introduces an auxiliary variable $Z$ and a conditional path $p_{t\mid Z}(\cdot\mid z)$ with conditional velocity field $v(x_t,t\mid z)$, while the marginal velocity is the conditional expectation
$$
v(x_t,t)=\mathbb{E}\!\left[v(X_t,t\mid Z)\mid X_t=x_t\right].
$$
A standard FM choice sets $Z=(X_0,X_1)$, with independent coupling $X_0\sim p_0$, $X_1\sim p_1$, and linear interpolant
$$
X_t=(1-t)X_0+tX_1,
$$
for which the conditional velocity is constant in time:
$$
v(X_t,t\mid Z)=X_1-X_0.
$$
This formulation is central in FM, rectified flow, and several later one-step generalizations [2603.15689, 2509.24936].

A closely related stochastic-interpolant formulation writes
$$
\hat{x}_t=\alpha_t \hat{x}+\sigma_t \epsilon,
$$
with boundary conditions $\alpha_1=\sigma_0=1$ and $\alpha_0=\sigma_1=0$, and often uses the linear schedule $\alpha_t=t$, $\sigma_t=1-t$. The corresponding reference velocity is
$$
\hat{v}(\hat{x},t,\epsilon)=\dot{\alpha}_t \hat{x}+\dot{\sigma}_t \epsilon.
$$
This version is widely used in latent-space FM implementations and makes explicit that VM is the regression of a neural velocity field toward a path-induced analytic target [2511.18942].

## 2. Canonical objectives in Flow Matching and rectified flow

The basic VM loss is a mean-squared regression objective. In the CFM form,
$$
\min_\theta \ \mathbb{E}_{z\sim\pi,\ t\sim\mathrm{Unif}[0,1],\ x\sim p_t(\cdot\mid z)}
\big[\|v_\theta(x,t)-v_t(x\mid z)\|^2\big].
$$
Under the linear bridge $x_t=(1-t)x_0+t x_1$, this becomes regression to the constant displacement target $x_1-x_0$. In the latent-space FM form used by VeCoR,
$$
L^{(\mathrm{FM})}(\theta)=\mathbb{E}_{t,\hat{x},\epsilon}
\big[\|v_\theta(\hat{x}_t,t)-\hat{v}(\hat{x},t,\epsilon)\|^2\big].
$$
The empirical loss is the corresponding finite-sample average over training tuples. The global minimizer of the conditional objective is the marginal velocity field, which gives VM its standard “train on conditional targets, recover marginal transport” interpretation [2511.18942, 2602.05435].

Rectified-flow formulations use the same principle but emphasize linear interpolants and straight paths. A common construction sets
$$
x_t=(1-t)x_0+t x_1,\qquad v^*(x_t,t)=x_1-x_0,
$$
and trains
$$
L_{\mathrm{MSE}}(\theta)=\mathbb{E}_{(x_0,x_1)\sim\pi,\ t\sim U[0,1]}
\big[\|v_\theta(x_t,t)-(x_1-x_0)\|_2^2\big].
$$
At inference, samples are generated by integrating the learned probability-flow ODE
$$
\frac{d x}{dt}=v_\theta(x,t),\qquad x(0)\sim p_0.
$$
The same framework also admits a likelihood interpretation through the instantaneous change-of-variables relation
$$
\log p_1(x_1)=\log p_0(x_0)+\int_1^0 \mathrm{div}\,v_\theta(x_t,t)\,dt.
$$
This establishes VM as both a transport-learning objective and a continuous normalizing-flow training rule [2502.09616].

## 3. Ambiguity, variance, and trajectory geometry

A central limitation of VM is that the conditional velocity target can be ambiguous or high-variance at fixed $(x,t)$. In rectified flow, many distinct couplings $(x_0,x_1)$ may satisfy
$$
x_t=(1-t)x_0+t x_1,
$$
yet induce different target velocities $x_1-x_0$. Under an MSE objective, the learned field regresses to the conditional mean rather than the full conditional law, which can cause the vector field to average incompatible directions. This is the basis of the multi-modality critique developed by Variational Rectified Flow Matching, which models a latent-conditioned conditional distribution over velocities rather than only its mean [2502.09616].

A distinct but related issue is target variance. “Stable Velocity” defines the CFM target variance proxy
$$
\mathcal{V}_{\mathrm{CFM}}(t)=
\mathbb{E}_{p_t(x_t)}
\!\left[\mathrm{Tr}\!\left(\mathrm{Cov}_{p_t(x_0\mid x_t)}(v_t(x_t\mid x_0))\right)\right]
=
\mathbb{E}_{q(x_0),\,p_t(x_t\mid x_0)}
\!\left[\|v_t(x_t\mid x_0)-v_t(x_t)\|^2\right].
$$
This work characterizes a low-variance regime near the data distribution, where conditional and marginal velocities nearly coincide, and a high-variance regime near the prior, where optimization becomes difficult [2602.05435].

Another line of analysis identifies a systematic magnitude contraction in the MSE estimator. “The Velocity Deficit” states that for the random target velocity $v\sim p(v_{\text{target}}\mid x_t)$, the conditional-mean solution satisfies
$$
\|\mathbb{E}[v\mid x_t]\|_2^2
<
\mathbb{E}\big[\|v\|_2^2\mid x_t\big],
$$
which is interpreted as underestimation of kinetic energy. In the linear-path setting with independent coupling and zero-mean data, the learned boundary magnitudes reduce to $\|x_0\|_2$ near $t\to 0$ and $\|x_1\|_2$ near $t\to 1$, while the target displacement magnitude is approximately $\sqrt{\|x_0\|_2^2+\|x_1\|_2^2}$. The paper terms the resulting undershoot during numerical integration “Integration Lag” and proposes asymmetric early-time correction [2605.14819].

Trajectory geometry provides a third perspective. OT-based FM methods implicitly pursue straighter trajectories by regressing toward constant velocities under optimal couplings, but OAT-FM argues that constant velocity is sufficient rather than necessary for straightness. It states that a trajectory is straight if and only if the velocity direction is time invariant and the acceleration is everywhere parallel to the velocity, then recasts straightening as a second-order transport problem in sample–velocity product space [2509.24936].

## 4. Major extensions of Velocity Matching

Recent work has extended VM along several axes: target distributions in velocity space, variance reduction, two-sided regularization, higher-order transport, and direct transition learning. The representative variants below are all described explicitly in the literature.

| Variant | Core modification | Representative source |
|---|---|---|
| Variational Rectified Flow Matching | Models a latent-conditioned multimodal velocity law | [2502.09616] |
| VeCoR | Adds contrastive attraction–repulsion supervision in velocity space | [2511.18942] |
| StableVM | Uses multi-reference aggregation under a composite GMM path for unbiased variance reduction | [2602.05435] |
| OAT-FM | Optimizes acceleration transport in sample–velocity product space | [2509.24936] |
| TVM / TFM / Mean-velocity models | Learns terminal-time, average-velocity, or transition operators for few-step generation | [2511.19797, 2603.15689] |
| CCVFM | Replaces an isotropic inner source with a coreset-induced surrogate conditional velocity law | [2605.12951] |

Variational Rectified Flow Matching introduces a latent variable $z\sim p(z)$ and defines
$$
p(v\mid x_t,t,z)=\mathcal{N}(v;v_\theta(x_t,t,z),I),\qquad
p(v\mid x_t,t)=\int p(v\mid x_t,t,z)p(z)\,dz.
$$
With a Gaussian posterior
$$
q_\phi(z\mid x_0,x_1,x_t,t)=\mathcal{N}(z;\mu_\phi,\sigma_\phi),
$$
training maximizes an ELBO whose data-fit term reduces to squared velocity error. This explicitly addresses the fact that “ground-truth” velocities can be multimodal at the same spatio-temporal point [2502.09616].

VeCoR keeps the FM target but augments it with negative velocity candidates. Its empirical loss is
$$
\widehat{L^{(\mathrm{VeCoR})}}
=
\frac{1}{N}\sum_{i=1}^N
\left[
\|v_\theta(\hat{x}_{t^{(i)}},t^{(i)})-\hat{v}^{(i)}_+\|_2^2
-
\lambda\sum_{j=1}^K
\|v_\theta(\hat{x}_{t^{(i)}},t^{(i)})-\hat{v}^{(ij)}_-\|_2^2
\right],
$$
with requirement $\lambda K<1$. The paper interprets this as converting FM from a purely attractive objective into a two-sided attract–repel signal that suppresses off-manifold directions [2511.18942].

StableVM replaces single-reference conditional targets with a self-normalized multi-reference estimator
$$
\widehat{v}_{\mathrm{StableVM}}(x_t;\{x_0^i\}_{i=1}^n)
=
\frac{\sum_{k=1}^n p_t(x_t\mid x_0^k)\,v_t(x_t\mid x_0^k)}
{\sum_{j=1}^n p_t(x_t\mid x_0^j)}.
$$
Its objective is unbiased, has the true marginal velocity as global minimizer, and satisfies
$$
\mathcal{V}_{\mathrm{StableVM}}(t)\le \mathcal{V}_{\mathrm{CFM}}(t),
$$
with an explicit $O(1/n)$ variance bound under stated assumptions [2602.05435].

OAT-FM generalizes the hidden straightening objective of OT-CFM to second-order transport. Its dynamic formulation minimizes squared acceleration under the Vlasov equation in $X\times V$, while the practical upper-level loss combines endpoint velocity alignment and endpoint velocity change penalties. This suggests a shift from “constant-velocity matching” toward “acceleration-aware straightness matching” when the goal is efficient few-step transport [2509.24936].

## 5. One-step generation, flow-map learning, and distillation

A major contemporary theme is the use of VM-derived objectives for one-step or few-step generation. Mean Velocity Flow models define the average velocity over an interval $[t,r]$ as
$$
u(x_t,t,r)=\frac{1}{r-t}\int_t^r v(x_\tau,\tau)\,d\tau,
$$
and exploit the identity
$$
u(x_t,t,r)=v(x_t,t)+(r-t)\frac{d}{dt}u(x_t,t,r)
$$
to learn directly jumpable quantities. Transition Flow Matching (TFM) further promotes the transition flow
$$
X(x_t,t,r)=\mathbb{E}\!\left[X(X_t,t,r\mid Z)\mid X_t=x_t\right],
$$
with the Transition Flow Identity
$$
X(x_t,t,r)=x_{t\to r}+(r-t)\frac{d}{dt}X(x_t,t,r).
$$
Its conditional objective is tractable under the standard linear interpolant and supports one-step generation by
$$
\hat{x}_1=X^\theta(x_0,0,1).
$$
The paper explicitly relates TFM and mean-velocity formulations as dual parameterizations of the same average-motion object [2603.15689].

Terminal Velocity Matching (TVM) reformulates the problem in terms of two-time displacements
$$
\Delta(x_t,t,s):=\psi(x_t,t,s)-x_t,
$$
represented as
$$
\Delta_\theta(x_t,t,s)=(s-t)u_\theta(x_t,t,s).
$$
Its key identity is the terminal-velocity condition
$$
\frac{d}{ds}\Delta(x_t,t,s)=u(\psi(x_t,t,s),s),
$$
which leads to an upper bound of displacement error by terminal-velocity error. The practical TVM loss combines a terminal-time consistency term with an FM boundary term at $s=t$, thereby interpolating between one-step displacement learning and standard FM. The paper also gives a Wasserstein-$2$ upper bound under Lipschitz continuity and reports state-of-the-art one/few-step ImageNet performance from scratch [2511.19797].

Distillation-based approaches re-express VM at selected times. “Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities” defines the Initial/Terminal Velocity Matching loss
$$
\mathcal{L}_{\mathrm{ITVM}}
=
\mathcal{L}_{\mathrm{IIVM}}
+
\mathcal{L}_{\mathrm{IAVM}}
+
\mathcal{L}_{\mathrm{TVM}},
$$
where the student two-timed flow map is parameterized as
$$
\phi_{s,t}^\theta(x)=x+(t-s)v_{s,t}^\theta(x).
$$
The initial terms match the teacher’s instantaneous and short-interval average velocities at time $s$, while the terminal term enforces a consistency relation near $t$ using an EMA-stabilized student target. The paper interprets the initial terms as redundant at the global optimum yet useful because they query the teacher on in-distribution inputs [2505.01169].

## 6. Applications, empirical regimes, and broader significance

VM now spans both generative modeling and scientific dynamics estimation. In image synthesis, it underlies latent-space FM with transformer and U-Net backbones, class-conditional ImageNet generation, text-to-image models, and video generation. Several papers emphasize low-NFE deployment. VeCoR reports improvements “particularly in low-step and lightweight settings”; Stable Velocity develops StableVS for accelerated low-variance-regime sampling; TVM and TFM directly target one-step and few-step synthesis; and “The Velocity Deficit” proposes a training-free Scale Schedule Corrector
$$
\tilde{v}_\theta(x_t,t)=\gamma(t)v_\theta(x_t,t),\qquad
\gamma(t)=s_{\text{start}}(1-t)+s_{\text{end}}t,
$$
together with the training-based Magnitude-Aware Flow Matching objective
$$
\mathcal{L}_{\mathrm{MAFM}}(\theta)
=
\mathbb{E}\big[\|v_\theta(x_t,t)-v_{\text{target}}(x_0,x_1)\|_2^2\big]
+
\lambda(t)\big(\|v_\theta(x_t,t)\|_2-\|x_1-x_0\|_2\big)^2,
$$
with $\lambda(t)=\lambda_0(1-t)$ [2511.18942, 2602.05435, 2605.14819, 2511.19797, 2603.15689].

In hierarchical velocity-space modeling, CCVFM replaces the isotropic Gaussian inner source of hierarchical rectified flow with a coreset-induced Gaussian-mixture surrogate. Its induced conditional velocity law has the closed form
$$
(v\mid x_t,t)=\sum_{b=1}^K \gamma_b(x_t,t)\,
\mathcal{N}(v;m_b(x_t,t),\Lambda_b(t)),
$$
and the correction flow is trained by the VM residual objective
$$
L(\theta)=\mathbb{E}\big[\|f_\theta(V_\tau,\tau,X_t,t)-(V_1-V_0)\|^2\big].
$$
This reframes VM as residual matching from a data-informed source rather than full noise-to-target transport [2605.12951].

Outside image generation, VGFM extends VM to unbalanced single-cell snapshot dynamics by jointly learning a state velocity and a mass-growth field. The continuity equation becomes
$$
\partial_t \rho+\nabla\cdot(\rho v)=r\rho,
$$
and the joint regression loss is
$$
L_{\mathrm{VGFM}}(\theta,\omega)
=
\sum_{i,j}\pi_{ij}\,
\mathbb{E}_{t\sim U(0,1)}
\left[
\|v_\theta(x_t,t)-(x_1^j-x_0^i)\|^2
+
|g_\omega(x_t,t)-\gamma_i|^2
\right].
$$
Here VM is generalized from mass-conserving transport to simultaneous matching of state transition and growth, derived from a semi-relaxed optimal transport interpretation [2505.13413].

A recurring misconception is that VM is merely “MSE on velocities.” The literature suggests a broader view. VM specifies which conditional transport quantity is being regressed, under which interpolant or coupling, and with what geometric, statistical, or dynamical regularization. A plausible implication is that the current diversity of VM variants reflects different answers to the same core question: whether the model should learn a local instantaneous field, an averaged future motion, a displacement map, a full conditional law in velocity space, or a corrected low-variance surrogate of one of these objects. Across FM, rectified flow, transition-flow, and application-specific formulations, VM remains the central mechanism by which continuous-time transport models are rendered trainable [2502.09616, 2509.24936, 2603.15689].

Source: https://www.emergentmind.com/topics/velocity-matching-vm