---
title: Rectified Flow Models in Generative Modeling
url: https://www.emergentmind.com/topics/rectified-flow-models
type: topic
---

# Rectified Flow Models in Generative Modeling

Rectified flow models are generative models that transport a simple source distribution (e.g., standard Gaussian) to a target data distribution (such as images, speech, or discrete text) by learning an ordinary differential equation (ODE) whose trajectories are as “straight” as possible in sample space. Unlike score-based diffusion models, which involve stochastic differential equations with hundreds of sampling steps, rectified flow learns a time-dependent velocity field regressed to straight-line displacements, enabling high-quality generation with drastically fewer steps. The rectified flow paradigm has seen rapid expansion and innovation, spanning continuous, discrete, hierarchical, and multi-modal domains, and has initiated new directions in optimal transport, model distillation, editing, and plug-and-play applications.

## 1. Mathematical Foundation and Core Objective

The core objective of rectified flow is to learn a velocity field $v_\theta(x, t)$ that transforms a base distribution $\pi_0$ (often $\mathcal{N}(0, I)$) into a target distribution $\pi_1$ (e.g., images, audio). For paired samples $(x_0, x_1) \sim \pi_0 \times \pi_1$, the straight-line interpolation is:
\[
x_t = (1-t)x_0 + t x_1, \quad t \in [0, 1].
\]
The model regresses $v_\theta$ to approximate the displacement direction $(x_1 - x_0)$ at each $x_t$:
\[
\min_\theta \int_0^1 \mathbb{E}_{x_0, x_1} \left\| (x_1 - x_0) - v_\theta(x_t, t) \right\|^2 dt.
\]
The generative ODE is:
\[
\frac{d Z_t}{dt} = v_\theta(Z_t, t), \quad Z_0 \sim \pi_0.
\]
If $v_\theta$ matches $(x_1 - x_0)$ everywhere, trajectories between $Z_0$ and $Z_1$ traced by the ODE are straight, and the final distribution of $Z_1$ matches $\pi_1$ exactly [2209.03003, 2309.17056].

Distinct from score-based diffusion, which models the (usually non-linear, stochastic) score field $\nabla_x \log p_t(x)$, rectified flow directly fits the conditional expectation of the data displacement, resulting in deterministic, straight or nearly-straight paths.

## 2. Recursive Rectification and Trajectory Straightening

Early rectified flow models observed that a single regression (1-RF) typically yields trajectories that are close to straight but not exact. By recursively re-coupling synthetic pairs generated from the current flow and retraining (the "reflow" step), each successive rectification (2-RF, 3-RF, etc.) yields increasingly straight ODE paths:
- After each reflow step, the conditional expectation velocity is updated using pairs from the previous flow's samples.
- Theoretical results prove a monotonic decrease in convex transport costs (e.g., Wasserstein distance) and decreasing "straightness" error with each recursion [2209.03003, 2410.14949].
- Empirical studies confirm that after one or two rectified steps, even a single-step Euler integration often suffices for high-fidelity generation (e.g., image FID, TTS MOS) [2309.17056, 2209.03003].

The recursive process:
- $(Z_0^{k}, Z_1^{k}) \gets$ rectify $(Z_0^{k-1}, Z_1^{k-1})$
- $v_\theta^{(k+1)}$ regressed as above on the new pairs

This procedure underlies one-step distillation and efficient few-step models [2407.12718].

## 3. Algorithmic and Architectural Details

The parameterization of $v_\theta(x, t)$ varies by domain:
- **Images/audio**: U-Nets, convolutional backbones, or diffusion-transformers (DiT), with time encoded via sinusoidal embeddings [2503.09242, 2412.05003].
- **Text/layouts**: Token-based transformers, often utilizing object/meta-embeddings and prompt conditioning for text-to-layout [2412.05003].
- **Fluid/PDEs**: Velocity networks, sometimes with FiLM conditioning on initial/boundary data and multi-head attention [2506.03111].
- **Discrete data**: Rectified discrete flows operate by recoupling discrete variables and leveraging conditional total correlation as an error metric [2507.15897].

The typical training pseudocode:
1. Sample $(x_0, x_1) \sim \pi_0 \times \pi_1$, $t \sim U[0, 1]$.
2. Compute $x_t = (1-t)x_0 + t x_1$.
3. Output $\hat{v} = v_\theta(x_t, t)$; minimize $\| x_1 - x_0 - \hat{v} \|^2$.
4. Update $\theta$.

For conditional generation (e.g., text-to-image, TTS), $v_\theta$ receives additional conditioning inputs (CLIP, FastSpeech, language embeddings) [2309.17056, 2503.09242].

## 4. Theoretical Guarantees, Optimal Transport, and Extensions

Rectified flow uniquely preserves marginals at every interpolation time, and, unlike extrinsic penalty-based OT solvers, maintains valid coupling throughout all recursive steps:
- **Marginal preservation**: The ODE-sampled points have the correct time-marginals by construction, irrespective of neural network error [2209.14577].
- **Cost monotonicity**: Any convex transport cost strictly decreases with each rectification; the iterative procedure converges to the optimal transport plan for $c(x)$ under mild regularity.
- **Strong straightness**: Successive rectification yields couplings where increments $Z_1 - Z_0$ are predictable functions of the chord $t Z_1 + (1-t) Z_0$, a stronger criterion than average curvature [2410.14949].
- **Extensions**: Hierarchical formulations model higher derivatives (velocity, acceleration), permitting intersecting ODE paths, further reducing trajectory curvature and number of function evaluations [2502.17436].
- **Balanced/Conic flows**: Recent approaches integrate real samples and geometric constraints (slerp-cones) to prevent error accumulation, sampling drift, or model collapse when relying heavily on synthetic pairs during recursive reflow [2510.25229, 2412.08175].

## 5. Empirical Applications and Benchmarks

Rectified flow models demonstrate state-of-the-art or near-SOTA performance in a wide array of generative modeling tasks:
- **Image Generation**: Euler or RK solvers with as few as 1–8 steps suffice for high-fidelity image synthesis (e.g., NAMI, Flux, InstaFlow) [2503.09242, 2209.03003, 2406.03293]. On CIFAR-10, one-step flows achieve FID$<$5.1 at $\sim 15$M parameters [2407.12718]. Progressive and multiresolution architectures (as in NAMI) further accelerate inference [2503.09242].
- **Text-to-Speech**: ReFlow-TTS matches or surpasses diffusion-based models with MOS$\approx$4.5 in $<1\%$ of the sampling time [2309.17056].
- **Fluid Simulation**: Rectified flows for multiscale PDEs recover fine statistical/moment properties, require only 4–8 inference steps, and outperform or match SDE-based conditional flows [2506.03111].
- **Discrete Data**: Rectified discrete flows (ReDi) guarantee monotonic reduction in conditional total correlation of the modeled coupling, enabling one-step or few-step discrete data synthesis with large IS/FID improvements [2507.15897].
- **Layout and Multimodal Generation**: SLayR and JanusFlow prove that transformer-based rectified flows can be embedded into LLMs for layout or joint understanding/generation with competitive quality, parameter, and speed tradeoffs [2412.05003, 2411.07975].
- **Editing and Control**: Semantic attribute disentanglement in FluxSpace demonstrates editable latent representations within rectified flow transformers [2412.09611]. Plug-and-play prior, zero-shot editing, and inversion are enabled by deterministic, time-symmetric ODE properties [2406.03293, 2412.00100].

## 6. Guidance, Constraints, and Sampling Enhancements

Classifier-free guidance (CFG), a mainstay in diffusion models, is not natively stable in rectified flow due to drift off the learned transport manifold. Recent advances propose geometry-aware, predictor-corrector schemes (Rectified-CFG++) that guarantee marginal proximity and bounded manifold deviation, enabling strong prompt adherence and artifact-free images across Flux, SD3/SD3.5, and Lumina [2510.07631]. Boundary-enforced variants further impose analytic constraints on $v_\theta(x, t)$ at $t=0, 1$, eliminating endpoint bias and stabilizing ODE/SDE sampling [2506.15864]. Steering via the vector field can be performed in a deterministic, gradient-skipping fashion, facilitating tasks like classifier guidance, inpainting, and inverse-problem solving in a unified, memory- and compute-efficient manner [2412.00100].

## 7. Open Challenges and Future Directions

Active research addresses several limitations and new directions:
- **Model collapse**: Iterative reflow steps relying solely on synthetic pairs induce model collapse, analogous to DAE degeneration; interleaving real pairs or careful SDE reverse-simulation avoids rank-loss and sample diversity degradation [2412.08175, 2510.25229].
- **Data efficiency and real-sample anchoring**: Balanced Conic Rectified Flow sharply reduces the need for millions of synthetic pairs and improves robustness along real-image reversals [2510.25229].
- **Scalability and compression**: SlimFlow extends rectified flow to highly compressed, small-footprint models with one-step sampling, via annealing and flow-guided distillation [2407.12718].
- **Hierarchical and multimodal flows**: Hierarchical rectified flows further minimize the number of NFEs by modeling full velocity (and higher derivative) distributions, permitting path intersection [2502.17436].
- **Connections to optimal transport**: The c-rectified flow generalizes the methodology to directly solve Monge–Kantorovich OT for user-specified costs while preserving marginal constraints at every step [2209.14577].
- **Domain transfer, adaptation, and fine-grained editing**: Applications in domain adaptation, fine-grained style transfer, and inversion-free semantic editing are ongoing [2412.09611, 2406.03293, 2412.00100].

Future advances are likely to focus on improved solver-adaptive schemes, further analysis of straightness and marginality in high dimensions, enhanced semantic disentanglement, and expansion into multimodal, hierarchical, and plug-and-play generative tasks.

---

**References:** [2209.03003], [2309.17056], [2209.14577], [2410.14949], [2407.12718], [2503.09242], [2412.00100], [2412.09611], [2412.05003], [2506.03111], [2510.25229], [2507.15897], [2502.17436], [2506.15864], [2412.08175], [2511.20307], [2510.07631], [2406.03293], [2411.07975]

Source: https://www.emergentmind.com/topics/rectified-flow-models