Rectified Flow Model: Theory & Applications

• Rectified flow models are neural ODE-based approaches that generate transport mappings by aligning data mass along nearly straight paths between distributions.
• They utilize nonlinear least-squares regression to learn a velocity field that ensures marginal preservation and minimizes convex transport costs.
• Recursive rectification (reflow) enhances numerical efficiency and sample quality by progressively straightening trajectories for tasks like image generation and domain transfer.

A rectified flow model is a neural ordinary differential equation (ODE) approach for learning mappings between empirical data distributions that explicitly seeks to straighten the flow of mass in the data space. By aligning generative trajectories with the shortest (straightest) possible paths between distributions, rectified flow models address distribution transport, generative modeling, domain transfer, and related tasks efficiently and with strong theoretical grounding in optimal transport. The method operates via nonlinear least squares regression on linear interpolation trajectories, producing numerically efficient models and, when recursively applied, flows of increasing straightness with provably decreasing convex transport costs.

1. Mathematical and Theoretical Foundations

Rectified flow constructs an ODE-driven transport between two empirical distributions $\pi_0$ and $\pi_1$. The key procedure involves:

• Defining initial couplings $(X_0, X_1)$ from $\pi_0$ and $\pi_1$ respectively, and considering the linear interpolation $X_t = t X_1 + (1-t) X_0$ for $t \in [0,1]$.
• Learning a velocity field $v(x,t)$ such that the ODE

$dZ_t = v(Z_t, t) dt, \qquad Z_0 = X_0$

transports $Z_0$ from $\pi_0$ to a terminal distribution at $t=1$ matching $\pi_1$.

• Solving the nonlinear least-squares problem:

$$\min_v \int_0^1 \mathbb{E}[\|X_1 - X_0 - v(X_t, t)\|^2] dt.$$

The optimal $v(x, t)$ is the conditional expectation $\mathbb{E}[X_1 - X_0 \mid X_t = x]$.

The method ensures two central theoretical properties:

• Marginal preservation: At all intermediate times, the distribution of $Z_t$ matches that of $X_t$; both processes satisfy the same continuity (transport) equation.
• Convex transport cost reduction: For any convex cost $c(\cdot)$, the expected transport cost $\mathbb{E}[c(Z_1 - Z_0)]$ for the rectified coupling never exceeds that of the original coupling.

This construction is deeply rooted in the theory of optimal transport, where in 1D the monotone (straight) coupling is optimal for all convex costs, and more generally the model systematically regularizes the flow to approach geometric straightness as characterized in optimal transport literature (e.g., existence and uniqueness under convex costs, with straight paths corresponding to low-cost couplings).

2. Recursive Rectification and the Reflow Process

A distinctive algorithmic innovation is the reflow operation, whereby rectified flow is applied recursively to further smooth and straighten transport trajectories:

• After the initial rectification, the resulting coupling $(Z_0, Z_1)$ can itself be "re-rectified" via the same mechanism.
• Each successive rectification stage further straightens the paths, so that after $K$ iterations, the trajectory straightness error (measured in an appropriate metric) decays as $O(1/K)$.
• As straightness increases, the discretization error from time-stepping integration diminishes, and a single-step Euler simulation can suffice for high-fidelity sampling.

This recursive smoothing is supported by convexity arguments and is parallel in spirit to optimal transport, where the straightest possible (monotone) coupling is the cost-minimizing solution.

3. Relation to Optimal Transport Theory

Rectified flow’s design reflects several key findings from the mathematical theory of optimal transport:

• For many convex costs, the optimal transport map is unique and can be characterized as the gradient of a convex potential (in high dimensions).
• The rectification learned by the model not only enforces marginal constraints but also reduces the expected transport cost across the entire family of convex cost functions.
• In 1D, strictly convex transport costs are uniquely minimized by monotone (straight) couplings; the rectified flow model leverages this fact by progressively rectifying arbitrary couplings toward optimal straightness.

This theoretical backdrop endows rectified flow models with guarantees not only of cost reduction and straightness but also regularity and stability of the learned transport.

4. Practical Implementation and Numerical Efficiency

Implementation involves straightforward regression, scalable to large neural architectures:

• No additional parameters are introduced beyond those used in the velocity field neural network (compatible with standard supervised learning pipelines).
• Training involves randomly sampling pairs $(X_0, X_1)$ and times $t \in [0,1]$, then minimizing the empirical squared error between the predictor $v(x, t)$ and the ground-truth direction $X_1 - X_0$ at the interpolated location $X_t$.
• The learned ODE is solved at inference time using numerical methods (Euler, Runge–Kutta), with high straightness allowing accurate results with very coarse (even single-step) discretization.

Empirically, rectified flow models demonstrate:

• Superior sampling efficiency: high-quality samples can be produced in very few integration steps compared to diffusion models or general neural ODEs.
• Robustness and speed: the ODE can be solved exactly if trajectories are exactly straight, dramatically shrinking computation at inference.

5. Applications and Empirical Impact

Experiments validate rectified flow across:

• Image generation, where rectified flows yield high fidelity and visually coherent results even with limited integration steps.
• Image-to-image translation and domain adaptation, demonstrating capacity to learn complex transport maps between diverse datasets.
• Domain transfer tasks, where aligning distributions via nearly straight flows supports more robust generalization.

The method serves as a practical unified solution for distribution transport, enjoying both sound theoretical guarantees and strong empirical performance.

6. Extensions and Generalizations

The rectified flow approach can be adapted:

• For single-object optimal transport (i.e., targeting a specific convex cost via projection of the velocity field onto the gradient of a convex conjugate), as detailed in contemporaneous work (Liu, 2022).
• As the foundation for multi-stage, recursively straightening flows applicable to high-dimensional, multimodal distributions observed in practice.
• By leveraging more expressive neural ODE architectures, enabling scaling without revisiting the core marginal-preserving and straightening mechanisms.

Future work is suggested in integrating rectified flow models with broader classes of generative modeling frameworks, incorporating more intricate cost function landscapes, and exploring extensions to settings such as semi-supervised or structured data domains.

7. Summary Table: Core Operations

Step	Operation Description	Mathematical Expression
Linear interp	$X_t = t X_1 + (1-t) X_0$
Velocity field	$\min_v \mathbb{E}[\|X_1 - X_0 - v(X_t, t)\|^2]$
ODE evolve	$dZ_t = v(Z_t, t) dt$
Reflow iter	Replace $(X_0, X_1)$ with rectified $(Z_0, Z_1)$

This tabulation summarizes the primary flow of the rectified flow algorithm: linear interpolation, velocity regression, ODE transport, and iterative reflow for straightness.

---

Rectified flow models provide a rigorous, practical, and numerically efficient bridge between optimal transport theory and modern generative modeling. Recursive rectification produces deterministic couplings that minimize convex transport cost and nearly align with the theoretical optimal transport paths, as validated both mathematically and through empirical experiments on generative tasks (Liu et al., 2022).