Rectified Flow Diffusion Transformer

• Rectified Flow-based Diffusion Transformers are generative models that employ time-conditioned deterministic transport using transformer architectures to predict velocity fields.
• They integrate flow matching and hierarchical transformer designs, including progressive and latent configurations, to optimize sampling efficiency and accuracy.
• Empirical results show reduced inference time and high-quality outputs across diverse domains such as image generation, 3D mesh production, video forecasting, and audio editing.

Rectified Flow-based Diffusion Transformer denotes a family of generative models that combine the rectified-flow formulation of deterministic probability transport with transformer backbones or diffusion-transformer design principles. In this family, the central learned object is a time-dependent velocity field $v_\theta$ in an ODE, rather than a score or a denoising residual, and training is typically based on regression to the displacement between paired source and target samples along an interpolation path (Liu et al., 2022). In the literature, the term spans several related cases: native rectified-flow transformers trained directly by flow matching; hierarchical or progressive DiT-style systems that preserve the rectified-flow objective while changing the spatial or temporal organization of computation; and rectified-diffusion retrofits that keep a pretrained diffusion model’s scheduler and prediction type but retrain it on matched noise–sample pairs to obtain more favorable few-step ODE trajectories (Wang et al., 2024). Several papers also stress that some such systems are “rectified-flow transformers” rather than diffusion models in the strict DDPM or score-based sense (Djouama et al., 21 Apr 2026).

1. Foundational formulation

Rectified flow was introduced as a simple way to learn an ODE that transports between two empirically observed distributions $\pi_0$ and $\pi_1$ by following straight paths as much as possible (Liu et al., 2022). The basic generative dynamics are

$dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$

with $Z_0\sim \pi_0$ and terminal distribution $Z_1\sim \pi_1$. Starting from paired random variables $(X_0,X_1)$ with $X_0\sim \pi_0$ and $X_1\sim \pi_1$, rectified flow defines the linear interpolation

$X_t = tX_1 + (1-t)X_0.$

The canonical objective is

$\pi_0$0

or, in practical parameterized form,

$\pi_0$1

The population minimizer is

$\pi_0$2

This formulation is architecture-agnostic: wherever a U-Net or transformer would ordinarily predict noise, score, or velocity, the same backbone can instead parameterize a rectified-flow vector field (Liu et al., 2022). That point is central to the transformer interpretation. A diffusion transformer that already accepts an intermediate sample, a time embedding, and optional conditioning can be reinterpreted as a rectified-flow model by changing the target from denoising or score estimation to direct transport-velocity prediction.

The original rectified-flow work also formalized why straightness matters. If trajectories are exactly straight,

$\pi_0$3

then a single Euler step from $\pi_0$4 to $\pi_0$5 is exact: $\pi_0$6 To measure deviation from this regime, it defines

$\pi_0$7

Small $\pi_0$8 explains why coarse discretization can work well (Liu et al., 2022).

The same paper also established two structural properties of rectification. First, the induced ODE preserves the interpolation marginals: $\pi_0$9 Second, convex transport costs do not increase: $\pi_1$0 for any convex $\pi_1$1. Recursive rectification, or reflow,

$\pi_1$2

was proposed as a way to obtain increasingly straight flows (Liu et al., 2022).

2. Transformer parameterizations and the diffusion–flow interface

In transformer-based systems, rectified flow typically appears as time-conditioned velocity prediction on an interpolation path. A generic transformer receives $\pi_1$3, $\pi_1$4, and conditioning, and outputs a tensor of the same shape as the state, interpreted as $\pi_1$5. In the original rectified-flow formulation, this means training on tuples

$\pi_1$6

and sampling by integrating

$\pi_1$7

with Euler or higher-order ODE solvers (Liu et al., 2022).

Several later papers instantiate this directly with transformers. NAMI uses the rectified-flow ODE

$\pi_1$8

together with the interpolation

$\pi_1$9

and the objective

$dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$0

inside a progressive MM-DiT architecture (Ma et al., 12 Mar 2025). MeshFlow uses a latent transformer trained by Conditional Flow Matching on a linear interpolation

$dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$1

with target velocity $dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$2, and conditions the transformer through $dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$3 and the desired vertex count (Li et al., 3 Jun 2026). The audio-editing transformer in “Hybrid Diffusion Transformer for Instruction-Guided Audio Editing via Rectified Flow” uses

$dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$4

and predicts

$dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$5

under a rectified-flow matching loss (Gao et al., 18 Jun 2026).

At the same time, the literature does not restrict “rectified” transformer models to native velocity regression. “Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow” argues that the decisive ingredient is retraining on matched noise–sample pairs from a pretrained diffusion model, not the specific use of flow-matching form or $dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$6-prediction (Wang et al., 2024). In its formulation, a general diffusion path is

$dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$7

and rectification is achieved by replacing random $dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$8 pairs with deterministic matched pairs $dZ_t = v(Z_t,t)\,dt,\qquad t\in[0,1],$9, while preserving the original scheduler and prediction type. This broadens the meaning of a rectified-flow-based diffusion transformer: it can be a native flow model, or a pretrained diffusion transformer retrained toward first-order ODE behavior without changing its denoiser parameterization (Wang et al., 2024).

A recurring implication is that “rectified flow-based diffusion transformer” is a methodological label more than a single probabilistic definition. Some works are direct RF transformers; others are transformer architectures derived from diffusion backbones but trained as deterministic transport ODEs; others are rectified-diffusion adaptations of conventional diffusion models (Djouama et al., 21 Apr 2026).

3. Architectural patterns

The transformer architectures associated with rectified flow are diverse, but the literature converges on a small set of recurring patterns.

RF-HiT uses an hourglass transformer backbone with a multi-scale Hierarchical Feature Encoder for conditional medical segmentation (Djouama et al., 21 Apr 2026). Its main Hourglass Flow Model is adapted from HDiT and has an encoder–bottleneck–decoder structure with token merging in the encoder, a low-resolution bottleneck, and symmetric upsampling in the decoder. The first two hierarchy levels use neighborhood attention, and only the bottleneck uses global self-attention. Anatomical conditioning is injected at each encoder level through learnable interpolation,

$Z_0\sim \pi_0$0

and the model predicts a velocity field with the same spatial dimensions as the segmentation state (Djouama et al., 21 Apr 2026).

NAMI organizes rectified flow progressively across both time and resolution (Ma et al., 12 Mar 2025). The full interval is split into $Z_0\sim \pi_0$1 stages,

$Z_0\sim \pi_0$2

and each stage is paired with a different image resolution. The model partitions MM-DiT blocks into nested stage modules,

$Z_0\sim \pi_0$3

so that early low-resolution stages use fewer layers and later high-resolution stages use more. Its main 1024-resolution system uses three stages at resolutions 256, 512, and 1024, with a 1:1:1 time-window ratio and a 9:7:6 layer ratio across the three stages for NAMI-2B (Ma et al., 12 Mar 2025).

MeshFlow is a latent DiT-style rectified-flow model for 3D mesh generation (Li et al., 3 Jun 2026). A MeshVAE first compresses a mesh into a compact latent token sequence. The flow transformer then operates on these latent tokens rather than raw face or vertex tokens. The reported generator has 18 transformer blocks, hidden size 1024, and 427M parameters, and it conditions on a voxelized point cloud through 3D rotary positional encoding. Geometry and topology are generated jointly because decoded latent tokens produce $Z_0\sim \pi_0$4, from which adjacency and faces are reconstructed (Li et al., 3 Jun 2026).

FREUD extends the pattern to spatio-temporal forecasting with a frame-wise encoder and a united decoder, both transformer-based (Schusterbauer et al., 29 May 2026). Its first stage uses a frame-wise transformer encoder for causal, streaming-compatible compression, and a transformer video decoder trained by rectified flow. The second stage is a latent-space rectified-flow transformer with 3D axial RoPE and factorized spatio-temporal attention. The latent-space models are reported in three sizes: S-LSM with 44M parameters, B-LSM with 141M, and L-LSM with 473M (Schusterbauer et al., 29 May 2026).

Instruction-guided audio editing introduces a hybrid diffusion transformer that explicitly mixes block types (Gao et al., 18 Jun 2026). Its first stage downsamples audio tokens and applies two DSJA-MMDiT blocks for coarse joint audio-text alignment. The second stage restores full resolution and alternates DSJA-MMDiT and AZCA-DiT blocks, so high-resolution refinement combines occasional joint attention with cheaper cross-attention and AdaLN-Zero modulation. This architecture is presented as a coarse-to-fine strategy for balancing editing quality and efficiency (Gao et al., 18 Jun 2026).

A concise view of this design space is useful:

System	Domain	Main architectural trait
RF-HiT (Djouama et al., 21 Apr 2026)	Medical segmentation	Hourglass transformer with hierarchical conditioning
NAMI (Ma et al., 12 Mar 2025)	Image generation	Progressive multi-resolution MM-DiT with piecewise flow
MeshFlow (Li et al., 3 Jun 2026)	3D mesh generation	Latent DiT over compact mesh tokens
FREUD (Schusterbauer et al., 29 May 2026)	Weather nowcasting	Frame-wise encoder plus latent-space RF transformer
Hybrid audio editing (Gao et al., 18 Jun 2026)	Instruction-guided audio editing	Two-stage hybrid DSJA-MMDiT/AZCA-DiT

These systems suggest that transformerization of rectified flow is not tied to one canonical DiT. A plausible implication is that the decisive design variables are token geometry, conditioning placement, and where expensive global or multimodal attention is spent, rather than the mere substitution of self-attention for convolution.

4. Straightness, first-order behavior, and few-step sampling

Sampling efficiency is the central technical motivation for rectified-flow-based transformers. The original rectified-flow paper tied few-step generation to the straightness metric $Z_0\sim \pi_0$5 and showed empirically that nearly straight flows could yield high-quality results even with a single Euler discretization step (Liu et al., 2022). On CIFAR-10 with a DDPM++ U-Net, full ODE simulation of the 1-rectified flow achieved FID 2.58 and recall 0.57, while one-step distilled variants of 2- and 3-rectified flow achieved FID 4.85/5.21 and recall 0.50/0.51, respectively (Liu et al., 2022). Although these experiments used a U-Net, the argument is backbone-independent: if a transformer backbone learns similarly straight trajectories, coarse ODE discretization should inherit the same benefit.

Theoretical support for this claim was sharpened in “On the Wasserstein Convergence and Straightness of Rectified Flow” (Bansal et al., 2024). That note studies the flow

$Z_0\sim \pi_0$6

with

$Z_0\sim \pi_0$7

and proves that under global Lipschitzness, or under weaker local $Z_0\sim \pi_0$8 and non-explosion assumptions, the rectified coupling is straight; in the paper’s terminology, 2-RF is straight. Its practical interpretation is explicit: if trajectories are straight or nearly straight, Euler sampling can be accurate at very low NFEs (Bansal et al., 2024).

The literature then develops several engineering strategies around this geometric idea. NAMI decomposes rectified flow into stage-wise windows and aligns them with a spatial pyramid, reporting a 40% reduction in inference time for 1024-resolution generation, with piecewise flow alone accounting for 28.68% and model partitioning adding a further 11.81% in the reported ablation (Ma et al., 12 Mar 2025). PeRFlow applies piecewise rectification to pretrained diffusion pipelines rather than native RF transformers, dividing the time horizon into $Z_0\sim \pi_0$9 windows and training one local linearization per interval; it is presented as a universal plug-and-play accelerator for pretrained diffusion models (Yan et al., 2024).

Rectified diffusion complicates the standard “straighter is always better” narrative (Wang et al., 2024). That paper argues that the real target is first-order approximate ODE behavior, not Euclidean straightness per se. For a general diffusion parameterization

$Z_1\sim \pi_1$0

the ideal first-order path may be curved in $Z_1\sim \pi_1$1-space, especially for DDPM and Sub-VP schedules. What matters is constancy of $Z_1\sim \pi_1$2 along a trajectory. This is a genuine controversy in the field: some works interpret few-step success through literal path straightening, whereas rectified diffusion interprets it through first-order trajectory consistency (Wang et al., 2024).

Transformer-specific work on inversion and editing further extends the low-NFE story. “Runge-Kutta Approximation and Decoupled Attention for Rectified Flow Inversion and Semantic Editing” treats RF inversion as a numerical ODE problem in FLUX-style MM-DiTs and reports that fourth-order Runge–Kutta inversion substantially improves reconstruction over first-order RF baselines, with the best reconstruction result reported as PSNR 25.68, SSIM 0.8364, and LPIPS 0.1241 (Chen et al., 16 Sep 2025). This suggests that even when few-step forward generation is the goal, higher-order solvers can remain important for inversion or editing workloads.

5. Empirical landscape across domains

The empirical record for rectified-flow-based transformers is broad rather than confined to image synthesis.

In medical image segmentation, RF-HiT targets the dual bottlenecks of quadratic transformer attention and diffusion-style iterative denoising (Djouama et al., 21 Apr 2026). It uses 13.6M parameters, 10.14 GFLOPs, and inference in as few as three steps. The reported performance is 91.27% mean Dice on ACDC and 87.40% on BraTS 2021. The paper presents this as a strong efficiency–performance trade-off for real-time clinical segmentation (Djouama et al., 21 Apr 2026).

In image generation, NAMI frames progressive rectified-flow transformers as an efficient alternative to uniform-resolution DiT inference (Ma et al., 12 Mar 2025). Its NAMI-2B model uses width 2048, depth 22, and 16 attention heads, and is trained on about 100 million samples. On GenEval it reports an overall score of 0.65, and on DPG-Benchmark an overall score of 83.8. More central to the rectified-flow transformer agenda is the efficiency result: approximately 40% lower inference time at 1024 resolution relative to a FLUX-based 2B baseline, while remaining competitive with strong open baselines (Ma et al., 12 Mar 2025).

In 3D mesh generation, MeshFlow uses a MeshVAE plus a latent rectified-flow transformer to generate all mesh vertices and edges in parallel (Li et al., 3 Jun 2026). The paper reports that it is approximately $Z_1\sim \pi_1$3 faster than the fastest autoregressive generator. On Toys4K point-cloud-conditioned mesh generation, the reported Chamfer and Hausdorff distances are $Z_1\sim \pi_1$4 and $Z_1\sim \pi_1$5, with inference time $Z_1\sim \pi_1$6 seconds for sampling plus mesh extraction (Li et al., 3 Jun 2026). The significance is that rectified flow becomes practical for meshes only after topology is made continuous and compact in latent space.

In precipitation nowcasting, FREUD combines a probabilistic first-stage rectified-flow video decoder with a latent-space rectified-flow transformer (Schusterbauer et al., 29 May 2026). On SEVIR, the reported result for FREUD + LSM-L is CRPS 0.0190, SSIM 0.7841, HSS 0.5011, and CSI 0.3864. The same work reports a reliability index $Z_1\sim \pi_1$7, versus $Z_1\sim \pi_1$8 for CasCast, emphasizing calibration and uncertainty preservation rather than only localization metrics (Schusterbauer et al., 29 May 2026).

In instruction-guided audio editing, the hybrid rectified-flow transformer achieves an Average Editing Time of 5.07 s, compared with 11.00 s for AUDIT, 11.23 s for RFM-Editing, and 101.87 s for AudioEditor on AudioCapsSubset (Gao et al., 18 Jun 2026). Its reported parameter count is 78.61M. Across AudioCapsSubset and AudioSetCapsSubset it is reported to be consistently strongest or near-strongest on LSD, FD, FAD, and KL, which the paper interprets as better acoustic fidelity and preservation of non-edited content (Gao et al., 18 Jun 2026).

Taken together, these results indicate that rectified-flow-based diffusion transformers are not confined to one modality or one statistical role. They have been used for unconditional generation, conditional generation, segmentation, structured 3D synthesis, forecasting, inversion, and editing. A plausible implication is that the abstraction “time-conditioned deterministic transport over tokens or latents” is sufficiently general to support both generative and predictive tasks, provided that conditioning interfaces are designed for the domain.

6. Scope, misconceptions, and open questions

A first misconception is terminological. Not every model described as a rectified-flow-based diffusion transformer is a diffusion model in the strict DDPM or score-based sense. RF-HiT explicitly states that it should be read more as a rectified-flow transformer than as a diffusion transformer in the strict probabilistic sense (Djouama et al., 21 Apr 2026). MeshFlow likewise uses a “Rectified Flow transformer” as the generative stage but does not implement a conventional diffusion noising-and-denoising chain (Li et al., 3 Jun 2026). The audio-editing transformer is trained by rectified flow matching and predicts a velocity field, not a denoising residual (Gao et al., 18 Jun 2026). The label therefore often refers to lineage from diffusion-transformer architecture, not strict adherence to diffusion-model training.

A second misconception is that transformerization alone solves low-step generation. The theoretical papers repeatedly tie performance at low NFEs to geometry of the learned probability path rather than to backbone choice (Liu et al., 2022). “On the Wasserstein Convergence and Straightness of Rectified Flow” makes that point explicit: the practical benefit arises because nearly straight trajectories are easier to discretize accurately, not because transformers intrinsically reduce ODE error (Bansal et al., 2024).

A third misconception is that literal straightness is the only meaningful target. Rectified diffusion argues instead for first-order approximate ODE paths and shows that, for DDPM and Sub-VP families, the natural first-order path may be curved (Wang et al., 2024). This does not invalidate rectified-flow training, but it does broaden the design space. A transformer may be “rectified” either by direct velocity regression on linear bridges or by matched-pair retraining within its native diffusion parameterization.

Several open directions are already visible in the literature. Source-distribution design is one such direction. MixFlow shows that mixed source distributions can reduce curvature and improve low-step quality in rectified flows by replacing the fixed standard Gaussian with a mixture between $Z_1\sim \pi_1$9 and a $(X_0,X_1)$0-conditioned Gaussian, reporting an average 12% improvement in FID over standard rectified flow and 7% over previous baselines under fixed sampling budget (Nayal et al., 10 Apr 2026). Although demonstrated with U-Nets, the mechanism is architecture-agnostic and could plausibly transfer to DiTs.

Another direction concerns editing and inversion interfaces for RF-based MM-DiTs. The Runge–Kutta/DDTA work shows that ODE solver choice and attention disentanglement remain bottlenecks for real-image workflows even when forward sampling is efficient (Chen et al., 16 Sep 2025). This suggests that rectified-flow-based diffusion transformers need not only faster generation, but also transformer-native inversion and control mechanisms.

Finally, the literature repeatedly notes what a transformer-specific paper still must contribute beyond the rectified-flow objective: tokenization or latent patching, positional encoding, conditioning interfaces, scaling strategy, and, in multimodal settings, a principled allocation of joint versus cross attention (Liu et al., 2022). The field’s present state therefore supports a precise but limited conclusion: rectified flow provides a coherent transport objective, a sampling rationale, and several acceleration strategies for transformer generative models; the transformer component determines how that objective is parameterized and scaled, but not the underlying geometry that makes few-step ODE generation possible.