Rectified Discrete Flow (ReDi)

Updated 4 July 2026

ReDi is a method for discrete flow models that reframes efficient discrete generation as a coupling-design problem by reducing Conditional Total Correlation.
It iteratively rectifies the source–target coupling, thereby mitigating the factorization error that hampers few-step or one-step generation.
Empirical results demonstrate significant improvements in image and text generation metrics, outperforming baselines like MaskGIT and achieving notable speedups.

Rectified Discrete Flow (ReDi) is a method for Discrete Flow-based Models (DFMs) that addresses the slow sampling speeds induced by iterative decoding on high-dimensional discrete data. The central claim is that the few-step failure mode of factorized DFMs can be rigorously characterized by Conditional Total Correlation (TC), and that an iterative rectification of the source–target coupling reduces this error. ReDi therefore reframes efficient discrete generation as a coupling-design problem: by rectifying the coupling between the source distribution and the target distribution, it monotonically decreases Conditional TC, improves few-step generation, and produces couplings that are suitable for training efficient one-step models (Yoo et al., 21 Jul 2025).

1. Formal setting in discrete flow-based modeling

DFMs define a probabilistic path that transforms a tractable source distribution $p(X_0)$ , such as uniform or masked tokens, into a target distribution $q(X_1)$ , such as images or text. The construction introduces a coupling $\pi(X_0,X_1)$ with marginals $p(X_0)$ and $q(X_1)$ , often $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ , together with a time-indexed conditional $p_t(x_t\mid x_0,x_1)$ interpolating between $x_0$ and $x_1$ , for example

$p_t=(1-\alpha_t)\delta_{x_0}+\alpha_t\delta_{x_1}.$

The marginal at time $q(X_1)$ 0 is

$q(X_1)$ 1

and the corresponding denoising transition is

$q(X_1)$ 2

A neural model is then trained to approximate $q(X_1)$ 3, and sampling from $q(X_1)$ 4 to $q(X_1)$ 5 uses a sequence of such transitions (Yoo et al., 21 Jul 2025).

For high-dimensional discrete data, exact modeling of $q(X_1)$ 6 requires outputs over $q(X_1)$ 7 possibilities, where $q(X_1)$ 8 is the number of token types and $q(X_1)$ 9 is the number of dimensions. DFMs therefore assume the factorization

$\pi(X_0,X_1)$ 0

which reduces complexity from $\pi(X_0,X_1)$ 1 to $\pi(X_0,X_1)$ 2.

This approximation is the source of the few-step challenge. The independence assumption ignores correlations among dimensions. When generation is compressed into few, or a single, transitions, the model must inject high inter-dimensional correlation all at once; the factorized form then fails, leading to poor sample quality or collapse. In ReDi, this failure mode is not treated as an incidental optimization artifact but as a structural consequence of the coupling used to define the DFM transitions.

2. Conditional Total Correlation as the factorization-error functional

ReDi formalizes factorization error through Conditional Total Correlation. For random vectors $\pi(X_0,X_1)$ 3 given $\pi(X_0,X_1)$ 4, the conditional total correlation is defined as

$\pi(X_0,X_1)$ 5

In DFMs, this is specialized to measure how far the true transition $\pi(X_0,X_1)$ 6 is from its factorized counterpart:

$\pi(X_0,X_1)$ 7

Because the ideal transition $\pi(X_0,X_1)$ 8 depends on the coupling $\pi(X_0,X_1)$ 9 through the denoising relation, the conditional TC also depends on $p(X_0)$ 0. A high TC means that large inter-dimensional dependencies are lost by the factorized model, which is precisely the approximation error that damages few-step sampling.

A key result is that the average KL between the true conditional $p(X_0)$ 1 and its factorization equals TC. ReDi therefore uses conditional TC as both a diagnostic and an optimization target at the coupling level. The paper’s monotonicity theorem states that if $p(X_0)$ 2 is the current coupling and $p(X_0)$ 3 is the rectified coupling induced by sampling $p(X_0)$ 4 from the trained DFM, then

$p(X_0)$ 5

The proof sketch proceeds by expressing TC as a KL divergence, viewing the ReDi update as a Markov kernel that keeps $p(X_0)$ 6 fixed and redraws $p(X_0)$ 7, and applying the Data Processing Inequality for KL divergence (Yoo et al., 21 Jul 2025).

3. Iterative rectification and convergence

ReDi begins with an initial coupling $p(X_0)$ 8, often $p(X_0)$ 9. For iterations $q(X_1)$ 0, it performs two steps. First, it trains a DFM $q(X_1)$ 1 on paired samples from $q(X_1)$ 2. Second, it rectifies the coupling by setting

$q(X_1)$ 3

In the one-step rectification view, the procedure is: train a DFM on a dataset of pairs sampled from $q(X_1)$ 4; draw $q(X_1)$ 5; sample $q(X_1)$ 6; collect the new dataset $q(X_1)$ 7; and define $q(X_1)$ 8 via $q(X_1)$ 9. In the multi-step view, this dataset-generation and retraining cycle is repeated for $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 0 rounds.

The convergence claim is correspondingly structural rather than heuristic. By the monotonicity theorem, the conditional TC is non-increasing with each iteration. Under mild technical conditions, such as sufficient model capacity and dense sampling of $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 1, the coupling converges to one whose TC is minimal, namely one best matched to a factorized decoder. This places ReDi in a distinct position among acceleration methods for discrete generation: it does not primarily change the neural architecture or the decoding rule, but instead changes the coupling from which the DFM learns (Yoo et al., 21 Jul 2025).

A common misconception is that one-step or few-step failure in discrete flows is solely a matter of insufficient distillation or inadequate training. ReDi instead identifies the factorization approximation itself as the bottleneck and treats coupling rectification as the mechanism for reducing that bottleneck.

4. Empirical performance in few-step and one-step generation

The experimental setup in ReDi covers both image and text generation. For images, the benchmark is ImageNet class-conditional generation, with images tokenized into $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 2 VQGAN codes and $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 3. For text, the benchmark is OpenWebText using the GPT-2 tokenizer with vocabulary approximately $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 4 and length $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 5. The image metrics are FID, Inception Score, Precision, Recall, Density, and Coverage; the text metric is generative perplexity using GPT-2-large. The baselines are MaskGIT, SDTT, Di4C, and DUO+DCD (Yoo et al., 21 Jul 2025).

The reported results are organized around the few-step and one-step regimes.

Setting	Configuration	Reported result
ImageNet few-step	4 steps	ReDi $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 6 improve MaskGIT’s 4-step FID $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 7
ImageNet one-step	After $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 8 rectifications	“ReDi $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$ 9-distill” achieves FID $p_t(x_t\mid x_0,x_1)$ 0 and IS $p_t(x_t\mid x_0,x_1)$ 1
Text few-step	DUO+DCD as $p_t(x_t\mid x_0,x_1)$ 2	ReDi $p_t(x_t\mid x_0,x_1)$ 3 match 1024-step perplexity at $p_t(x_t\mid x_0,x_1)$ 4 steps

On ImageNet few-step generation, ReDi $p_t(x_t\mid x_0,x_1)$ 5 improve MaskGIT’s 4-step FID from $p_t(x_t\mid x_0,x_1)$ 6 to $p_t(x_t\mid x_0,x_1)$ 7, rivaling or exceeding SDTT and Di4C. On ImageNet one-step generation, after $p_t(x_t\mid x_0,x_1)$ 8 rectifications, a one-step model denoted “ReDi $p_t(x_t\mid x_0,x_1)$ 9-distill” achieves FID $x_0$ 0 with IS $x_0$ 1, far better than SDTT and Di4C one-step results with FID approximately $x_0$ 2. On text generation, using DUO+DCD as $x_0$ 3, ReDi $x_0$ 4 match 1024-step perplexity at $x_0$ 5 steps, corresponding to a $x_0$ 6– $x_0$ 7 speedup. Combining Di4C with ReDi also yields further gains (Yoo et al., 21 Jul 2025).

These experiments also motivate the one-step training strategy from rectified couplings. Once a “good” coupling $x_0$ 8 with low TC has been obtained, a one-step DFM $x_0$ 9 can be trained directly on pairs sampled from $x_1$ 0. No architectural change is needed; the training objective is simply set to match $x_1$ 1 transitions at $x_1$ 2. The intended effect is that the one-step model inherits the low factorization error of $x_1$ 3 and can therefore perform high-quality single-pass sampling.

The term “ReDi” is not unique to the coupling-rectification method of discrete flow-based models. In the rectified-flow literature, “ReDi” is also used for “Discretized-RF,” a family of rectified flow models that discretizes the straight path into $x_1$ 4 sub-paths, introduces a stochastic velocity or momentum field, and injects noise into the velocity on each sub-path in order to trade off between Rectified Flow’s efficiency and diffusion models’ diversity (Ma et al., 10 Jun 2025).

A separate but related use appears in motion generation. DisCoRD, “Discrete Tokens to Continuous Motion via Rectified Flow Decoding,” leverages rectified flow to decode discrete motion tokens in the continuous, raw motion space. Its decoder is a learned ODE vector field conditioned on per-frame features extracted from discrete tokens, and it is presented as plug-and-play for discrete motion generators such as T2M-GPT, MMM, MoMask, TalkSHOW, ProbTalk, and TM2D (Cho et al., 2024).

The discrete ReDi framework of coupling rectification also serves as a basis for subsequent extensions. AReUReDi, “Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidance,” explicitly builds on Rectified Discrete Flows and extends ReDi to multi-objective sampling in discrete state spaces. It combines Tchebycheff scalarization, locally balanced proposals, and annealed Metropolis-Hastings updates to bias sampling toward Pareto-optimal states while preserving distributional invariance (Chen et al., 30 Sep 2025).

This pattern suggests a terminological overlap rather than a single unified method: “rectified” methods in discrete settings include coupling rectification for DFMs, discretized rectified-flow trajectories in continuous spaces, and rectified-flow decoders for discrete-to-continuous generation.

6. Practical implications, limitations, and open directions

ReDi is presented as simple to implement on top of any existing DFM and as not requiring specialized distillation losses or teacher/student co-training. Its practical significance is therefore tied to a specific theoretical perspective: efficient discrete generation is improved by modifying the coupling so that the factorized decoder is asked to model transitions with lower conditional TC. This perspective provides a new route to few-step and one-step synthesis, rather than treating speedup as a purely architectural or solver-design problem (Yoo et al., 21 Jul 2025).

The stated limitations are also coupling-centric. Sampling-error accumulation when generating pairs may degrade eventual model capacity. The current focus is on non-autoregressive DFMs, while autoregressive models are identified as an intriguing extension.

The future directions are correspondingly explicit. The paper points to deeper theoretical links between discrete and continuous rectified flows, automatic design of optimal couplings $x_1$ 5 that minimize TC subject to target marginals, and extension of ReDi to accelerate autoregressive sampling by rectifying implicit transition couplings. A plausible implication is that the broader research program around ReDi is not limited to faster decoding; it also concerns how couplings should be constructed so that factorized generative transitions are intrinsically easier to learn and execute.