---
title: Rectified Discrete Flow (ReDi)
url: https://www.emergentmind.com/topics/rectified-discrete-flow-redi
type: topic
---

# Rectified Discrete Flow (ReDi)

Rectified Discrete Flow (ReDi) is a method for Discrete Flow-based Models (DFMs) that addresses the slow sampling speeds induced by iterative decoding on high-dimensional discrete data. The central claim is that the few-step failure mode of factorized DFMs can be rigorously characterized by Conditional Total Correlation (TC), and that an iterative rectification of the source–target coupling reduces this error. ReDi therefore reframes efficient discrete generation as a coupling-design problem: by rectifying the coupling between the source distribution and the target distribution, it monotonically decreases Conditional TC, improves few-step generation, and produces couplings that are suitable for training efficient one-step models [2507.15897].

## 1. Formal setting in discrete flow-based modeling

DFMs define a probabilistic path that transforms a tractable source distribution $p(X_0)$, such as uniform or masked tokens, into a target distribution $q(X_1)$, such as images or text. The construction introduces a coupling $\pi(X_0,X_1)$ with marginals $p(X_0)$ and $q(X_1)$, often $\pi(X_0,X_1)=p(X_0)\cdot q(X_1)$, together with a time-indexed conditional $p_t(x_t\mid x_0,x_1)$ interpolating between $x_0$ and $x_1$, for example
$$
p_t=(1-\alpha_t)\delta_{x_0}+\alpha_t\delta_{x_1}.
$$
The marginal at time $t$ is
$$
p_t(x_t)=\sum_{x_0,x_1}\pi(x_0,x_1)p_t(x_t\mid x_0,x_1),
$$
and the corresponding denoising transition is
$$
p_{s\mid t}(x_s\mid x_t)=\sum_{x_0,x_1}p_s(x_s\mid x_0,x_1,x_t)\cdot p_t(x_0,x_1\mid x_t).
$$
A neural model is then trained to approximate $p_{s\mid t}(\cdot\mid x_t)$, and sampling from $X_0$ to $X_1$ uses a sequence of such transitions [2507.15897].

For high-dimensional discrete data, exact modeling of $p_{s\mid t}(x_s\mid x_t)$ requires outputs over $D^n$ possibilities, where $D$ is the number of token types and $n$ is the number of dimensions. DFMs therefore assume the factorization
$$
p_{s\mid t}(x_s\mid x_t)\approx\prod_{i=1}^n p_{s\mid t}(x_s^i\mid x_t),
$$
which reduces complexity from $D^n$ to $n\cdot D$.

This approximation is the source of the few-step challenge. The independence assumption ignores correlations among dimensions. When generation is compressed into few, or a single, transitions, the model must inject high inter-dimensional correlation all at once; the factorized form then fails, leading to poor sample quality or collapse. In ReDi, this failure mode is not treated as an incidental optimization artifact but as a structural consequence of the coupling used to define the DFM transitions.

## 2. Conditional Total Correlation as the factorization-error functional

ReDi formalizes factorization error through Conditional Total Correlation. For random vectors $X,Y$ given $Z$, the conditional total correlation is defined as
$$
\mathrm{TC}(X;Y\mid Z) = \sum_{x,y,z}p(x,y,z)\, \log\frac{p(x,y\mid z)}{p(x\mid z)\,p(y\mid z)}.
$$
In DFMs, this is specialized to measure how far the true transition $p_{s\mid t}(X_s\mid X_t)$ is from its factorized counterpart:
$$
\mathrm{TC}_\pi(X_s\mid X_t)
=
\mathbb{E}_{x_t\sim p_t}
\Bigl[
D_{KL}\Bigl(
p_{s\mid t}(X_s\mid x_t)\;\Vert\;\prod_{i=1}^n p_{s\mid t}(X_s^i\mid x_t)
\Bigr)
\Bigr].
$$

Because the ideal transition $p_{s\mid t}(x_s\mid x_t)$ depends on the coupling $\pi$ through the denoising relation, the conditional TC also depends on $\pi$. A high TC means that large inter-dimensional dependencies are lost by the factorized model, which is precisely the approximation error that damages few-step sampling.

A key result is that the average KL between the true conditional $p_{s\mid t}(X_s\mid X_t)$ and its factorization equals TC. ReDi therefore uses conditional TC as both a diagnostic and an optimization target at the coupling level. The paper’s monotonicity theorem states that if $\pi_k$ is the current coupling and $\pi_{k+1}(x_0,x_1)=p(x_0)\,p_\theta(x_1\mid x_0)$ is the rectified coupling induced by sampling $X_1$ from the trained DFM, then
$$
\mathrm{TC}_{\pi_{k+1}}(X_1\mid X_0)\le \mathrm{TC}_{\pi_k}(X_1\mid X_0).
$$
The proof sketch proceeds by expressing TC as a KL divergence, viewing the ReDi update as a Markov kernel that keeps $X_0$ fixed and redraws $X_1\sim p_\theta(\cdot\mid X_0)$, and applying the Data Processing Inequality for KL divergence [2507.15897].

## 3. Iterative rectification and convergence

ReDi begins with an initial coupling $\pi_0$, often $\pi_0(X_0,X_1)=p(X_0)q(X_1)$. For iterations $k=0,\dots,K-1$, it performs two steps. First, it trains a DFM $p_\theta^{(k)}(X_1\mid X_0)$ on paired samples from $\pi_k$. Second, it rectifies the coupling by setting
$$
\pi_{k+1}(X_0,X_1)=p(X_0)\,p_\theta^{(k)}(X_1\mid X_0).
$$

In the one-step rectification view, the procedure is: train a DFM on a dataset of pairs sampled from $\pi_k$; draw $x_0^i\sim p(X_0)$; sample $x_1^i\sim p_\theta(X_1\mid x_0^i)$; collect the new dataset $D_{k+1}=\{(x_0^i,x_1^i)\}$; and define $\pi_{k+1}$ via $D_{k+1}$. In the multi-step view, this dataset-generation and retraining cycle is repeated for $K$ rounds.

The convergence claim is correspondingly structural rather than heuristic. By the monotonicity theorem, the conditional TC is non-increasing with each iteration. Under mild technical conditions, such as sufficient model capacity and dense sampling of $D_k$, the coupling converges to one whose TC is minimal, namely one best matched to a factorized decoder. This places ReDi in a distinct position among acceleration methods for discrete generation: it does not primarily change the neural architecture or the decoding rule, but instead changes the coupling from which the DFM learns [2507.15897].

A common misconception is that one-step or few-step failure in discrete flows is solely a matter of insufficient distillation or inadequate training. ReDi instead identifies the factorization approximation itself as the bottleneck and treats coupling rectification as the mechanism for reducing that bottleneck.

## 4. Empirical performance in few-step and one-step generation

The experimental setup in ReDi covers both image and text generation. For images, the benchmark is ImageNet class-conditional generation, with images tokenized into $16\times 16$ VQGAN codes and $D=1024$. For text, the benchmark is OpenWebText using the GPT-2 tokenizer with vocabulary approximately $50\mathrm{k}$ and length $1024$. The image metrics are FID, Inception Score, Precision, Recall, Density, and Coverage; the text metric is generative perplexity using GPT-2-large. The baselines are MaskGIT, SDTT, Di4C, and DUO+DCD [2507.15897].

The reported results are organized around the few-step and one-step regimes.

| Setting | Configuration | Reported result |
|---|---|---|
| ImageNet few-step | 4 steps | ReDi$^{1/2}$ improve MaskGIT’s 4-step FID $=10.9\rightarrow 7.6/7.9$ |
| ImageNet one-step | After $K=3$ rectifications | “ReDi$^3$-distill” achieves FID $=11.7$ and IS $=182$ |
| Text few-step | DUO+DCD as $\pi_0$ | ReDi$^{1/2/3}$ match 1024-step perplexity at $32/16/8$ steps |

On ImageNet few-step generation, ReDi$^{1/2}$ improve MaskGIT’s 4-step FID from $10.9$ to $7.6/7.9$, rivaling or exceeding SDTT and Di4C. On ImageNet one-step generation, after $K=3$ rectifications, a one-step model denoted “ReDi$^3$-distill” achieves FID $=11.7$ with IS $=182$, far better than SDTT and Di4C one-step results with FID approximately $90$. On text generation, using DUO+DCD as $\pi_0$, ReDi$^{1/2/3}$ match 1024-step perplexity at $32/16/8$ steps, corresponding to a $32\times$–$128\times$ speedup. Combining Di4C with ReDi also yields further gains [2507.15897].

These experiments also motivate the one-step training strategy from rectified couplings. Once a “good” coupling $\pi_K$ with low TC has been obtained, a one-step DFM $p_\phi(X_1\mid X_0)$ can be trained directly on pairs sampled from $\pi_K$. No architectural change is needed; the training objective is simply set to match $\pi_K$ transitions at $t=0\rightarrow 1$. The intended effect is that the one-step model inherits the low factorization error of $\pi_K$ and can therefore perform high-quality single-pass sampling.

## 5. Terminological scope and related research lines

The term “ReDi” is not unique to the coupling-rectification method of discrete flow-based models. In the rectified-flow literature, “ReDi” is also used for “Discretized-RF,” a family of rectified flow models that discretizes the straight path into $T$ sub-paths, introduces a stochastic velocity or momentum field, and injects noise into the velocity on each sub-path in order to trade off between Rectified Flow’s efficiency and diffusion models’ diversity [2506.08796].

A separate but related use appears in motion generation. DisCoRD, “Discrete Tokens to Continuous Motion via Rectified Flow Decoding,” leverages rectified flow to decode discrete motion tokens in the continuous, raw motion space. Its decoder is a learned ODE vector field conditioned on per-frame features extracted from discrete tokens, and it is presented as plug-and-play for discrete motion generators such as T2M-GPT, MMM, MoMask, TalkSHOW, ProbTalk, and TM2D [2411.19527].

The discrete ReDi framework of coupling rectification also serves as a basis for subsequent extensions. AReUReDi, “Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidance,” explicitly builds on Rectified Discrete Flows and extends ReDi to multi-objective sampling in discrete state spaces. It combines Tchebycheff scalarization, locally balanced proposals, and annealed Metropolis-Hastings updates to bias sampling toward Pareto-optimal states while preserving distributional invariance [2510.00352].

This pattern suggests a terminological overlap rather than a single unified method: “rectified” methods in discrete settings include coupling rectification for DFMs, discretized rectified-flow trajectories in continuous spaces, and rectified-flow decoders for discrete-to-continuous generation.

## 6. Practical implications, limitations, and open directions

ReDi is presented as simple to implement on top of any existing DFM and as not requiring specialized distillation losses or teacher/student co-training. Its practical significance is therefore tied to a specific theoretical perspective: efficient discrete generation is improved by modifying the coupling so that the factorized decoder is asked to model transitions with lower conditional TC. This perspective provides a new route to few-step and one-step synthesis, rather than treating speedup as a purely architectural or solver-design problem [2507.15897].

The stated limitations are also coupling-centric. Sampling-error accumulation when generating pairs may degrade eventual model capacity. The current focus is on non-autoregressive DFMs, while autoregressive models are identified as an intriguing extension.

The future directions are correspondingly explicit. The paper points to deeper theoretical links between discrete and continuous rectified flows, automatic design of optimal couplings $\pi$ that minimize TC subject to target marginals, and extension of ReDi to accelerate autoregressive sampling by rectifying implicit transition couplings. A plausible implication is that the broader research program around ReDi is not limited to faster decoding; it also concerns how couplings should be constructed so that factorized generative transitions are intrinsically easier to learn and execute.

Source: https://www.emergentmind.com/topics/rectified-discrete-flow-redi