---
title: Rectified Guidance in Generative Models
url: https://www.emergentmind.com/topics/rectified-guidance-recfg
type: topic
---

# Rectified Guidance in Generative Models

Rectified Guidance (ReCFG) refers to a family of theoretically principled, geometry-aware conditioning methods for diffusive and flow-based generative models. These approaches resolve fundamental theoretical inconsistencies and stability issues in conventional guidance techniques—such as Classifier Guidance (CG) and Classifier-Free Guidance (CFG)—by either reparameterizing guidance coefficients, correcting vector fields via Jacobian information, or re-anchoring conditional trajectories to learned transport manifolds. ReCFG has been developed in several recent instantiations for diffusion, rectified flow, and flow matching models, each with distinct mathematical foundations and algorithmic frameworks [2405.14677, 2410.18737, 2501.18865, 2510.07631].

## 1. Theoretical Foundations of Rectified Guidance

Naive guidance schemes, including standard CFG and CG, often introduce theory-practice discrepancies. In conditional diffusion models, CFG computes the reverse-time vector field at each step as a weighted linear combination of conditional and unconditional predictions:
$$
s_{t,\gamma}(x_t|c) = \gamma\, s_t(x_t|c) + (1-\gamma)\, s_t(x_t)
$$
where $s_t(x_t|c) = \nabla_{x_t}\log q_t(x_t|c)$ and $s_t(x_t) = \nabla_{x_t}\log q_t(x_t)$. However, if $\gamma \neq 1$, this form induces a nonzero mean in the effective score, violating the reciprocal principle of score-based sampling and causing an expectation shift in the output distribution, which impairs sample fidelity as guidance strength increases [2410.18737]. 

Worse, standard guidance operates under a “scaled marginal” objective—applying the same modification across all timesteps—which is theoretically over-constrained and results in a degenerate diffusion process [2501.18865]. ReCFG addresses these mismatches by either correcting the guidance coefficients (to restore zero-mean property) or redesigning the guidance field using joint-scaling, detector Jacobians, or anchored manifold corrections, as further detailed below.

## 2. Rectified Guidance in Diffusion Models

### 2.1 Coefficient-Rectified CFG

Rectified CFG (also denoted “ReCFG” in [2410.18737]) eliminates the expectation shift by independently optimizing the weights for the conditional and unconditional scores. The rectified combination is
$$
s_{t,\gamma_1,\gamma_0}(x_t|c) = \gamma_1\, s_t(x_t|c) + \gamma_0\, s_t(x_t)
$$
with constraints:
- zero-mean: $\mathbb{E}_{q_t(\cdot | c)}[s_{t,\gamma_1,\gamma_0}(x_t|c)] = 0$
- sharpening: $\gamma_1>1$, $\gamma_0\le0$, $\gamma_1+\gamma_0\ge1$
- closed form solution: for target strength $w=\gamma_1$,
  $$
  \gamma_0(c,t) = -w\, R(c,t),\quad R(c,t) = \frac{\mathbb{E}[\varepsilon_\theta(x_t,c,t)]}{\mathbb{E}[\varepsilon_\theta(x_t,t)]}
  $$
where $\varepsilon_\theta$ is the network prediction. This correction is efficiently implemented: $R(c,t)$ is pre-computed, and inference uses a lookup per timestep and prompt, with no additional runtime overhead beyond two forward passes per sample step [2410.18737].

### 2.2 Rectified Gradient Guidance (REG)

Rectified Gradient Guidance (REG) [2501.18865] extends to a broader family of guidance functions, improving theoretical consistency by rederiving the optimal guidance field from a joint-scaled distribution objective. Let $R_0(x_0, y)$ be the reward applied at the endpoint, and $E_t(x_t, y)$ the future-trajectory expected reward,
$$
E_t(x_t, y) = \int p_\theta(x_0|x_t, y) R_0(x_0, y)\, dx_0
$$
The optimal (but intractable) guidance field is
$$
\bar{\epsilon}^\star_{\theta, t} = \epsilon_{\theta, t} - \sqrt{1-\bar{\alpha}_t}\, \nabla_{x_t} \log E_t(x_t, y)
$$
In practice, $E_t$ is approximated using the diagonal part of the local reward and the Jacobian of the sample-to-endpoint mapping, yielding the REG update:
$$
\bar{\epsilon}^{\rm REG}_{\theta, t} =
\epsilon_{\theta, t}
-\sqrt{1-\bar{\alpha}_t} \nabla_{x_t}\log R_t(x_t, y)
\odot
\left[ 1 - \sqrt{1-\bar{\alpha}_t}\, \frac{\partial}{\partial x_t}(\mathbf{1}^T\epsilon_{\theta, t})\right]
$$
This approach consistently reduces the discrepancy between practical and theoretically optimal guidance, at a modest overhead cost (one extra backward pass per sampling step) [2501.18865].

## 3. Rectified Guidance for Rectified Flows and Flow Matching

### 3.1 Anchored Classifier Guidance for Rectified Flows

Rectified Classifier Guidance (also denoted “ReCFG” in [2405.14677]) is designed specifically for flow-based ODE generative models employing rectified flows. The method transforms test-time classifier guidance into a fixed-point problem on the clean endpoint $z_1$, regularized by anchoring the guided trajectory to a reference (unguided) rectified flow:
$$
\hat{z}_1 = z_1 + s \cdot J \cdot \nabla_{z_1} \log p(c | z_1), \quad J := \nabla_{z_0} z_1
$$
where $J$ is the linearized transport Jacobian and $p(c | z)$ is an off-the-shelf image discriminator. Under mild Lipschitz assumptions, this fixed-point map is a contraction for sufficiently small $s$, ensuring convergence and stability. Piecewise updating and local linearization are used in practice [2405.14677].

This stabilization enables training-free identity personalization in rectified flows, avoiding the need for noise-aware classifiers and maintaining endpoint fidelity over diverse personalization prompts and discriminators.

### 3.2 Geometry-Aware Rectified-CFG++ for Flow Matching

Rectified-CFG++ [2510.07631] extends rectified guidance to general flow-matching architectures. Each ODE solver step consists of a predictor–corrector:
- Predictor: make a conditional flow (conditional velocity $v_\theta(x_t, t, y)$) step.
- Corrector: interpolate between conditional and unconditional velocities at the intermediate point using a scheduled weight $\alpha(t)$:
  $$
  \hat{v}_t = v^c_t + \alpha(t)\, (v_{t-\frac{\Delta t}{2}}^c - v_{t-\frac{\Delta t}{2}}^u)
  $$
Theoretical guarantees include marginal consistency (the marginal flow is unbiased with respect to the data manifold) and bounded deviation from the data manifold, proportional to $\max_t \alpha(t)$ and the velocity difference. This approach ensures stability across strong guidance scales and suppresses off-manifold drift, which is especially acute in deterministic rectified flow models when naïvely applying CFG [2510.07631].

## 4. Comparative Empirical Evaluation

Empirical evaluations consistently demonstrate that Rectified Guidance methods yield improvements over standard guidance in image fidelity, prompt adherence, and stability, often with minimal inference overhead:

| Model Type         | Method         | FID ↓         | CLIP Score ↑ | Comment                                   |
|--------------------|---------------|---------------|--------------|--------------------------------------------|
| EDM2 ImageNet      | CFG           | 5.59          | —            | Baseline (w=3.0)                           |
| EDM2 ImageNet      | ReCFG         | 4.84          | —            | Rectified coefficients [2410.18737]        |
| Stable Diffusion 3 | CFG           | 156.6         | 0.209        | 5 steps, w=7.5                             |
| Stable Diffusion 3 | ReCFG         | 140.9         | 0.229        | +9.6% CLIP, -10% FID [2410.18737]          |
| SD 3.5/Flux        | CFG           | 20.29/37.86   | 0.3506/0.3351| Baseline (ω=3.0)                           |
| SD 3.5/Flux        | ReCFG++       | 20.22/32.23   | 0.3497/0.3493| 5–15% FID gain [2510.07631]                |

In rectified flows, anchored classifier guidance achieves superior identity preservation on CelebA-HQ (identity ≃ 0.593 vs. 0.581 for state-of-the-art baselines), with robust handling of multiple identities and styles [2405.14677]. REG shows similar consistent gains (ΔFID ≈ 0.3) across diverse class- and text-conditional models [2501.18865].

## 5. Practical Implementation and Limitations

Rectified Guidance methods are designed for drop-in adoption in existing workflows. Coefficient rectification (ReCFG [2410.18737]) requires precomputing a lookup table of means for each prompt and timestep. REG [2501.18865] introduces a backprop-based correction factor, generally doubling gradient evaluation at inference. Anchored guidance [2405.14677] and predictor-corrector schemes [2510.07631] are compatible with any off-the-shelf discriminators or velocity field predictors, and bring negligible or moderate computational overhead.

Practical issues may arise in open-vocabulary text models for coefficient lookup, memory/compute in very large networks (REG’s extra backward), and out-of-manifold drift in non-rectified flows (alleviated by geometry-aware correction in Rectified-CFG++). All methods assume sufficiently accurate underlying unconditional and conditional predictors, and that transport trajectories can be locally linearized or reasonably approximated.

## 6. Significance, Variants, and Extensions

Rectified Guidance reconciles conditional generative modeling with foundational probability flows, yielding:
- Theoretical alignment with time-reversible diffusion and flow matching.
- Closed-form correction for expectation shift and marginal consistency.
- Tunable, precomputed, or data-driven guidance schedules and weights.
- Empirical robustness to strong guidance strengths, out-of-manifold deviation, and prompt-drift artifacts.

Variants include joint-scaling objectives, Jacobian-corrected vector fields, fixed-point-anchored personalization, and multi-modal predictor–corrector frameworks. These methodologies generalize to continuous normalizing flows and are extensible to spatio-temporal and multimodal domains. A plausible implication is increased reliability and sample quality for high-fidelity conditional generation tasks—including personalized and compositional image synthesis, video, and audio generation [2405.14677, 2410.18737, 2501.18865, 2510.07631].

Source: https://www.emergentmind.com/topics/rectified-guidance-recfg