CFG Resolution Weighting (CFG-RW)

• CFG-RW is a method that rectifies the expectation shift in conventional classifier-free guidance by modifying the coefficient constraints.
• It relaxes the traditional linear sum-to-one restriction, enforcing a zero-mean property to maintain diffusion process consistency.
• Empirical results show that CFG-RW enhances FID scores and conditional alignment across various diffusion samplers with minimal computational overhead.

CFG Resolution Weighting (CFG-RW), more rigorously characterized as Rectified Classifier-Free Guidance (ReCFG), refers to a post-hoc modification of the coefficient selection used for classifier-free guidance in diffusion model sampling. Conventional classifier-free guidance (CFG) employs a linear combination of conditional and unconditional score estimates, governed by coefficients that sum to unity. However, this approach introduces a systematic bias—an "expectation shift"—which theoretically disrupts the reciprocity of the reverse diffusion process. CFG Resolution Weighting corrects this bias by relaxing the “sum-to-one” constraint, instead solving for guidance coefficients that enforce a zero-mean property of the combined score, thereby restoring theoretical consistency with the forward–reverse SDE/ODE framework and improving sampling fidelity in conditional generative modeling (Xia et al., 2024).

1. Theoretical Basis and Expectation Shift

Standard CFG replaces the true conditional score $\nabla_{x_t}\log q_t(x_t\mid c)$ with a weighted mixture: $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$ for $\gamma > 1$. In the $\epsilon$-prediction formulation this corresponds to

$$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$

While this sharpens the conditional distribution $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$, it violates the zero-mean property that is critical for diffusion-theoretic reversibility. Specifically,

$$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$

and thus,

$$\mathbb{E}_{x_t}[\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c)] = (1-\gamma)\,\mathbb{E}_{x_t}[\nabla_{x_t}\log q_t(x_t)] \neq 0$$

This expectation shift prevents the reverse process from precisely inverting the forward diffusion, resulting in a systematic bias away from $\mathbb{E}[x_0\mid c]$ (Xia et al., 2024).

2. Derivation of Rectified Guidance Weights

CFG Resolution Weighting introduces two free coefficients, $\alpha_c$ and $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$0, corresponding to conditional and unconditional branches: $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$1 with the $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$2-space form: $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$3 A zero-expectation (annihilation) constraint is enforced: $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$4 Estimating $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$5 and $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$6 via Monte Carlo, the optimal coefficient is found in closed form: $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$7 Practically, $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$8 is set to the guidance strength $$\nabla_{x_t}\log q_{t,\gamma}(x_t\mid c) = \gamma\,\nabla_{x_t}\log q_t(x_t\mid c) + (1-\gamma)\,\nabla_{x_t}\log q_t(x_t)$$9, so

$\gamma > 1$0

with practical constraints $\gamma > 1$1 and $\gamma > 1$2 typically satisfied for $\gamma > 1$3.

The relationship to original CFG is outlined as follows:

Approach	Coefficient Form	Constraint
CFG	$\gamma > 1$4	$\gamma > 1$5
ReCFG	$\gamma > 1$6	No sum constraint

3. Computation of Resolution Weights

CFG-RW requires precomputing the ratio

$\gamma > 1$7

for each condition $\gamma > 1$8 and timestep $\gamma > 1$9. This is achieved through a single-pass Monte Carlo estimate across the dataset:

1. Initialize accumulators $\epsilon$0, $\epsilon$1, $\epsilon$2 for each $\epsilon$3.
2. For each data sample $\epsilon$4 and each time $\epsilon$5:
   • Draw $\epsilon$6, set $\epsilon$7.
   • Compute $\epsilon$8 and $\epsilon$9.
   • Accumulate $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$0, $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$1, $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$2.
3. After traversal, set $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$3, $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$4, and $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$5.

This lookup table enables efficient runtime, as the coefficients can be retrieved with minimal computational overhead (Xia et al., 2024).

4. Integration with Diffusion Model Samplers

Most state-of-the-art diffusion samplers (e.g., DDIM, Euler–Maruyama, EDM2, SD3) use the following procedure in each denoising step: $$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$5 ReCFG replaces $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$6 with $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$7 and $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$8: $$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$6 No retraining or modification to the network is required. This modification is compatible with both class-conditioned (e.g., EDM2 on ImageNet) and text-conditioned (e.g., SD3 on CC12M) models (Xia et al., 2024).

5. Empirical Performance and Ablation Highlights

Empirical studies show quantifiable gains in both fidelity and conditional faithfulness:

Model/Dataset	Standard CFG	ReCFG (CFG-RW)	Metric	Change
LDM, ImageNet 256×256, 20 steps	FID ≈ 18.9	FID ≈ 16.9	FID	↓ 2.0
EDM2-S, ImageNet 512×512, 63 steps	FID ≈ 5.9	FID ≈ 4.8	FID	↓ 1.1
SD3, CC12M 512×512, 25 steps	CLIP ≈ 0.268, FID ≈ 72.2	CLIP ≈ 0.270, FID ≈ 71.8	CLIP, FID	↑0.002, ↓0.4

Ablation reveals:

• Lookup table estimates saturate in performance after ≈300 traversals per condition.
• The mean ratio $$\epsilon_{\gamma}(x_t,c,t) = \gamma\,\epsilon_\theta(x_t,c,t) + (1-\gamma)\,\epsilon_\theta(x_t,t)$$9 varies minimally across $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$0, justifying use of a global or average $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$1 for open-vocabulary text models with negligible loss (CLIP-Score loss ≤ 0.001).
• Storing pixel-wise $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$2 yields marginal additional benefit over a scalar per $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$3.

A one-dimensional Gaussian toy example demonstrates that standard CFG systematically shifts the mean, while ReCFG retrieves the exact mean and reduces variance (Xia et al., 2024).

6. Practical Recommendations

• Guidance strength $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$4 mediates the trade-off between fidelity and diversity, with optimal performance at $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$5–$q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$6 for class-conditional models and higher $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$7 for open-vocabulary prompting.
• In practice, $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$8, so $q_t(x_t\mid c)^\gamma q_t(x_t)^{1-\gamma}$9 with minor corrections; thus, ReCFG closely approximates boosting the conditional branch by $$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$0 while ensuring zero expectation in the unconditional branch.
• For high-resolution synthesis, as $$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$1, $$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$2, allowing $$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$3 for stability in late denoising steps.
• ReCFG can be rapidly implemented post-hoc for any pretrained conditional diffusion model with negligible computational overhead and consistent performance gains.

7. Significance and Implications

CFG Resolution Weighting enforces the theoretical zero-mean property absent in standard CFG by removing the linear coefficient constraint, aligning the sampling process with the requirements of diffusion SDE/ODE theory. The post-hoc nature and closed-form solution for the rectified coefficients permit integration without retraining or architecture changes. Empirical results indicate systematic improvements in FID and conditional alignment for both class-labeled and open-vocabulary generative tasks. The minimal variation in $$\mathbb{E}_{x_t\sim q_t(\cdot\mid c)}[\epsilon_\theta(x_t,t)] \neq 0$$4 across conditions suggests potential for further optimization in lookup-table storage and runtime efficiency. A plausible implication is that this approach may generalize beyond diffusion samplers currently demonstrated, offering a template for theoretical corrections to guidance heuristics in other generative domains (Xia et al., 2024).