---
title: Discrete Classifier-Based Guidance (D-CBG)
url: https://www.emergentmind.com/topics/discrete-classifier-based-guidance-d-cbg
type: topic
---

# Discrete Classifier-Based Guidance (D-CBG)

Discrete Classifier-Based Guidance (D-CBG) is a family of algorithms for conditional sampling in discrete diffusion and flow models, enabling controllable generative modeling of categorical-valued data such as nucleotide sequences, molecules, and discretized images. D-CBG generalizes the continuous-state classifier guidance principle to discrete state spaces, combining an unconditional discrete denoiser or flow with a (possibly time-dependent) classifier to direct generation toward desired attributes or labels. The core mechanism adapts Bayes-inspired guidance to discrete Markov chains, employing reweighting of transition probabilities or rates by the likelihood ratio of the target property under a classifier, with a temperature hyperparameter modulating control strength. D-CBG is computationally efficient, requiring no retraining of the base generator, and achieves state-of-the-art conditional control across diverse discrete domains [2412.10193][2406.01572][2310.11311].

## 1. Discrete Diffusion and Flow: Model Setup

In D-CBG, data consists of sequences \(x \in V^L\) over a vocabulary of size \(N\), with each token represented in a one-hot encoding. Discrete diffusion models are constructed as forward Markov chains, where each time-step transition is defined by a categorical distribution parameterized by a transition matrix \(Q_{t|t-1}\):
\[
q(x_t \mid x_{t-1}) = \mathrm{Cat}(x_t; Q_{t|t-1} x_{t-1})
\]
A common “interpolating” schedule linearly mixes the input state with the uniform distribution, controlled by a time-varying parameter \(\alpha_t\).

The goal of generative modeling is to learn a reverse process approximating the true posterior \(q(x_{t-1} \mid x_t)\), which, due to intractable dependence on the original uncorrupted data, is parameterized by a neural denoiser \(p_\theta(x_{t-1} \mid x_t)\) trained via a variational objective over the trajectory. For continuous-time settings, models define a time-dependent rate/generator matrix \(R_t\) for the continuous-time Markov chain (CTMC) on the discrete state space [2406.01572].

## 2. Principle and Mathematical Formulation of D-CBG

D-CBG augments unconditional sampling to generate samples from a conditional distribution (e.g., with specified class \(y\) or property \(c\)). The guidance is implemented by combining the base model with a separately trained classifier. Let \(p_\phi(y \mid x)\) be the classifier output for attribute \(y\):
\[
p(x_{t-1} \mid x_t, y) \propto p_\phi(y \mid x_{t-1})^\gamma\, p_\theta(x_{t-1} \mid x_t)
\]
Here, the guidance strength \(\gamma > 0\) amplifies conditioning. In continuous-time settings, transition rates in the CTMC are reweighted for each possible state transition \(x \to \tilde x\) as:
\[
R_t^{(\gamma)}(x \to \tilde x \mid c) = \left[\frac{p^\phi(c \mid \tilde x, t)}{p^\phi(c \mid x, t)}\right]^\gamma R_t(x \to \tilde x)
\]
This guided process preserves ergodicity and ensures that, in the limit, samples are biased according to the specified attribute [2406.01572][2412.10193].

## 3. Efficient Implementation: Tokenwise and Taylor Approximations

Naïvely, evaluating the classifier for all candidate state replacements scales as \(O(LN)\) for sequences (or \(O(D(S-1))\) for general discrete objects), quickly becoming prohibitive for long sequences or extensive vocabularies. D-CBG exploits a first-order Taylor expansion, treating the classifier log-probability as a smooth function of the input:
\[
\log p_\phi(y \mid \tilde x) \approx \log p_\phi(y \mid x) + (\tilde x - x)^\top \nabla_x \log p_\phi(y \mid x)
\]
Since \(\tilde x\) and \(x\) differ in only one token, the entire batch of guided probabilities for all candidate mutations can be computed with a single forward and backward pass through the classifier. This enables practical application to high-dimensional biosequence and molecule spaces [2412.10193][2406.01572].

A standard pseudocode loop for the discrete-time case consists of, for each step and each token: computing denoiser logits, one classifier forward-backward for the current \(x_t\), constructing the guided logits, renormalizing with softmax, and sampling the next token (see Section 4 below).

## 4. Algorithmic Workflow and Pseudocode

The D-CBG sampling loop is realized as follows [2412.10193][2406.01572]:

1. **Unconditional Prediction:** For each position, compute denoiser logits \(\ell_{\theta}^{(\ell)}\).
2. **Classifier Pass:** Compute \(\log p_\phi(y \mid x_t)\) and its gradient w.r.t. \(x_t\).
3. **Guided Logits Construction (per token):**
    \[
    \ell_{\mathrm{guided}, i}^{(\ell)} = \ell_{\theta, i}^{(\ell)} + \gamma \left( \log p_\phi(y \mid x_t) + \delta \cdot \nabla_{x_t}\log p_\phi(y \mid x_t) \right)
    \]
    where \(\delta = \mathrm{onehot}(i) - x_t^{(\ell)}\).
4. **Softmax Renormalization:** Apply softmax across candidate indices to yield new categorical probabilities.
5. **Sampling:** Draw samples for \(x_{t-1}\).
6. **Iterate:** Set \(x_t \gets x_{t-1}\) and repeat.

In the continuous-time CTMC setting, one constructs the guided generator via reweighting as above and samples trajectories with standard CTMC samplers (Euler, Gillespie, or τ-leaping), with the Taylor expansion similarly amortizing predictor evaluations [2406.01572].

## 5. Practical Considerations and Guidance Strength

The classifier for D-CBG is typically trained under the same noise schedule as the diffusion model, predicting the label from noised latents \(x_t\) using cross-entropy loss [2412.10193]. The temperature parameter \(\gamma\) moderates the tradeoff between sample fidelity and conditional control—guidance of \(\gamma \sim 3-5\) is optimal for small-vocabulary tasks, while higher \(\gamma\) (up to 10) is effective in molecular domains. Fewer diffusion steps (e.g., \(T=128-512\)) suffice in uniform noise schemes due to the possibility of token flips at any step. CTMC simulating (\(\tau\)-leaping, Gillespie) can optimize accuracy in continuous-time D-CBG [2412.10193][2406.01572].

Resource requirements are dominated by backpropagation through the classifier for the gradient; a single GPU is adequate for typical biological sequence or molecular tasks with vocabulary sizes up to 32 and lengths in the range \(D\sim100-1000\) [2406.01572]. The Taylor-linear approximation enables all guided probabilities to be computed in \(O(1)\) classifier calls per time step.

## 6. Empirical Results Across Modalities

D-CBG achieves robust, high-performance controllable generation on a range of discrete domains:

| Domain                          | Quality/Controllability Metrics                | D-CBG Outcome                                                                          |
|----------------------------------|-----------------------------------------------|----------------------------------------------------------------------------------------|
| Genomic Sequences (Species10)    | 3-mer JS, Class F1                           | D-CBG (\(\gamma=3\)): 3-mer JS ≈ 0.11, F1 improved from 0.81 to 0.94                   |
| Molecular Design (QM9)           | Validity, Novelty, QED, Rings                 | QED maximization: 99.5% valid, 63.8% novel, mean QED 0.61 at \(\gamma=10\)             |
| Discretized CIFAR-10             | IS, FID, Class-conditional F1                 | IS 6.74→9.02, FID 33.8→15.6, F1 0.63→0.99 with \(\gamma=4\)                            |
| Small Molecule SMILES            | Rings purity, LogP targets                    | Targeted property shifts (90–93% purity), LogP control within MAE ≈ 0.7–0.9             |
| Discrete Images (CTMC, CIFAR-10) | IS, FID                                      | IS increases, FID decreases with stronger guidance (\(T=0.3\): IS 9.09, FID ≈ 9.04)    |
| Cell-type DNA enhancer design    | Fréchet Biological Distance, Target Prob.     | Outperforms Dirichlet FM classifier-guidance across control strengths                   |
| Protein inverse-folding          | Success rate (\(\Delta\Delta G,\,\)RMSD), Diversity | Success rate up to ∼40–93% (vs. 0–10%) with preserved diversity                         |

These results demonstrate that D-CBG preserves sample diversity while sharply increasing attribute controllability; the approach consistently outperforms autoregressive+FUDGE baselines and previous classifier-free methods [2412.10193][2406.01572].

## 7. Extensions, Related Work, and Limitations

D-CBG adapts classical classifier-based guidance for continuous diffusion processes [2310.11311] to discrete data, overcoming the absence of gradients in discrete spaces via likelihood-ratio modulation and smooth Taylor approximations. D-CBG enables the use of both bespoke and off-the-shelf classifiers, e.g., pretrained ResNet or CLIP for discreteized images and text-guided generation, with further improvements obtained through calibration (Softplus activation, temperature scaling) and input pre-conditioning [2310.11311].

In contrast to classifier-free and universal guidance, D-CBG requires a trainable or pretrained classifier, but does not necessitate retraining of the base generative model. In all applications, D-CBG achieves conditional control with minimal computational overhead, enabling state-of-the-art results for discrete data [2412.10193][2406.01572][2310.11311].

A plausible implication is that D-CBG establishes a general recipe for conditional sampling in any probabilistic discrete generative flow or diffusion model, with temperature scheduling and efficient gradient-based amortization as key enablers for scaling to long sequences and large vocabularies.

Source: https://www.emergentmind.com/topics/discrete-classifier-based-guidance-d-cbg