Draft-Conditioned Constrained Decoding (DCCD)

• The paper introduces DCCD, a two-step process that first generates an unconstrained draft and then applies hard constrained decoding to preserve structural validity while reducing semantic distortion.
• It achieves significant improvements in strict structured accuracy, for example boosting GSM8K results from 15.24% to 39.04% on a 1B model by leveraging draft-conditioned projections.
• DCCD decouples reasoning from formatting without retraining, enabling efficient handling of tasks like JSON object generation, executable code, and formal logic structures.

Draft-Conditioned Constrained Decoding (DCCD) is a two-step, training-free inference procedure for structured generation in LLMs. It was introduced to address a failure mode of standard constrained decoding: although token-level masking and renormalization guarantee structural validity, they can also distort the model’s sequence distribution when valid continuations receive low probability mass, thereby damaging reasoning and final answer quality. DCCD decouples semantic planning from structural enforcement by first generating an unconstrained draft and then performing constrained decoding conditioned on that draft, while preserving the hard validity guarantee of the second stage. The method is analyzed through a KL-projection view and is reported to improve strict structured accuracy by up to nearly 24 percentage points over standard constrained decoding, for example from $15.24\%$ to $39.04\%$ on GSM8K with a 1B model (Reddy et al., 8 Feb 2026).

1. Problem setting and motivation

DCCD is motivated by structured generation settings in which a single syntax error can render an output unusable. The target outputs include executable outputs, JSON objects, and API calls, and the benchmark suite in the original study spans GSM8K, MATH500, GSM-Symbolic, and FOLIO. The required structures are strict and machine-checkable: GSM8K and MATH500 use JSON with fields such as "steps" and "answer", GSM-Symbolic uses a symbolic expression wrapped in delimiters such as <<...>>, and FOLIO uses a first-order logic format with Predicates:, Premises:, and Conclusion: sections; for FOLIO, outputs are checked against a prover-based formal verification setup (Reddy et al., 8 Feb 2026).

The central claim behind DCCD is that standard constrained decoding is not merely a formatting filter. At every token, it masks invalid continuations and renormalizes over the remaining valid tokens. When the required structure forces low-probability syntax decisions—such as braces, quotes, delimiters, field names, or operators—the feasible set may receive only a small fraction of the model’s mass. Repeating this operation token by token can bias generation toward prefixes that are easy to keep syntactically valid rather than prefixes that are globally semantically correct. The paper emphasizes that this is especially harmful in reasoning-heavy tasks, where the model may already “know” the answer in an unconstrained form, but hard formatting constraints force it through unlikely trajectories (Reddy et al., 8 Feb 2026).

DCCD addresses this by separating free-form reasoning from constrained realization. The draft stage is permitted to produce reasoning, a plan, an outline, or other intermediate text without any requirement that it satisfy the final output schema. The structured output is then produced in a second stage under hard constraints, but now with access to the draft as auxiliary context. This suggests that the method is best understood as a conditional realization procedure rather than a relaxation of constrained decoding.

2. Reverse-KL formulation and the “projection tax”

The paper formalizes the base autoregressive model as

$$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$

For each prefix $h_t$, the valid next-token set is

$$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$

Standard constrained decoding then uses

$$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$

where $\alpha(h_t)$ is the feasible mass assigned by the base model to valid next tokens (Reddy et al., 8 Feb 2026).

A key result is that this per-step masking is exactly a reverse-KL projection: $$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$ The single-step distortion is

$$\mathrm{KL}\!\left(q(\cdot\mid h_t)\,\|\,\pi_\theta(\cdot\mid h_t)\right)=\log\frac{1}{\alpha(h_t)},$$

and the sequence-level distortion becomes

$$\mathrm{KL}\!\left(\rho_q(\cdot\mid x)\,\|\,\rho_\theta(\cdot\mid x)\right) = \mathbb{E}_{z\sim \rho_q(\cdot\mid x)}\Bigg[\sum_{t=1}^T \log\frac{1}{\alpha(h_t)}\Bigg].$$

The paper describes the cumulative quantity $39.04\%$0 as a “projection tax.” The interpretation is that whenever many valid prefixes have low feasible mass, constrained decoding repeatedly pays a distortion penalty (Reddy et al., 8 Feb 2026).

The paper also notes that valid sequences are reweighted by a trajectory-dependent factor: $39.04\%$1 Even among valid outputs, this favors trajectories with larger cumulative feasible mass. A direct implication is that constrained decoding can systematically prefer locally easy-to-project continuations over semantically correct ones.

An additional appendix argument ties this distortion to validity-gated utility. If

$39.04\%$2

then for two sequence distributions $39.04\%$3 and $39.04\%$4,

$39.04\%$5

Because standard constrained decoding incurs KL distortion equal to the expected cumulative projection tax, increasing feasible mass tightens the bound on utility degradation. This is the theoretical rationale for conditioning constrained realization on a draft (Reddy et al., 8 Feb 2026).

3. Two-stage procedure and draft conditioning

DCCD changes the conditioning context rather than the constraint itself. If an auxiliary draft $39.04\%$6 is appended to the context, the next-token distribution becomes $39.04\%$7, and the feasible mass becomes

$39.04\%$8

The paper’s core claim is that if the draft encodes the intended semantic plan, then the model becomes more confident in the valid structural tokens needed by the schema, so $39.04\%$9, and the subsequent hard projection is less distortive (Reddy et al., 8 Feb 2026).

Algorithmically, DCCD is a simple two-stage inference pipeline. In the first stage, a draft model generates an unconstrained draft

$$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$0

where $$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$1 may be free-form reasoning, a plan, an outline, or another intermediate representation not required to satisfy the final structure. In the second stage, a projector model generates the final structured output conditioned on both the prompt and the draft. Writing the projector’s conditional distribution as

$$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$2

DCCD performs constrained decoding under

$$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$3

The hard validity set $$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$4 is unchanged, because the constraint applies to the final output prefix $$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$5, not to the draft (Reddy et al., 8 Feb 2026).

The method is presented with a two-model abstraction consisting of a draft model $$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$6 and a projector model $$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$7. These can be the same model, or the projector can be smaller. The paper argues that this matters operationally because the second stage is primarily structured rendering rather than open-ended reasoning, which creates a route to decoupling reasoning capacity from formatting capacity.

DCCD also supports optional best-of-$$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$8 draft selection. The method samples $$\rho_\theta(z_{1:T}\mid x)=\prod_{t=1}^T \pi_\theta(z_t\mid h_t), \qquad h_t=(x,z_{<t}).$$9 drafts $h_t$0, runs constrained decoding for each, and scores candidate $h_t$1 by the cumulative log feasible mass

$h_t$2

The selected output satisfies

$h_t$3

This criterion prefers realizations that required less distortion during constrained decoding, though the paper notes that other selectors such as total log-likelihood under the constrained model, an external verifier, a task-specific judge, or majority voting among valid realizations are also possible (Reddy et al., 8 Feb 2026).

4. Evaluation protocol and empirical results

The experiments evaluate DCCD on models ranging from 1B to 14B parameters: Llama 3.2 1B, Qwen 2.5 1.5B, Qwen 2.5 3B, Qwen 2.5 7B, Llama 3.1 8B, and Qwen 2.5 14B. The baselines are Constrained Prompting (CP), Constrained Few-Shot (CF), and Constrained Decoding (CD), with CD implemented using XGrammar integrated into vLLM. The principal metric is strict structured accuracy, under which a response counts as correct only if it is both semantically correct and structurally valid (Reddy et al., 8 Feb 2026).

The headline finding is that DCCD consistently improves strict structured accuracy across every model size and every benchmark in the reported main table. On GSM8K with the 1B model, standard constrained decoding achieves $h_t$4 strict accuracy, while DCCD reaches $h_t$5. On the 1.5B model, GSM8K improves from $h_t$6 under CD to $h_t$7 under DCCD. Representative additional values reported in the text include GSM8K improvements for 3B from $h_t$8 to $h_t$9, for 7B from $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$0 to $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$1, and for 14B from $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$2 to $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$3. The paper states that the largest absolute improvements occur in the low-parameter regime, where the projection tax is most severe (Reddy et al., 8 Feb 2026).

The empirical comparisons are designed to separate structural validity from semantic correctness. CP and CF can sometimes improve answer quality compared with CD on some datasets, but they do not guarantee validity. CD guarantees structure but can reduce correctness. DCCD is presented as dominating both sides of that tradeoff by preserving exact format compliance through the second-stage masking while recovering semantic quality that is lost under standard CD.

Several analyses further characterize the method. In test-time scaling experiments with $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$4, DCCD scales better than CD: on GSM8K, DCCD rises from about $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$5 at $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$6 to $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$7 at $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$8, while CD rises from $$A(h_t) \triangleq \{a\in\mathcal{V}:\exists z_{t+1:T}\text{ such that }(z_{<t},a,z_{t+1:T})\in \mathcal{L}(x)\}.$$9 to $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$0; on MATH500, DCCD goes from $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$1 to $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$2 while CD goes from $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$3 to $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$4. The gains saturate after moderate $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$5, around $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$6, suggesting diminishing returns once sufficiently good drafts are found. For Llama 3.2 3B on GSM8K, DCCD also has a higher mean response confidence than CD, reported as $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$7 versus $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$8, a $$q(z_t\mid h_t)=\frac{\pi_\theta(z_t\mid h_t)\,\mathbb{I}[z_t\in A(h_t)]}{\alpha(h_t)}, \qquad \alpha(h_t)\triangleq \sum_{a\in A(h_t)} \pi_\theta(a\mid h_t),$$9 increase, and the token-wise plots show that DCCD preserves high confidence across positions while CD’s confidence decays over long sequences (Reddy et al., 8 Feb 2026).

The appendix extends the scope beyond formally verified reasoning tasks. On non-verifiable summarization-style tasks, DCCD is reported to beat CD under an LLM-as-judge across quality, coverage, and faithfulness, with roughly $\alpha(h_t)$0–$\alpha(h_t)$1 win rates. This suggests that the mechanism is not confined to exact-answer math or logic benchmarks, although the paper’s central evaluation remains strict structured accuracy on verified structured-generation tasks (Reddy et al., 8 Feb 2026).

5. Relation to adjacent constrained decoding paradigms

DCCD belongs to a broader family of methods that steer generation or inference toward valid outputs by using auxiliary structure during decoding, but its mechanism is specific: it conditions hard constrained realization on an unconstrained semantic draft. Two adjacent examples discussed in the supplied literature are LAVE for diffusion LLMs and the use of $\alpha(h_t)$2-constraint nodes during LDPC decoding in a Bliss scheme.

Method	Proposal or intermediate state	Constraint mechanism
DCCD	Unconstrained draft $\alpha(h_t)$3	Hard valid-next-token set $\alpha(h_t)$4 in second-stage constrained decoding
LAVE	dLLM-proposed token plus sampled lookahead completions	Accept only if a sampled complete prefix is extendable under the CFG
Bliss/LDPC with $\alpha(h_t)$5-constraint nodes	Tentative LDPC soft-bit estimates	Factor-graph augmentation with explicit $\alpha(h_t)$6-constraint nodes

LAVE, introduced for diffusion LLMs, is described as related to DCCD in spirit but methodologically different. It targets non-autoregressive generation under context-free grammars and uses a lookahead-then-verify mechanism: when the model proposes a token, it samples plausible values for the remaining masked positions using the model’s parallel token distributions, forms complete prefixes, and accepts the token only if at least one sampled completion is syntactically extendable. The paper explicitly states that LAVE is not a draft-conditioned method in the classic speculative or draft-then-verify sense used for autoregressive models, because the proposal comes from the dLLM itself and the verifier is grammar-based over sampled lookahead completions rather than a separate draft model (Zhang et al., 31 Jan 2026).

The earlier work on Bliss schemes and LDPC decoding offers a more distant but illuminating analogy. There, a run-length-limited $\alpha(h_t)$7 constraint is exploited during LDPC decoding by adding explicit $\alpha(h_t)$8-constraint nodes to the factor graph. These nodes send messages that exert a corrective “force” on the soft-bit vector so that it tends to comply with forbidden-pattern constraints such as $\alpha(h_t)$9 and $$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$0, and the paper reports about $$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$1 dB PSNR advantage, a channel bit length decrease of about $$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$2, and a storage density increase of about $$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$3. The supplied synopsis explicitly describes this as an instance of the same broad principle as DCCD: start with a baseline decoder producing a tentative output, inject hard output constraints or soft constraint feedback during decoding, and steer the result toward valid outputs (0707.3925).

A plausible implication is that DCCD should be viewed less as an isolated decoding trick than as one realization of a general constrained inference pattern. What distinguishes it is the use of an unconstrained semantic draft as side information for a hard constrained projector in autoregressive LLMs.

6. Interpretation, deployment implications, and common misconceptions

A common misconception is that DCCD relaxes or weakens constrained decoding. The paper states the opposite: the hard validity set $$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$4 is unchanged, because the constraint applies to the final output prefix rather than to the draft. The second stage still enforces exact validity token by token; what changes is the model’s conditional context during structured realization (Reddy et al., 8 Feb 2026).

A second misconception is that DCCD necessarily requires two different models. The paper presents a two-model abstraction, but it explicitly allows the draft model and projector model to be the same model. It also allows the projector to be smaller, because the second stage is largely structured rendering after the draft has supplied the semantic plan. This is the basis for the paper’s parameter-efficiency claim that smaller model pairs can match or exceed much larger constrained baselines on strict structured accuracy (Reddy et al., 8 Feb 2026).

A third misconception is that DCCD is a training or finetuning method. It is described as training-free and deployment-friendly: no retraining is required, only an extra draft-generation pass. The computational trade-off lies primarily in sampling and selection. When best-of-$$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$5 is used, test-time cost grows roughly linearly with the number of drafts, but the paper reports that additional samples are more effective for DCCD than for standard constrained decoding.

The paper’s case studies clarify the mechanism at the instance level. In a GSM8K example, constrained decoding yields a valid JSON answer but the wrong numeric result, whereas DCCD first reasons freely and then converts that reasoning into a valid JSON object with the correct answer. In a MATH500 example, standard constrained decoding produces a mathematically plausible but incorrect expression, while DCCD’s draft contains the correct derivation and the second stage renders it into the required JSON structure. The same pattern appears in GSM-Symbolic, where the unconstrained draft captures the right algebraic relationship and the final structured output preserves it (Reddy et al., 8 Feb 2026).

The broader interpretation advanced by the paper is that constrained decoding fails when structural validity is imposed too early on a reasoning process that has not yet stabilized semantically. DCCD’s response is to let the model “think” in an unconstrained medium and only then “format” under hard constraints. Within the formalism of feasible mass and reverse-KL projection, this corresponds to increasing $$q(\cdot\mid h_t)=\arg\min_{p\in\Delta_{A(h_t)}} \mathrm{KL}\!\left(p\,\|\,\pi_\theta(\cdot\mid h_t)\right).$$6 before projection and therefore lowering the cumulative projection tax. In that sense, DCCD is a conditional structured-realization strategy for LLMs: it preserves exact-validity guarantees while attempting to minimize the semantic damage introduced by hard constraints (Reddy et al., 8 Feb 2026).