---
title: 'Draft & Verify: Two-Stage Decoding Process'
url: https://www.emergentmind.com/topics/draft-verify
type: topic
---

# Draft & Verify: Two-Stage Decoding Process

Searching arXiv for recent "draft & verify" papers and the primary paper.
Draft & Verify denotes a broad decoding-as-selection pattern in which one stage generates candidate outputs and a subsequent stage evaluates, reranks, verifies, or refines them. In recent work, the pattern appears in embodied control, where a diffusion action expert drafts continuous action chunks and a Vision-Language Model (VLM) selects the lowest-perplexity chunk [2603.18091]; in multimodal grounding, where an initial Large Vision-Language Model (LVLM) answer is refined by selecting the expert-cued response with the largest utilization gain [2511.11005]; and in speculative decoding, where a drafter proposes tokens or trees and a target model verifies them while preserving the target distribution [2309.08168]. Across these settings, the common structure is not a single algorithm but a family of systems that separate proposal from adjudication, and that use the verify stage either to preserve exactness, to improve robustness under shift, or to increase grounding in external evidence.

## 1. Canonical pattern and design space

The recent literature uses “draft & verify,” “draft & refine,” and “draft-and-verify” for closely related two-stage procedures. In the most generic form, a drafter proposes multiple candidate outputs, and a verifier, critic, or refinement stage evaluates these candidates according to a learned criterion such as log-likelihood, perplexity, utilization gain, or an exact speculative-decoding acceptance rule [2603.18091]. The same high-level template has been instantiated for continuous robot actions, visual question answering, captioning, long-context LLM decoding, retrieval-augmented speculative decoding, and diffusion sampling [2511.11005], [2606.24957], [2603.25872].

| Domain | Draft stage | Verify/refine stage |
|---|---|---|
| Vision-Language-Action | Diffusion action expert drafts multiple action chunks | VLM scores candidates by a perplexity-style metric [2603.18091] |
| LVLM grounding | Base LVLM drafts an answer | Utilization-guided expert refinement selects the answer with the largest utilization increase [2511.11005] |
| Speculative decoding | Drafter proposes tokens, chains, or trees | Target model verifies under exact or adaptive rules [2309.08168] |
| Diffusion sampling | Skip transitions generate future draft states | Standard denoising process refines them [2603.25872] |

A central distinction in this design space concerns the role of verification. In speculative decoding, verification is typically tied to distributional correctness and end-to-end latency. In embodied control, verification is used to filter catastrophic or incoherent trajectories under distribution shift rather than to preserve exact token distributions [2603.18091]. In LVLM grounding, the verify stage measures reliance on visual evidence and uses that measurement to decide when and how to invoke visual experts [2511.11005].

## 2. Action Draft-and-Verify in Vision-Language-Action models

“Action Draft and Verify” (ADV) is an explicit draft-and-verify scheme for Vision-Language-Action models in which a diffusion action expert drafts multiple candidate continuous action chunks and a VLM verifies them by computing a perplexity-style score for each candidate in a single forward pass [2603.18091]. The motivation is to combine two complementary priors. Diffusion experts generate continuous, high-precision, temporally smooth actions and are efficient at low-level control, particularly when decoding chunks instead of single actions. Auto-regressive VLM-style action generation is slower and less precise for low-level control, but it leverages rich semantic and world knowledge from large-scale language-vision pre-training and is empirically more stable under distribution shift.

The base architecture is a VLM plus diffusion action expert. The vision encoder and language encoder take observation \(o\) and language instruction \(\ell\), and output a final-layer hidden feature \(f_c\) that serves as a conditioning context. The diffusion action expert is a transformer-style action model with 16 layers, hidden size 1536, 32 attention heads, alternating self-attention and cross-attention layers, dropout 0.2, and GELU activation. It is implemented as a flow-matching diffusion model conditioned on \(f_c\), and produces continuous action chunks \(A_n=\langle a_0,\dots,a_{H-1}\rangle\), where each \(a_k\) is a 7-DoF end-effector action, or concatenated 14-DoF for bimanual settings. The same VLM is also trained as an autoregressive policy over discrete action tokens and is used only for scoring at inference [2603.18091].

The diffusion expert is trained with a flow-matching objective,
$$
\mathcal{L}_\text{dif}(\theta,\phi)
=
\mathbb{E}_{(o,\ell,A)\sim\mathcal{D},\,\epsilon,\,\tau}
\Big[\big\|\pi_{\phi}(A^\tau,f_c,\tau)-(A-\epsilon)\big\|^2 \Big],
$$
where \(A^\tau=\tau A+(1-\tau)\epsilon\) and \(\epsilon\sim\mathcal{N}(0,I)\). The VLM verifier is trained with an autoregressive objective over discrete action tokens,
$$
\mathcal{L}_\text{VLM}(\theta)
=
\mathbb{E}_{(o,\ell,\tilde{A} )\sim\mathcal{D}}
\Big[-\sum_{i=0}^{T-1}\log \pi_\theta(t_i \mid o,\ell,t_{< i})\Big].
$$
Co-training combines the two losses as
$$
\mathcal{L}_\text{co-train}(\theta,\phi)=\mathcal{L}_\text{VLM}(\theta)+\beta\,\mathcal{L}_\text{dif}(\theta,\phi).
$$

At inference, the diffusion expert samples a set of candidate chunks
$$
S=\{A_0,A_1,\dots,A_{M-1}\},
$$
with \(M=16\) by default and 4 denoising steps by default. Each candidate is converted to discrete tokens via Textual FAST. The verifier then computes the length-normalized perplexity-style score
$$
C_{A_n} =\exp\!\left( -\frac{1}{T_n}\sum_{t_i\in \tilde{A}_n}\log \pi_{\theta}(t_i \mid o,\ell,t_{<i}) \right),
$$
and selects
$$
A^*=\arg\min_{A_n\in S} C_{A_n}.
$$
Textual FAST is crucial to this pipeline. It uses FAST to compress continuous actions through quantile normalization, Discrete Cosine Transform, quantization, flatten, and BPE compression; renders these action tokens as text; and then re-tokenizes with the standard VLM tokenizer. The stated motivation is to avoid introducing rare, action-specific vocabulary, align action tokens with the textual distribution of the base VLM, and make perplexity-based verification meaningful [2603.18091].

## 3. Empirical behavior in embodied control

Under matched backbone, data, and chunk length, ADV improves success by \(+4.3\) points in simulation and \(+19.7\) points in real-world over diffusion-only, with a single-pass VLM reranking overhead [2603.18091]. On LIBERO, Qwen2.5-VL plus diffusion reaches 93.9% average success and ADV reaches 96.8%; InternVL3.5 goes from 94.7% to 97.8%; and \(\pi_{0.5}\) goes from 96.5% to 98.4%. On RoboTwin2.0 with Qwen2.5-VL, Easy goes from 30.1% to 39.3%, and Hard goes from 6.7% to 10.2%. On real-world tasks, Qwen2.5-VL goes from 49.5% to 71.4%, InternVL3.5 from 58.6% to 76.1%, and \(\pi_{0.5}\) from 65.6% to 77.6%. On real-world unseen tasks, Qwen diffusion goes from 18.3% to 36.3%, InternVL diffusion from 18.5% to 36.0%, and \(\pi_{0.5}\) from 30.3% to 44.0%.

The paper’s failure analysis makes the rationale for verification explicit. Under out-of-distribution shift on RoboTwin2.0, the diffusion-based VLA has almost no recovery attempts, dropping from 4.5 to 0.4, and shows environment collision before first grasp rising from 0.0% to 29.7%. The auto-regressive baseline has lower success rates overall but is more stable, with many recovery attempts and almost no jitter, from 0.0% to 2.3%. ADV uses VLM perplexity to filter such failure modes. On LIBERO-Long, selecting the best, second-best, or third-best ranked chunk yields 94.6%, 94.8%, and 94.2%, whereas performance drops significantly for \(K\ge 4\) and a random candidate is about 90.2%. This suggests that the verifier mostly needs to reject bad chunks rather than identify a unique global optimum.

Additional analyses support this interpretation. On RoboTwin Hard, compared to diffusion-only, ADV reduces Avg. Success Steps by 45–283 steps, increases Recovery Attempts by 0.6–2.6, and reduces Env. Collision Before First Grasp by 14–43 percentage points. In a real “push blocks” recovery experiment with artificially long chunks of 32 steps, success after 1 long chunk and then normal 5-step chunks is 33% for diffusion and 83% for ADV; after 2 long chunks it is 17% and 33%. On LIBERO, the average verifier score in failures is about 2.110, while the average score in successes is about 1.465. In a noise-robustness study, when noise is added to all but one candidate, the chosen rate for the clean action increases up to about 99.6%. Textual FAST is also decisive: on LIBERO with Qwen2.5-VL-ADV, Action Bins-based verification reaches 93.0%, FAST 94.4%, VLA-0 94.9%, and Textual FAST 96.8%, while the diffusion-only baseline is 93.9% [2603.18091].

The stated limitations are equally important. ADV only selects among drafted actions; if the diffusion expert fails to produce any viable candidate chunk, verification cannot fix the trajectory. Compared with diffusion-only inference, ADV requires sampling multiple candidates and one batched VLM scoring pass. Long-horizon tasks may make per-step verification costly, and the authors identify improved chunking, selective verification, hierarchical schemes, better verification metrics, tighter joint training, and distillation of the verify stage as open directions [2603.18091].

## 4. Multimodal grounding and Draft-and-Refine with visual experts

“Draft and Refine” (DnR) applies the same overall pattern to LVLM grounding. For each image-question pair \((x,q)\), the base LVLM first outputs a draft answer \(\hat{y}\), then computes a question-conditioned utilization metric \(U_q(x)\), then invokes a pool of external visual experts, renders their outputs as visual cues, re-queries the LVLM, and selects the response whose utilization increase \(U_q^{(j)}-U_q^{\mathrm{base}}\) is largest if positive [2511.11005]. The verify stage is therefore not a token-level acceptance test but a quantitative measurement of reliance on question-relevant visual evidence.

The utilization metric has four parts. First, the question is decomposed into visual queries \(Q=\{q_1,\dots,q_m\}\) by LLaMA-3-70B. Second, CLIPSeg produces a relevance map for each query term, and these are averaged into
$$
r(x \mid q)=\frac{1}{m}\sum_{q_i\in Q} R(x\mid q_i).
$$
Third, relevance-guided probabilistic masking uses Gumbel-Top-k sampling to create Top-\(k\) and Bottom-\(k\) masks. Fourth, semantic deviation between the original answer and masked-answer variants is measured with CLIP-L/14 or SentenceTransformer all-MiniLM-L6-v2 embeddings, and aggregated into
$$
U_q(x) = \alpha \cdot \mathbb{E}_{\tau \in \mathcal{M}_q^{\text{top}}}[ d_{\tau} ] + (1 - \alpha) \cdot \mathbb{E}_{\tau \in \mathcal{M}_q^{\text{bottom}}}[ d_{\tau} ].
$$
Here \(\alpha\) is an adaptive weight derived from normalized entropy and contrast of the relevance distribution. A higher \(U_q(x)\) means that answers change when relevant regions are masked and do not change when irrelevant regions are masked.

DnR integrates GroundingDINO, SAM, DepthAnything V2, and mDETR as visual experts. Their outputs are rendered as GRAY or HIGHLIGHT cues, as well as boxes, masks, or depth maps, and re-fed to the LVLM without architectural changes. Experiments span 16 benchmarks and multiple backbones, including IDEFICS-2, InstructBLIP, MiniGPT-v2, LLaVA-1.6, PaliGemma, CogVLM, and Qwen2.5-VL. Reported gains include VQAv2 gains of \(+1\)–\(3\) points, captioning CIDEr gains \(+2\)–\(12\), visual reasoning gains \(+0.5\)–\(5.5\) points, and reduced hallucination on HaloQuest, MMHal-Bench, VizWiz hallucination annotations, and COCO caption hallucination categories [2511.11005]. Revision rates are often 2–10%, and on VQAv2 with LLaVA-1.6 the corrected-versus-degraded revised answers are 66.7% versus 9.2%.

The framework is also used as a basis for tool selection. A 3-layer MLP selector \(S_\theta(j\mid s)\), with state \(s=(x,Q,\hat{y},r(x\mid q))\), is trained to predict the expert that maximizes utilization gain. On PaliGemma, the selector reduces compute cost by about 70% on average across benchmarks, while accuracy or CIDEr drops by only about 0.2–0.4 points on VQAv2, COCO, VSR, and ScienceQA, and about 20 in MME’s large-scale score [2511.11005]. The main limitations are rendering sensitivity, weak effect when knowledge is missing, and the fact that the framework is single-step rather than multi-step.

## 5. Speculative decoding: self-speculation, adaptive trees, hybrid verification, and sparse verification

In LLM inference, Draft & Verify is most closely associated with speculative decoding. “Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding” uses the same LLM as both drafter and verifier by skipping layers during drafting and using the full model for verification, requires no additional neural network training and no extra memory footprint, and reports speedup up to 1.99\(\times\) on LLaMA-2 variants [2309.08168]. Subsequent work expands the design space along four axes: how drafts are generated, how verification is allocated, how verification itself is improved, and how the draft/verify loop is trained or parallelized.

On the drafting side, adaptive tree construction, parallel drafting, and non-autoregressive drafters all appear. OPT-Tree searches the draft tree that maximizes the expected acceptance length under a node budget and reports speed-up ratio of up to 3.2 compared with autoregressive decoding [2406.17276]. PEARL overlaps drafting and verification through pre-verify and post-verify, derives the optimal window size \(\gamma^\prime=c\), and reports up to 4.43\(\times\) over autoregressive decoding and 1.50\(\times\) over vanilla speculative decoding [2408.11850]. PARD adapts an autoregressive draft model into a parallel draft model that predicts multiple future tokens in a single forward pass, uses conditional drop token training with a 3\(\times\) training-efficiency gain, and reaches 4.08\(\times\) speedup with 311.5 tokens per second on LLaMA3.1-8B [2504.18583]. DEER replaces the autoregressive drafter with a diffusion language model, reports draft acceptance lengths of up to 32 tokens, and reaches 5.54\(\times\) on HumanEval with Qwen3-30B-A3B, compared with 2.41\(\times\) for EAGLE-3 [2512.15176].

On the verification side, recent work shows that acceptance rate is not the only variable. Reflective Verification augments target-model verification with prompt-based semantic reflection in a single forward pass and yields additional 5–15% improvements in decoding speed when combined with existing statistical verification methods [2505.18629]. UniVer casts tree-based verification as a conditional OT problem, proves losslessness and optimal acceptance under its framework, and improves acceptance length by 4.2% to 8.5% over standard recursive rejection sampling without replacement [2605.04543]. Dustin targets the long-context verification bottleneck: by combining draft-model lookahead signals with target-model historical attention and restricting scoring to a small subset of heads, it achieves a 27.85\(\times\) speedup in self-attention and a 9.17\(\times\) end-to-end decoding speedup at a 32k sequence length with negligible accuracy degradation [2606.24957].

A second line of work treats verification as a runtime allocation problem. Hybrid Verified Decoding predicts the accepted length of a cache draft before verification and chooses between cache verification and a model-based drafter; across three LLMs and sixteen datasets it is especially effective on agentic workflows, where it outperforms EAGLE3 in every setting with a 2.73\(\times\) average speedup [2606.01019]. Learning to Draft formulates the choice of draft depth and verification size as a reinforcement-learning problem that directly maximizes throughput per cycle, and reports speedup ratios ranging from 2.24\(\times\) to 4.32\(\times\), outperforming EAGLE3 up to 36.4% [2603.01639]. Graft identifies the budget released by pruning dense draft trees and spends it on retrieval: its sequential prune-then-graft mechanism is training-free and lossless, reaches up to 5.41\(\times\) speedup, and improves average speedup over EAGLE-3 by up to 21.8% on Qwen3-235B [2605.20104].

Training-aware verification closes the loop between inference and learning. DVI turns verifier accept/reject decisions into online supervision for a self-speculative drafter, uses a KL\(\rightarrow\)RL schedule, remains lossless and single-model at deployment, and reaches a 2.16\(\times\) wall-time speedup on Spec-Bench with orders of magnitude less training data than prior heavily trained baselines [2510.05421]. Across these variants, the verify stage has become a locus for exactness, efficiency, semantic filtering, cache selection, OT-based allocation, and online adaptation rather than a single acceptance/rejection primitive.

## 6. Trade-offs, misconceptions, and broader extensions

A common misconception is that Draft & Verify is primarily an acceleration trick. In embodied control, ADV uses verification to recover robustness and generalization under distribution shift, not merely to reduce latency [2603.18091]. In LVLM grounding, DnR uses verification to measure visual utilization and selectively invoke experts, not to imitate the target model’s next-token distribution [2511.11005]. By contrast, many speculative decoding systems are explicitly lossless and define verification so that the final sequence matches the target model’s distribution exactly, as in self-speculative decoding [2309.08168], UniVer [2605.04543], and Graft [2605.20104].

A second misconception is that better verification can compensate for arbitrarily poor drafts. The robotics literature explicitly states that if the diffusion expert fails to produce any viable candidate chunk, verification cannot fix the trajectory [2603.18091]. Similar draft-quality dependence appears in speculative decoding: HVD only reallocates verification between cache and model drafters, DEER still depends on diffusion-to-AR alignment, and LTD directly optimizes throughput because draft depth and verification size interact strongly with model speed and acceptance behavior [2606.01019], [2512.15176], [2603.01639].

The main trade-offs are recurrent. Verification improves robustness or acceptance length, but it introduces overhead. ADV samples \(M=16\) candidates and adds one batched VLM scoring pass; DnR may need exhaustive expert evaluation unless a selector is used; PEARL requires concurrent scheduling of the draft and target models; Dustin sparsifies verification attention but explicitly does not reduce total KV memory footprint; DEER shows training sensitivity in its Stage II refinement and notes that mature KV-cache support for dLLMs in mainstream stacks is still incomplete [2603.18091], [2511.11005], [2408.11850], [2606.24957], [2512.15176].

The paradigm is also no longer confined to text or robotics. DRiffusion transfers the same draft-and-refine idea to diffusion sampling: skip transitions generate multiple draft states for future timesteps, their noises are computed in parallel, and the standard denoising process produces refined results. The paper reports 1.4\(\times\)–3.7\(\times\) speedup across multiple diffusion models, with FID and CLIP largely on par with the original model and only minor average drops of 0.17 in PickScore and 0.43 in HPSv2.1 on MS-COCO [2603.25872]. This suggests that Draft & Verify is best understood not as a single verification rule but as a general computational pattern: draft with a cheap or parallelizable mechanism, then use a stronger model, a grounding metric, a utilization score, or an exact acceptance rule to decide what survives.

Source: https://www.emergentmind.com/topics/draft-verify