Draft & Verify: Two-Stage Decoding Process

• Draft & Verify is a two-stage pattern where a candidate-drafter first generates multiple outputs and a verifier then selects or refines them using metrics like perplexity or utilization gain.
• The approach is applied across domains such as embodied control, LVLM grounding, and speculative decoding to improve performance and handle distribution shifts.
• Empirical studies reveal notable gains in success rates and robustness while also highlighting limitations when initial draft quality is insufficient.

Searching arXiv for recent "draft & verify" papers and the primary paper. Draft & Verify denotes a broad decoding-as-selection pattern in which one stage generates candidate outputs and a subsequent stage evaluates, reranks, verifies, or refines them. In recent work, the pattern appears in embodied control, where a diffusion action expert drafts continuous action chunks and a Vision-LLM (VLM) selects the lowest-perplexity chunk (Zhao et al., 18 Mar 2026); in multimodal grounding, where an initial Large Vision-LLM (LVLM) answer is refined by selecting the expert-cued response with the largest utilization gain (Jeong et al., 14 Nov 2025); and in speculative decoding, where a drafter proposes tokens or trees and a target model verifies them while preserving the target distribution (Zhang et al., 2023). Across these settings, the common structure is not a single algorithm but a family of systems that separate proposal from adjudication, and that use the verify stage either to preserve exactness, to improve robustness under shift, or to increase grounding in external evidence.

1. Canonical pattern and design space

The recent literature uses “draft & verify,” “draft & refine,” and “draft-and-verify” for closely related two-stage procedures. In the most generic form, a drafter proposes multiple candidate outputs, and a verifier, critic, or refinement stage evaluates these candidates according to a learned criterion such as log-likelihood, perplexity, utilization gain, or an exact speculative-decoding acceptance rule (Zhao et al., 18 Mar 2026). The same high-level template has been instantiated for continuous robot actions, visual question answering, captioning, long-context LLM decoding, retrieval-augmented speculative decoding, and diffusion sampling (Jeong et al., 14 Nov 2025, Lee et al., 23 Jun 2026, Bai et al., 26 Mar 2026).

Domain	Draft stage	Verify/refine stage
Vision-Language-Action	Diffusion action expert drafts multiple action chunks	VLM scores candidates by a perplexity-style metric (Zhao et al., 18 Mar 2026)
LVLM grounding	Base LVLM drafts an answer	Utilization-guided expert refinement selects the answer with the largest utilization increase (Jeong et al., 14 Nov 2025)
Speculative decoding	Drafter proposes tokens, chains, or trees	Target model verifies under exact or adaptive rules (Zhang et al., 2023)
Diffusion sampling	Skip transitions generate future draft states	Standard denoising process refines them (Bai et al., 26 Mar 2026)

A central distinction in this design space concerns the role of verification. In speculative decoding, verification is typically tied to distributional correctness and end-to-end latency. In embodied control, verification is used to filter catastrophic or incoherent trajectories under distribution shift rather than to preserve exact token distributions (Zhao et al., 18 Mar 2026). In LVLM grounding, the verify stage measures reliance on visual evidence and uses that measurement to decide when and how to invoke visual experts (Jeong et al., 14 Nov 2025).

2. Action Draft-and-Verify in Vision-Language-Action models

“Action Draft and Verify” (ADV) is an explicit draft-and-verify scheme for Vision-Language-Action models in which a diffusion action expert drafts multiple candidate continuous action chunks and a VLM verifies them by computing a perplexity-style score for each candidate in a single forward pass (Zhao et al., 18 Mar 2026). The motivation is to combine two complementary priors. Diffusion experts generate continuous, high-precision, temporally smooth actions and are efficient at low-level control, particularly when decoding chunks instead of single actions. Auto-regressive VLM-style action generation is slower and less precise for low-level control, but it leverages rich semantic and world knowledge from large-scale language-vision pre-training and is empirically more stable under distribution shift.

The base architecture is a VLM plus diffusion action expert. The vision encoder and language encoder take observation $o$ and language instruction $\ell$, and output a final-layer hidden feature $f_c$ that serves as a conditioning context. The diffusion action expert is a transformer-style action model with 16 layers, hidden size 1536, 32 attention heads, alternating self-attention and cross-attention layers, dropout 0.2, and GELU activation. It is implemented as a flow-matching diffusion model conditioned on $f_c$, and produces continuous action chunks $A_n=\langle a_0,\dots,a_{H-1}\rangle$, where each $a_k$ is a 7-DoF end-effector action, or concatenated 14-DoF for bimanual settings. The same VLM is also trained as an autoregressive policy over discrete action tokens and is used only for scoring at inference (Zhao et al., 18 Mar 2026).

The diffusion expert is trained with a flow-matching objective,

$$\mathcal{L}_\text{dif}(\theta,\phi) = \mathbb{E}_{(o,\ell,A)\sim\mathcal{D},\,\epsilon,\,\tau} \Big[\big\|\pi_{\phi}(A^\tau,f_c,\tau)-(A-\epsilon)\big\|^2 \Big],$$

where $A^\tau=\tau A+(1-\tau)\epsilon$ and $\epsilon\sim\mathcal{N}(0,I)$. The VLM verifier is trained with an autoregressive objective over discrete action tokens,

$$\mathcal{L}_\text{VLM}(\theta) = \mathbb{E}_{(o,\ell,\tilde{A} )\sim\mathcal{D}} \Big[-\sum_{i=0}^{T-1}\log \pi_\theta(t_i \mid o,\ell,t_{< i})\Big].$$

Co-training combines the two losses as

$\ell$0

At inference, the diffusion expert samples a set of candidate chunks

$\ell$1

with $\ell$2 by default and 4 denoising steps by default. Each candidate is converted to discrete tokens via Textual FAST. The verifier then computes the length-normalized perplexity-style score

$\ell$3

and selects

$\ell$4

Textual FAST is crucial to this pipeline. It uses FAST to compress continuous actions through quantile normalization, Discrete Cosine Transform, quantization, flatten, and BPE compression; renders these action tokens as text; and then re-tokenizes with the standard VLM tokenizer. The stated motivation is to avoid introducing rare, action-specific vocabulary, align action tokens with the textual distribution of the base VLM, and make perplexity-based verification meaningful (Zhao et al., 18 Mar 2026).

3. Empirical behavior in embodied control

Under matched backbone, data, and chunk length, ADV improves success by $\ell$5 points in simulation and $\ell$6 points in real-world over diffusion-only, with a single-pass VLM reranking overhead (Zhao et al., 18 Mar 2026). On LIBERO, Qwen2.5-VL plus diffusion reaches 93.9% average success and ADV reaches 96.8%; InternVL3.5 goes from 94.7% to 97.8%; and $\ell$7 goes from 96.5% to 98.4%. On RoboTwin2.0 with Qwen2.5-VL, Easy goes from 30.1% to 39.3%, and Hard goes from 6.7% to 10.2%. On real-world tasks, Qwen2.5-VL goes from 49.5% to 71.4%, InternVL3.5 from 58.6% to 76.1%, and $\ell$8 from 65.6% to 77.6%. On real-world unseen tasks, Qwen diffusion goes from 18.3% to 36.3%, InternVL diffusion from 18.5% to 36.0%, and $\ell$9 from 30.3% to 44.0%.

The paper’s failure analysis makes the rationale for verification explicit. Under out-of-distribution shift on RoboTwin2.0, the diffusion-based VLA has almost no recovery attempts, dropping from 4.5 to 0.4, and shows environment collision before first grasp rising from 0.0% to 29.7%. The auto-regressive baseline has lower success rates overall but is more stable, with many recovery attempts and almost no jitter, from 0.0% to 2.3%. ADV uses VLM perplexity to filter such failure modes. On LIBERO-Long, selecting the best, second-best, or third-best ranked chunk yields 94.6%, 94.8%, and 94.2%, whereas performance drops significantly for $f_c$0 and a random candidate is about 90.2%. This suggests that the verifier mostly needs to reject bad chunks rather than identify a unique global optimum.

Additional analyses support this interpretation. On RoboTwin Hard, compared to diffusion-only, ADV reduces Avg. Success Steps by 45–283 steps, increases Recovery Attempts by 0.6–2.6, and reduces Env. Collision Before First Grasp by 14–43 percentage points. In a real “push blocks” recovery experiment with artificially long chunks of 32 steps, success after 1 long chunk and then normal 5-step chunks is 33% for diffusion and 83% for ADV; after 2 long chunks it is 17% and 33%. On LIBERO, the average verifier score in failures is about 2.110, while the average score in successes is about 1.465. In a noise-robustness study, when noise is added to all but one candidate, the chosen rate for the clean action increases up to about 99.6%. Textual FAST is also decisive: on LIBERO with Qwen2.5-VL-ADV, Action Bins-based verification reaches 93.0%, FAST 94.4%, VLA-0 94.9%, and Textual FAST 96.8%, while the diffusion-only baseline is 93.9% (Zhao et al., 18 Mar 2026).

The stated limitations are equally important. ADV only selects among drafted actions; if the diffusion expert fails to produce any viable candidate chunk, verification cannot fix the trajectory. Compared with diffusion-only inference, ADV requires sampling multiple candidates and one batched VLM scoring pass. Long-horizon tasks may make per-step verification costly, and the authors identify improved chunking, selective verification, hierarchical schemes, better verification metrics, tighter joint training, and distillation of the verify stage as open directions (Zhao et al., 18 Mar 2026).

4. Multimodal grounding and Draft-and-Refine with visual experts

“Draft and Refine” (DnR) applies the same overall pattern to LVLM grounding. For each image-question pair $f_c$1, the base LVLM first outputs a draft answer $f_c$2, then computes a question-conditioned utilization metric $f_c$3, then invokes a pool of external visual experts, renders their outputs as visual cues, re-queries the LVLM, and selects the response whose utilization increase $f_c$4 is largest if positive (Jeong et al., 14 Nov 2025). The verify stage is therefore not a token-level acceptance test but a quantitative measurement of reliance on question-relevant visual evidence.

The utilization metric has four parts. First, the question is decomposed into visual queries $f_c$5 by LLaMA-3-70B. Second, CLIPSeg produces a relevance map for each query term, and these are averaged into

$f_c$6

Third, relevance-guided probabilistic masking uses Gumbel-Top-k sampling to create Top-$f_c$7 and Bottom-$f_c$8 masks. Fourth, semantic deviation between the original answer and masked-answer variants is measured with CLIP-L/14 or SentenceTransformer all-MiniLM-L6-v2 embeddings, and aggregated into

$f_c$9

Here $f_c$0 is an adaptive weight derived from normalized entropy and contrast of the relevance distribution. A higher $f_c$1 means that answers change when relevant regions are masked and do not change when irrelevant regions are masked.

DnR integrates GroundingDINO, SAM, DepthAnything V2, and mDETR as visual experts. Their outputs are rendered as GRAY or HIGHLIGHT cues, as well as boxes, masks, or depth maps, and re-fed to the LVLM without architectural changes. Experiments span 16 benchmarks and multiple backbones, including IDEFICS-2, InstructBLIP, MiniGPT-v2, LLaVA-1.6, PaliGemma, CogVLM, and Qwen2.5-VL. Reported gains include VQAv2 gains of $f_c$2–$f_c$3 points, captioning CIDEr gains $f_c$4–$f_c$5, visual reasoning gains $f_c$6–$f_c$7 points, and reduced hallucination on HaloQuest, MMHal-Bench, VizWiz hallucination annotations, and COCO caption hallucination categories (Jeong et al., 14 Nov 2025). Revision rates are often 2–10%, and on VQAv2 with LLaVA-1.6 the corrected-versus-degraded revised answers are 66.7% versus 9.2%.

The framework is also used as a basis for tool selection. A 3-layer MLP selector $f_c$8, with state $f_c$9, is trained to predict the expert that maximizes utilization gain. On PaliGemma, the selector reduces compute cost by about 70% on average across benchmarks, while accuracy or CIDEr drops by only about 0.2–0.4 points on VQAv2, COCO, VSR, and ScienceQA, and about 20 in MME’s large-scale score (Jeong et al., 14 Nov 2025). The main limitations are rendering sensitivity, weak effect when knowledge is missing, and the fact that the framework is single-step rather than multi-step.

5. Speculative decoding: self-speculation, adaptive trees, hybrid verification, and sparse verification

In LLM inference, Draft & Verify is most closely associated with speculative decoding. “Draft & Verify: Lossless LLM Acceleration via Self-Speculative Decoding” uses the same LLM as both drafter and verifier by skipping layers during drafting and using the full model for verification, requires no additional neural network training and no extra memory footprint, and reports speedup up to 1.99$A_n=\langle a_0,\dots,a_{H-1}\rangle$0 on LLaMA-2 variants (Zhang et al., 2023). Subsequent work expands the design space along four axes: how drafts are generated, how verification is allocated, how verification itself is improved, and how the draft/verify loop is trained or parallelized.

On the drafting side, adaptive tree construction, parallel drafting, and non-autoregressive drafters all appear. OPT-Tree searches the draft tree that maximizes the expected acceptance length under a node budget and reports speed-up ratio of up to 3.2 compared with autoregressive decoding (Wang et al., 2024). PEARL overlaps drafting and verification through pre-verify and post-verify, derives the optimal window size $A_n=\langle a_0,\dots,a_{H-1}\rangle$1, and reports up to 4.43$A_n=\langle a_0,\dots,a_{H-1}\rangle$2 over autoregressive decoding and 1.50$A_n=\langle a_0,\dots,a_{H-1}\rangle$3 over vanilla speculative decoding (Liu et al., 2024). PARD adapts an autoregressive draft model into a parallel draft model that predicts multiple future tokens in a single forward pass, uses conditional drop token training with a 3$A_n=\langle a_0,\dots,a_{H-1}\rangle$4 training-efficiency gain, and reaches 4.08$A_n=\langle a_0,\dots,a_{H-1}\rangle$5 speedup with 311.5 tokens per second on LLaMA3.1-8B (An et al., 23 Apr 2025). DEER replaces the autoregressive drafter with a diffusion LLM, reports draft acceptance lengths of up to 32 tokens, and reaches 5.54$A_n=\langle a_0,\dots,a_{H-1}\rangle$6 on HumanEval with Qwen3-30B-A3B, compared with 2.41$A_n=\langle a_0,\dots,a_{H-1}\rangle$7 for EAGLE-3 (Cheng et al., 17 Dec 2025).

On the verification side, recent work shows that acceptance rate is not the only variable. Reflective Verification augments target-model verification with prompt-based semantic reflection in a single forward pass and yields additional 5–15% improvements in decoding speed when combined with existing statistical verification methods (Wang et al., 24 May 2025). UniVer casts tree-based verification as a conditional OT problem, proves losslessness and optimal acceptance under its framework, and improves acceptance length by 4.2% to 8.5% over standard recursive rejection sampling without replacement (Weng et al., 6 May 2026). Dustin targets the long-context verification bottleneck: by combining draft-model lookahead signals with target-model historical attention and restricting scoring to a small subset of heads, it achieves a 27.85$A_n=\langle a_0,\dots,a_{H-1}\rangle$8 speedup in self-attention and a 9.17$A_n=\langle a_0,\dots,a_{H-1}\rangle$9 end-to-end decoding speedup at a 32k sequence length with negligible accuracy degradation (Lee et al., 23 Jun 2026).

A second line of work treats verification as a runtime allocation problem. Hybrid Verified Decoding predicts the accepted length of a cache draft before verification and chooses between cache verification and a model-based drafter; across three LLMs and sixteen datasets it is especially effective on agentic workflows, where it outperforms EAGLE3 in every setting with a 2.73$a_k$0 average speedup (Su et al., 31 May 2026). Learning to Draft formulates the choice of draft depth and verification size as a reinforcement-learning problem that directly maximizes throughput per cycle, and reports speedup ratios ranging from 2.24$a_k$1 to 4.32$a_k$2, outperforming EAGLE3 up to 36.4% (Zhang et al., 2 Mar 2026). Graft identifies the budget released by pruning dense draft trees and spends it on retrieval: its sequential prune-then-graft mechanism is training-free and lossless, reaches up to 5.41$a_k$3 speedup, and improves average speedup over EAGLE-3 by up to 21.8% on Qwen3-235B (Shen et al., 19 May 2026).

Training-aware verification closes the loop between inference and learning. DVI turns verifier accept/reject decisions into online supervision for a self-speculative drafter, uses a KL$a_k$4RL schedule, remains lossless and single-model at deployment, and reaches a 2.16$a_k$5 wall-time speedup on Spec-Bench with orders of magnitude less training data than prior heavily trained baselines (Bhansali et al., 6 Oct 2025). Across these variants, the verify stage has become a locus for exactness, efficiency, semantic filtering, cache selection, OT-based allocation, and online adaptation rather than a single acceptance/rejection primitive.

6. Trade-offs, misconceptions, and broader extensions

A common misconception is that Draft & Verify is primarily an acceleration trick. In embodied control, ADV uses verification to recover robustness and generalization under distribution shift, not merely to reduce latency (Zhao et al., 18 Mar 2026). In LVLM grounding, DnR uses verification to measure visual utilization and selectively invoke experts, not to imitate the target model’s next-token distribution (Jeong et al., 14 Nov 2025). By contrast, many speculative decoding systems are explicitly lossless and define verification so that the final sequence matches the target model’s distribution exactly, as in self-speculative decoding (Zhang et al., 2023), UniVer (Weng et al., 6 May 2026), and Graft (Shen et al., 19 May 2026).

A second misconception is that better verification can compensate for arbitrarily poor drafts. The robotics literature explicitly states that if the diffusion expert fails to produce any viable candidate chunk, verification cannot fix the trajectory (Zhao et al., 18 Mar 2026). Similar draft-quality dependence appears in speculative decoding: HVD only reallocates verification between cache and model drafters, DEER still depends on diffusion-to-AR alignment, and LTD directly optimizes throughput because draft depth and verification size interact strongly with model speed and acceptance behavior (Su et al., 31 May 2026, Cheng et al., 17 Dec 2025, Zhang et al., 2 Mar 2026).

The main trade-offs are recurrent. Verification improves robustness or acceptance length, but it introduces overhead. ADV samples $a_k$6 candidates and adds one batched VLM scoring pass; DnR may need exhaustive expert evaluation unless a selector is used; PEARL requires concurrent scheduling of the draft and target models; Dustin sparsifies verification attention but explicitly does not reduce total KV memory footprint; DEER shows training sensitivity in its Stage II refinement and notes that mature KV-cache support for dLLMs in mainstream stacks is still incomplete (Zhao et al., 18 Mar 2026, Jeong et al., 14 Nov 2025, Liu et al., 2024, Lee et al., 23 Jun 2026, Cheng et al., 17 Dec 2025).

The paradigm is also no longer confined to text or robotics. DRiffusion transfers the same draft-and-refine idea to diffusion sampling: skip transitions generate multiple draft states for future timesteps, their noises are computed in parallel, and the standard denoising process produces refined results. The paper reports 1.4$a_k$7–3.7$a_k$8 speedup across multiple diffusion models, with FID and CLIP largely on par with the original model and only minor average drops of 0.17 in PickScore and 0.43 in HPSv2.1 on MS-COCO (Bai et al., 26 Mar 2026). This suggests that Draft & Verify is best understood not as a single verification rule but as a general computational pattern: draft with a cheap or parallelizable mechanism, then use a stronger model, a grounding metric, a utilization score, or an exact acceptance rule to decide what survives.