Learning-to-Draft: Accelerating Inference

• Learning-to-Draft (LTD) is a framework where a lightweight draft model proposes tokens that are refined by a powerful verifier, enhancing overall inference throughput.
• The method integrates speculative decoding, reinforcement learning, and online adaptation to optimize drafting behavior and reduce processing latency.
• LTD has practical applications in translation, reasoning, and code completion, demonstrating significant speedups and improved efficiency over traditional models.

Searching arXiv for papers on Learning-to-Draft and related speculative decoding/adaptive drafter work. I’ll look up the most relevant arXiv records for “Learning-to-Draft”, speculative decoding, adaptive drafter, and online draft adaptation. Learning-to-Draft (LTD) denotes a family of methods in which a model learns to produce an initial draft that is subsequently refined, verified, or selectively accepted by a stronger model. In contemporary LLM inference, LTD is most closely associated with speculative decoding: a lightweight drafter proposes multiple tokens, and a larger target model verifies them in parallel, so that accepted tokens can be committed in bulk. The same drafting principle also appears in earlier two-stage generation systems, where a first-pass draft supplies global or right-context information for a second-pass model. Across these settings, LTD concerns how the draft is trained, aligned, adapted, and controlled so that drafting cost is outweighed by gains in acceptance length, throughput, or end-to-end latency (Li et al., 2017, Zafrir et al., 2024, Zhang et al., 2 Mar 2026, Ramakrishnan et al., 3 Jul 2025).

1. Definition and formal structure

In speculative decoding, a small draft model proposes a block of future tokens and a large target model verifies those proposals in parallel, accepting the draft up to the first mismatch. The central LTD problem is therefore not only to make the draft locally accurate, but to make it useful under the verifier’s acceptance rule. In the RL formulation of LTD, each draft-and-verify cycle has acceptance length $L_A$, total wall-clock time $T_{\mathrm{draft}} + T_{\mathrm{verify}}$, and throughput

$$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$

The objective is to choose drafting and verification behavior that maximizes average throughput rather than a proxy such as draft depth or acceptance length alone (Zhang et al., 2 Mar 2026).

This formulation clarifies several standard LTD metrics. FastDraft defines acceptance rate for block size $\gamma$ as

$$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$

and block efficiency as

$\tau^\gamma = 1 + \alpha^\gamma \gamma.$

Other work instead reports average accepted length, tokens/sec, or wall-time speedup. TLT, for example, emphasizes “acceptance lengths” and continuous alignment of a draft model $q_\phi$ to an evolving target policy $p_\theta$ during reinforcement-learning training, while OmniDraft focuses on acceptance rate and speedup under cross-vocabulary deployment (Zafrir et al., 2024, Hu et al., 20 Nov 2025).

2. Historical lineage: from refinement models to speculative decoding

A precursor to modern LTD appears in neural machine translation. “Enhanced Neural Machine Translation by Learning from Draft” introduced a two-stage process in which a conventional attention-based NMT system first produces a draft translation $\widetilde{Y}$, and a second double-attention model then refines the output using both the source $X$ and the draft: $T_{\mathrm{draft}} + T_{\mathrm{verify}}$0 The key motivation was that a left-to-right decoder lacks explicit right context, whereas a draft provides approximate future information. On Chinese-English translation, the double-attention model improved BLEU by $T_{\mathrm{draft}} + T_{\mathrm{verify}}$1 on the NIST task with $T_{\mathrm{draft}} + T_{\mathrm{verify}}$2 M pairs and by $T_{\mathrm{draft}} + T_{\mathrm{verify}}$3 on the IWSLT task with $T_{\mathrm{draft}} + T_{\mathrm{verify}}$4 K pairs (Li et al., 2017).

Modern speculative-decoding LTD shifted the emphasis from output refinement to inference acceleration. FastDraft exemplifies the offline-training paradigm. It trains a compact Transformer that shares the exact vocabulary of the target model, using efficient pre-training over $T_{\mathrm{draft}} + T_{\mathrm{verify}}$5 B unlabeled tokens and then fine-tuning on synthetic target-generated data. The draft variants include a Phi-3-mini-based 50 M model, a 120 M model, and a Llama-3.1-based 150 M model; training is reported on a single server with $T_{\mathrm{draft}} + T_{\mathrm{verify}}$6 Intel Gaudi 2 accelerators in under $T_{\mathrm{draft}} + T_{\mathrm{verify}}$7 hours. The resulting drafts achieve up to $T_{\mathrm{draft}} + T_{\mathrm{verify}}$8 memory-bound speedup in code completion and up to $T_{\mathrm{draft}} + T_{\mathrm{verify}}$9 in summarization, text completion, and instruction tasks, with up to $$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$0 wall-clock speedup on Intel Core Ultra hardware (Zafrir et al., 2024).

A related but distinct extension of the drafting idea appears in reasoning. Draft-Thinking trains models to produce concise “draft-style” chain-of-thought via supervised fine-tuning and progressive curriculum RL under hard length constraints. On MATH500, it reports an $$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$1 reduction in reasoning budget at the cost of only a $$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$2 performance drop, indicating that draft learning can target reasoning cost even outside verifier-based speculative decoding (Cao et al., 28 Feb 2026).

3. What LTD learns: alignment objectives and mismatch correction

A persistent LTD issue is that the draft is trained under one objective but evaluated under a different acceptance mechanism. FastDraft addresses this with a two-stage objective: standard next-token cross-entropy for pre-training,

$$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$3

followed by sequence-level knowledge distillation on target-generated synthetic responses,

$$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$4

Its ablations report that pure CE on synthetic examples gave the largest acceptance-rate boost, illustrating a recurrent LTD theme: alignment data sampled from the target’s own generation policy is often more valuable than generic pre-training data (Zafrir et al., 2024).

OmniDraft extends LTD to the cross-vocabulary case, where draft and target tokenizers differ. It introduces an online $$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$5-gram cache $$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$6 that records mappings from drafter sub-token sequences to target tokens, together with a translator and a hybrid distillation objective. For a cached mapping $$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$7, the translated draft probability is

$$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$8

The distillation loss combines reverse-KL on direct-mapping tokens with maximum-likelihood on merged $$X_c \;=\; \frac{L_A}{T_{\mathrm{draft}} + T_{\mathrm{verify}}}.$$9-gram tokens. This allows a single Llama-68M drafter to work with Vicuna-7B, Qwen2-7B, and Llama3-8B targets, and supports online adaptation on math reasoning, coding, and text generation tasks (Ramakrishnan et al., 3 Jul 2025).

EDA addresses a different form of misalignment: target models fine-tuned for specific domains. Its draft architecture separates a frozen shared expert $\gamma$0 from a trainable private expert $\gamma$1, with gate scores

$\gamma$2

and combined representation

$\gamma$3

EDA further uses target self-generation and representation-shift-based sample selection via PCA and Mahalanobis distance. Under a $\gamma$4 data budget, this preserves $\gamma$5 of full-data performance on average acceptance length $\gamma$6, while using $\gamma$7 MB trainable parameters and $\gamma$8 h adaptation time versus $\gamma$9 MB and $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$0 h for full retraining in the reported math transfer setting (Lin et al., 10 Mar 2026).

Two additional strands explicitly align training to the verifier’s actual reward structure. For block-diffusion drafters, “Teaching Diffusion to Speculate Left-to-Right” identifies the mismatch between bidirectional block generation and left-to-right autoregressive verification, then adds token positional weighting, first-error focal loss, and a chain reward surrogate for expected accepted length. Across six benchmarks, the full combination improves average accepted-draft length from $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$1 to $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$2 over a position-uniform baseline (Whalen et al., 10 Jun 2026). GTO addresses “draft policy misalignment” in tree-based speculative decoding by defining a Draft Tree Reward

$$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$3

and optimizing it with group-standardized advantages and a PPO-style surrogate. Averaged over tasks and models, it reports $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$4 in acceptance length and $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$5 speedup over EAGLE-3 at $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$6 (Hu et al., 26 Sep 2025).

4. Online and continual LTD

A major shift in the LTD literature is from static offline training to online adaptation driven by verification feedback. TLT develops this idea for reasoning RL training, where rollout lengths have a long-tail distribution. Its Adaptive Drafter is a single transformer decoder layer that re-uses the target model’s Embedding and LM-Head layers, with weights frozen. The draft consumes the target’s last hidden state and is updated opportunistically on idle GPUs via a Spot Trainer. Average of $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$7–$$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$8 training steps on idle GPUs is reported as sufficient to restore alignment after each target-model update. In evaluation, end-to-end RL throughput improves by $$\alpha^\gamma =\frac{1}{N\gamma}\sum_{n=1}^{N}\#\{\text{tokens accepted in block }n\},$$9 for $\tau^\gamma = 1 + \alpha^\gamma \gamma.$0B and $\tau^\gamma = 1 + \alpha^\gamma \gamma.$1 for $\tau^\gamma = 1 + \alpha^\gamma \gamma.$2B on H100s, while accept length increases from $\tau^\gamma = 1 + \alpha^\gamma \gamma.$3 to $\tau^\gamma = 1 + \alpha^\gamma \gamma.$4 tokens during the RL stage (Hu et al., 20 Nov 2025).

DVI makes the online loop explicit within a self-speculative single-model architecture. A decoder-only LLM is split at layer $\tau^\gamma = 1 + \alpha^\gamma \gamma.$5: shallow layers plus a trainable LoRA head act as the drafter, and deeper layers plus the frozen verifier head act as the target. During generation, accept/reject outcomes are stored as tuples $\tau^\gamma = 1 + \alpha^\gamma \gamma.$6, and the drafter is updated online with a KL$\tau^\gamma = 1 + \alpha^\gamma \gamma.$7RL schedule: $\tau^\gamma = 1 + \alpha^\gamma \gamma.$8 On Spec-Bench, DVI reaches a $\tau^\gamma = 1 + \alpha^\gamma \gamma.$9 wall-time speedup, with only $q_\phi$0 prompt exposures compared with $q_\phi$1 for Medusa, $q_\phi$2 for Kangaroo, and $q_\phi$3 for EAGLE-1/2 (Bhansali et al., 6 Oct 2025).

OnlineSpec pushes the same idea into online learning theory. It models each generation step as an online round with loss

$q_\phi$4

and links dynamic regret to accepted length and acceleration. It instantiates three algorithms: online gradient descent, optimistic mirror descent, and online ensemble learning. Across seven benchmarks and three foundation models, OnlineSPEC variants achieve up to $q_\phi$5 end-to-end wall-clock speedup over best prior SOTA, while preserving or improving output quality (Qian et al., 13 Mar 2026).

TIDE integrates online draft adaptation directly into a serving engine. It reuses target hidden states generated during inference as training signals, with no re-loading of the target model on training GPUs. Adaptive runtime control decides when speculation and training are beneficial, and inference and training are mapped to heterogeneous GPU classes. Reported gains include up to $q_\phi$6 throughput improvement over static speculative decoding and $q_\phi$7 faster draft training than approaches that recompute training signals (Park et al., 5 Feb 2026).

5. Runtime control as an LTD problem

LTD is not limited to learning draft parameters; it also includes learning how much to draft and how much to verify. The 2026 paper titled “Learning to Draft: Adaptive Speculative Decoding with Reinforcement Learning” formulates one speculative cycle as an MDP with two coupled actions. A depth policy decides whether to continue expanding the draft tree, and a size policy chooses verification size $q_\phi$8. The shared reward is throughput,

$q_\phi$9

and both policies are trained with PPO, first independently and then through iterative co-adaptation. Across five LLMs and four tasks, LTD reports speedup ratios from $p_\theta$0 to $p_\theta$1, outperforming Eagle3 by up to $p_\theta$2 (Zhang et al., 2 Mar 2026).

RADAR addresses a closely related question in tree-based speculative sampling. It treats repeated calls to the draft model as a finite-horizon MDP whose state is the vector of top-$p_\theta$3 confidence scores from the previous forward pass, action $p_\theta$4 means stop or continue drafting, and terminal reward is

$p_\theta$5

The policy is trained offline with REINFORCE on trajectories collected from EAGLE-3 draft trees. In reported experiments, RADAR achieves $p_\theta$6–$p_\theta$7 speedup over autoregressive decoding, while reducing the average number of draft-model calls per cycle by $p_\theta$8–$p_\theta$9 and lowering average $\widetilde{Y}$0 by only $\widetilde{Y}$1 relative to static tree construction (Ma et al., 16 Dec 2025).

Runtime adaptivity also appears in non-RL forms. OmniDraft trains a small head $\widetilde{Y}$2 to predict per-token acceptance probability and stop early when the running rejection probability exceeds a threshold $\widetilde{Y}$3, while TLT uses BEG-MAB to select $\widetilde{Y}$4 as a function of batch size. These designs indicate that drafting depth, verification breadth, and online alignment are interdependent rather than separable engineering choices (Ramakrishnan et al., 3 Jul 2025, Hu et al., 20 Nov 2025).

6. Empirical profile, scope, and recurrent limitations

Representative results illustrate the breadth of LTD outcomes across settings.

LTD setting	Representative result	Source
Cross-vocabulary on-device drafting	single Llama-68M with Vicuna-7B, Qwen2-7B, Llama3-8B; up to $\widetilde{Y}$5–$\widetilde{Y}$6 speedup	(Ramakrishnan et al., 3 Jul 2025)
Offline same-vocabulary draft training	under $\widetilde{Y}$7 h on $\widetilde{Y}$8 Gaudi 2; up to $\widetilde{Y}$9 wall-clock and $X$0 memory-bound speedup	(Zafrir et al., 2024)
Adaptive drafter for reasoning RL	$X$1 to $X$2 end-to-end RL training speedup	(Hu et al., 20 Nov 2025)
Training-aware self-speculation	$X$3 wall-time speedup on Spec-Bench	(Bhansali et al., 6 Oct 2025)
RL control of draft and verify	$X$4 to $X$5 speedup ratios	(Zhang et al., 2 Mar 2026)
Tree-policy alignment	$X$6 acceptance length and $X$7 speedup over EAGLE-3	(Hu et al., 26 Sep 2025)

Several recurrent limitations define the current scope of LTD. One is vocabulary or tokenizer incompatibility, which older speculative-decoding setups often avoided by requiring an identical token space; OmniDraft makes this constraint explicit and proposes cache-based translation to remove it (Ramakrishnan et al., 3 Jul 2025). A second is objective mismatch: single-path token-level training may not match tree-policy verification, as emphasized by GTO, or left-to-right acceptance in diffusion drafting, as emphasized by the chain and focal losses of diffusion LTD (Hu et al., 26 Sep 2025, Whalen et al., 10 Jun 2026). A third is distribution drift. TLT, DVI, OnlineSpec, TIDE, and EDA all treat evolving targets or workloads as first-class LTD problems rather than post hoc failure cases (Hu et al., 20 Nov 2025, Bhansali et al., 6 Oct 2025, Qian et al., 13 Mar 2026, Park et al., 5 Feb 2026, Lin et al., 10 Mar 2026).

The literature also uses non-identical evaluation targets. Some papers report acceptance rate, others average accepted length $X$8, block efficiency, memory-bound speedup, throughput in tokens/sec, or wall-time speedup. This makes direct numerical comparison difficult without matching hardware, batch size, target model, and draft-control policy. TLT, for example, reports batch-size sensitivity and an empirically optimal SD activation threshold of $X$9 requests, while TIDE explicitly enables or disables speculation depending on predicted practical speedup (Hu et al., 20 Nov 2025, Park et al., 5 Feb 2026).

A common narrowing of the term is to equate LTD with offline distillation of a fixed auxiliary drafter. The cited literature shows a broader usage: LTD now includes two-stage refinement, offline pre-training, online distillation, self-speculative partitioning, runtime policy learning, and serving-engine-native adaptation. This suggests that LTD is best understood as a general framework for learning a cheap proposer whose utility is measured not by standalone language-model quality, but by its interaction with a stronger verifier, refiner, or downstream reasoning process (Li et al., 2017, Zafrir et al., 2024, Zhang et al., 2 Mar 2026, Qian et al., 13 Mar 2026).