LK Losses for Speculative Decoding

• LK losses are training objectives for speculative decoding that directly optimize token acceptance rate by replacing KL divergence with adaptive hybrid KL-TV and likelihood-based losses.
• The method employs an adaptive mix of KL and total variation objectives to effectively improve average acceptance length by up to 8–10% in low-capacity draft models.
• Empirical evaluations across multiple architectures demonstrate that LK losses enhance decoding efficiency without adding computational overhead.

Searching arXiv for the main paper and core speculative decoding references. arxiv_search(query="^^^^^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^^^^^", max_results=^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^) Searching arXiv for speculative decoding and acceptance-rate references. arxiv_search(query="^^^^^^^^4speculative decoding Leviathan acceptance total variation4^^^^^^^^", max_results=^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^) arxiv_search(query="^^^^^^^^4(Samarin et al., 27 Feb 2026)4 max_results=5) arxiv_search(query="^^^^^^^^4\4 Losses: Direct Acceptance Rate Optimization for Speculative Decoding^^^^^^^^4\4 max_results=5) LK losses are training objectives for speculative decoding that directly optimize the acceptance rate, and consequently the average acceptance length, of a draft or speculator model against a fixed target model instead of relying on Kullback–Leibler divergence as a proxy objective. Introduced for lossless speculative sampling, they are formulated as drop-in replacements for standard KL-based draft training and comprise two variants: an adaptive hybrid KL–TV objective and a likelihood-based objective that minimizes the negative log acceptance. The method is motivated by the observation that, although KL divergence and acceptance rate share the same global optimum at PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^, limited-capacity draft models often converge to regimes in which minimizing KL does not maximize acceptance. Across four draft architectures and six target models ranging from PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4^^^^ to PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4^^^^ parameters, LK losses are reported to improve acceptance metrics consistently, with gains of up to PRESERVED_PLACEHOLDER_^^^^4\4^^^^–10%10\% in average acceptance length, without computational overhead ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

^^^^4speculative decoding Leviathan acceptance total variation4^^^^. Speculative decoding setting and acceptance metrics

Speculative decoding accelerates autoregressive LLM inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. In the standard lossless speculative sampling setting, the target model defines a next-token distribution $p(x \mid c, x_{<i})$ and the draft model defines $q(x \mid c, x_{<i})$, where $c$ is the context and $x_{<i}=(x_1,\dots,x_{i-1})$ is the drafted prefix. Draft tokens are sampled from $q$ and then verified with the rejection-sampling criterion introduced for speculative decoding by Leviathan et al. (&&&^^^^4\4^^^^&&&).

For a drafted token PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^, the verifier uses the per-token acceptance probability

PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4^^^^

Tokens are verified in parallel but accepted sequentially; the first rejection terminates the accepted chunk and discards later drafts. The expected acceptance probability at position PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4^^^^ is

PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4^^^^

When the conditioning is clear, the paper writes

PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44^^^^

A central identity is the equivalence between acceptance and total variation distance:

PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4^^^^5

The global optimum is therefore attained at PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4^^^^6, where PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4^^^^7 and PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4^^^^8 ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The paper treats per-step acceptance PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4^^^^9 as the fundamental optimization target and uses average acceptance length per speculation round, PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^, as the principal evaluation metric:

PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4speculative decoding Leviathan acceptance total variation4^^^^

where PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4(Samarin et al., 27 Feb 2026)4^^^^ is the maximum draft length and the “PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4\4^^^^” accounts for the guaranteed post-verification target token. Under an independence approximation used only for intuition, the expected number of accepted tokens in a chain of length PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)44^^^^ would be PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4^^^^5; the training procedure nevertheless optimizes the per-position PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4^^^^6 directly ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

^^^^4(Samarin et al., 27 Feb 2026)4^^^^. Why KL divergence is a proxy rather than the target

Standard speculator training minimizes PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4^^^^7. Under infinite draft capacity, this is consistent with maximizing acceptance, because

PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4^^^^8

The paper’s claim is not that KL is incorrect in the limit, but that it is only a proxy objective when the draft cannot match the target exactly due to limited capacity, architectural mismatch, or truncated vocabularies ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The acceptance objective depends on overlap,

PRESERVED_PLACEHOLDER_^^^^4(Samarin et al., 27 Feb 2026)4^^^^9

whereas forward KL is mode-covering. This distinction is operationally important for small drafts: probability mass can be arranged in a way that reduces divergence without increasing the overlap that determines verifier acceptance. The exact identity PRESERVED_PLACEHOLDER_^^^^4\4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ implies that maximizing acceptance is equivalent to minimizing total variation, not KL. The paper further invokes the Pinsker-type inequality

PRESERVED_PLACEHOLDER_^^^^4\4speculative decoding Leviathan acceptance total variation4^^^^

to show that large KL implies large TV, but small KL does not tightly imply small TV.

The argument is sharpened at the gradient level. For draft logits PRESERVED_PLACEHOLDER_^^^^4\4(Samarin et al., 27 Feb 2026)4^^^^ with PRESERVED_PLACEHOLDER_^^^^4\4\4^^^^, the forward-KL gradient is

PRESERVED_PLACEHOLDER_^^^^4\44^^^^

For total variation, if PRESERVED_PLACEHOLDER_^^^^4\4^^^^5 and PRESERVED_PLACEHOLDER_^^^^4\4^^^^6, then

PRESERVED_PLACEHOLDER_^^^^4\4^^^^7

The TV gradient depends on error sign rather than error magnitude and is non-smooth along PRESERVED_PLACEHOLDER_^^^^4\4^^^^8. In the early-training regime with large vocabularies and diffuse PRESERVED_PLACEHOLDER_^^^^4\4^^^^9, the paper states that pure TV gradients are tiny, with norm $10\%$^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ when $10\%$^^^^4speculative decoding Leviathan acceptance total variation4^^^^ is concentrated on an effective support of size $10\%$^^^^4(Samarin et al., 27 Feb 2026)4^^^^, whereas KL gradients remain well scaled at $10\%$^^^^4\4^^^^. This is the core failure mode behind the paper’s critique of pure direct TV optimization from random initialization ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

A common misconception is therefore that the exact identity $10\%$4 makes pure TV an immediately practical replacement for KL. The reported ablations reject that conclusion: pure TV substantially underperforms because its gradients vanish early, while KL remains stable but mismatched to the ultimate acceptance objective ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

^^^^4\4^^^^. Formal definition of the LK objectives

The paper proposes two LK objectives. The first is a hybrid KL–TV loss:

$10\%$5

Its mixing coefficient is adaptive rather than fixed:

$10\%$6

with $10\%$7 and $10\%$8 the stop-gradient operator. The coefficient is computed per position using batch-aggregated $10\%$9 for that head. As acceptance is small, $p(x \mid c, x_{<i})$^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ and training is KL-dominated; as acceptance rises, $p(x \mid c, x_{<i})$^^^^4speculative decoding Leviathan acceptance total variation4^^^^ and the loss becomes TV-dominated. The schedule is intended to retain KL’s optimization stability early and shift toward direct acceptance optimization later ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The second variant is a likelihood-based objective:

$p(x \mid c, x_{<i})$^^^^4(Samarin et al., 27 Feb 2026)4^^^^

This objective directly maximizes the marginal probability of acceptance. In the degenerate case of a point-mass target $p(x \mid c, x_{<i})$^^^^4\4^^^^, it reduces to the standard negative log-likelihood,

$p(x \mid c, x_{<i})$4

That reduction connects the objective to conventional language-model training while preserving its acceptance-centered meaning ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The likelihood-based gradient reveals why it avoids TV’s vanishing-gradient pathology:

$p(x \mid c, x_{<i})$5

The paper interprets this as TV optimization with adaptive gradient scaling by $p(x \mid c, x_{<i})$6. In the early regime with diffuse $p(x \mid c, x_{<i})$7 and peaked $p(x \mid c, x_{<i})$8, where $p(x \mid c, x_{<i})$9, the resulting gradient norm is approximately $q(x \mid c, x_{<i})$^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^, matching KL’s magnitude while following TV’s direction.

Neither objective is convex in $q(x \mid c, x_{<i})$^^^^4speculative decoding Leviathan acceptance total variation4^^^^. KL is smooth, whereas TV and $q(x \mid c, x_{<i})$^^^^4(Samarin et al., 27 Feb 2026)4^^^^ are non-smooth. The paper nevertheless treats both as workable with the adaptive blend and scaling mechanisms above. At $q(x \mid c, x_{<i})$^^^^4\4^^^^, all objectives attain the same global optimum: $q(x \mid c, x_{<i})$4, $q(x \mid c, x_{<i})$5, and KL is minimized ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

4. Training procedure, implementation, and truncated vocabularies

LK losses are trained against a frozen target model. The inputs are the target logits $q(x \mid c, x_{<i})$6 and probabilities $q(x \mid c, x_{<i})$7, the draft logits $q(x \mid c, x_{<i})$8 and probabilities $q(x \mid c, x_{<i})$9, a dataset of prompt–continuation pairs generated by the target model, and $c$^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ speculative heads. For each training step, the procedure computes $c$^^^^4speculative decoding Leviathan acceptance total variation4^^^^ and $c$^^^^4(Samarin et al., 27 Feb 2026)4^^^^ at each drafted position $c$^^^^4\4^^^^, evaluates

$c$4

constructs either $c$5 or $c$6 per head, and aggregates across heads with exponential decay

$c$7

Gradients are backpropagated into draft parameters only; the target is not updated ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The reported implementation choices are specific. The optimizer is AdamW with $c$8, learning rate $c$9, cosine schedule, warmup of $x_{<i}=(x_1,\dots,x_{i-1})$^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ steps, and gradient clipping at $x_{<i}=(x_1,\dots,x_{i-1})$^^^^4speculative decoding Leviathan acceptance total variation4^^^^. The primary regime is stochastic decoding with $x_{<i}=(x_1,\dots,x_{i-1})$^^^^4(Samarin et al., 27 Feb 2026)4^^^^, with additional evaluation at $x_{<i}=(x_1,\dots,x_{i-1})$^^^^4\4^^^^ for greedy decoding. Aggregation decay is set to $x_{<i}=(x_1,\dots,x_{i-1})$4 to prioritize early heads. The paper describes $x_{<i}=(x_1,\dots,x_{i-1})$5 with $x_{<i}=(x_1,\dots,x_{i-1})$6 as the strongest default, while noting that MEDUSA benefits from larger $x_{<i}=(x_1,\dots,x_{i-1})$7, for example $x_{<i}=(x_1,\dots,x_{i-1})$8, because its acceptance improves more slowly. Fixed-weight blends, such as $x_{<i}=(x_1,\dots,x_{i-1})$9, are reported to underperform adaptive scheduling ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

A practical implementation point concerns truncated draft vocabularies, as in FR-Spec or EAGLE-^^^^4\4^^^^ heads. For KL training, $q$^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ can become infinite when $q$^^^^4speculative decoding Leviathan acceptance total variation4^^^^ but $q$^^^^4(Samarin et al., 27 Feb 2026)4^^^^, so a standard workaround masks target logits and replaces $q$^^^^4\4^^^^ with

$q$4

LK losses avoid this approximation. When a token lies outside the draft vocabulary, $q$5 and $q$6, so it simply contributes nothing to $q$7 or TV. The paper presents this as a practical advantage over KL-based training ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

Another implementation issue is evaluation at nonzero temperature. The authors state that at $q$8 they patched vLLM to implement correct rejection sampling of draft tokens. Otherwise, greedy draft sampling yields

$q$9

which underestimates acceptance at high temperature. This point addresses a measurement pitfall rather than a change in the training objective itself ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

5. Empirical performance across architectures, targets, and domains

The empirical study spans six target models—LLaMA-^^^^4\4^^^^.^^^^4speculative decoding Leviathan acceptance total variation4^^^^-8B-Instruct, LLaMA-^^^^4\4^^^^.^^^^4\4 speculative decoding LK losses Leviathan4^^^^B-Instruct, GPT-OSS-^^^^4(Samarin et al., 27 Feb 2026)4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^B, GPT-OSS-^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^B, Qwen^^^^4\4^^^^-^^^^4(Samarin et al., 27 Feb 2026)4\4 and DeepSeek-V^^^^4\4^^^^—and four draft architectures: EAGLE-^^^^4\4^^^^, MEDUSA, a multi-stage MLP speculator, and DeepSeek-V^^^^4\4^^^^ MTP. Training uses PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ prompts from Infinity-Instruct-^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^6^^^^4(Samarin et al., 27 Feb 2026)4^^^^5 with target responses generated by each target model. Evaluation covers MT-bench, HumanEval, and GSM8K under chain sampling at PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4speculative decoding Leviathan acceptance total variation4^^^^ and PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4(Samarin et al., 27 Feb 2026)4^^^^ ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The principal reported metric is average acceptance length PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4\4^^^^. Throughput or speedup is not directly measured; instead, PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan44^^^^ is treated as the driver of speed. Across all configurations, LK losses improve acceptance metrics relative to KL-based training. The paper’s selected highlights include the following stochastic-decoding results at PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^5:

• For LLaMA-^^^^4\4^^^^.^^^^4speculative decoding Leviathan acceptance total variation4^^^^-8B with EAGLE-^^^^4\4^^^^, the mean PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^6 across MT-bench, HumanEval, and GSM8K rises from PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^7 under KL to PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^8 under PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^9, a PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ gain; PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4^^^^ reaches PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4^^^^.
• For LLaMA-^^^^4\4^^^^.^^^^4\4 speculative decoding LK losses Leviathan4^^^^B with EAGLE-^^^^4\4^^^^, PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4\4^^^^ rises from PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation44^^^^ to PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4^^^^5, a PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4^^^^6 gain.
• For GPT-OSS-^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^B with EAGLE-^^^^4\4^^^^, PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4^^^^7 rises from PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4^^^^8 to PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4speculative decoding Leviathan acceptance total variation4^^^^9, a PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^ gain.
• For Qwen^^^^4\4^^^^-^^^^4(Samarin et al., 27 Feb 2026)4\4 with EAGLE-^^^^4\4^^^^, PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4speculative decoding Leviathan acceptance total variation4^^^^ rises from PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4(Samarin et al., 27 Feb 2026)4^^^^ to PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4\4^^^^, a PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)44^^^^ gain.
• For DeepSeek-V^^^^4\4^^^^-685B with fine-tuned MTP, PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4^^^^5 rises from PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4^^^^6 to PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4^^^^7, a PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4^^^^8 gain ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The gains are larger for smaller drafts. Under stochastic sampling, MEDUSA and the MLP speculator show average improvements of roughly PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4(Samarin et al., 27 Feb 2026)4^^^^9 and PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^, respectively, whereas EAGLE-^^^^4\4^^^^ shows PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4speculative decoding Leviathan acceptance total variation4^^^^. This supports the paper’s central thesis that direct optimization matters most when draft capacity is limited relative to the target ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The ablations clarify the role of each design choice. On LLaMA-^^^^4\4^^^^.^^^^4speculative decoding Leviathan acceptance total variation4^^^^-8B with EAGLE-^^^^4\4^^^^, pure TV substantially underperforms; for example, at PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4(Samarin et al., 27 Feb 2026)4^^^^ on MT-bench, TV yields PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4\4^^^^ versus PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\44^^^^ for KL. A fixed-weight hybrid with PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4^^^^5 also underperforms the adaptive schedule. PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4^^^^6 is generally stronger than KL, and PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4^^^^7 is reported as the best setting overall ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

6. Practical interpretation, limitations, and relation to prior work

The paper situates LK losses against the background of KL-based knowledge distillation, which is standard for MEDUSA, EAGLE, and MTP-like heads. In that literature, draft training aligns distributions but only indirectly affects verifier acceptance. LK losses replace that indirect objective with losses that target the overlap quantity PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4^^^^8 explicitly ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The most direct practical guidance concerns where the method helps. The reported gains are strongest for low-capacity drafts such as MEDUSA and MLP, for large target models including MoE targets and models in the PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4\4^^^^9–PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^–PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44speculative decoding Leviathan acceptance total variation4^^^^ range, and for mismatched regimes such as pretrained MTP modules trained only for early positions. Truncated vocabularies are another favorable regime because LK objectives handle them without masking the target distribution ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The main limitations are optimization-related rather than conceptual. Pure TV from random initialization suffers from vanishing gradients and should be avoided unless PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44(Samarin et al., 27 Feb 2026)4^^^^ and the vocabulary is small. Over-rapid decay of PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44\4^^^^ can move training into a non-smooth TV-dominated regime prematurely, whereas insufficient decay preserves too much KL guidance and leaves acceptance suboptimal. The paper recommends monitoring per-head PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation444^^^^ during training: if early stagnation appears, increasing PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44^^^^5 or mixing in PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44^^^^6 is suggested. Later heads often have lower acceptance, and the adaptive schedule responds by increasing KL guidance there ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

The relation to inference-side accelerations is complementary rather than competitive. The paper evaluates only chain sampling to isolate the training objective’s contribution, but it argues that gains in per-position PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44^^^^7 should transfer to blockwise or tree-style speculative schemes because their acceptance is governed by the same overlap quantity. This suggests that LK losses are best understood as a training-layer modification for speculative systems, not as an alternative decoding algorithm ^^^^^^^^4(Samarin et al., 27 Feb 2026)4

A final point of interpretation follows from the paper’s “no computational overhead” claim. LK losses use the same target and draft tensors as KL training. The additional operations—PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44^^^^8, absolute differences for TV, and the scalar PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation44^^^^9—are PRESERVED_PLACEHOLDER_^^^^4speculative decoding Leviathan acceptance total variation4^^^^5^^^^4(Samarin et al., 27 Feb 2026) speculative decoding LK losses Leviathan4^^^^, like KL, and require no extra forward passes, caches, or sampling steps. The paper therefore presents the method as a direct objective replacement: acceptance is optimized more faithfully, especially in capacity-limited regimes, while retaining the training footprint of existing speculative distillation pipelines ^^^^^^^^4(Samarin et al., 27 Feb 2026)4