Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion

Abstract: Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at temperature zero, can draft-side behavior leak into safety-scored outputs? We answer with Typical-Acceptance Invariance Screen (TAIS), a behavioral-equivalence screen that pairs target-only and speculative outputs on the same safety battery and requires byte-identity evidence, TOST equivalence at +/-3pp, and per-task Cohen's h below a calibrated null cutoff of |h| < 0.1. Applied to a 16,783-sample confirmatory core plus 44,066 matched expansion samples (fp16/bf16 execution, canonical and DPO-adversarial drafts, GPTQ-4bit drafts, two seeds, and four safety benchmarks), the tested temperature-zero vLLM stacks show no detectable safety divergence under TAIS. The largest absolute Cohen's h on matched target-only versus speculative refusal is 0.024, roughly an order of magnitude below the conventional trivial-effect floor; 25 of 27 per-task TOST contrasts pass at the +/-3pp margin (the two non-pass contrasts are capability-domain Wald-CI edge cases at identical ceiling rates, not genuine non-equivalence); the DPO-adversarial draft produces byte-identical output to the canonical draft across 4,006 samples; and bf16 changes 36%-53% of output bytes without moving any per-task safety rate outside equivalence. A separate 4,006-sample 70B production-scale probe, which lacks a matched 70B target-only arm and is therefore not counted as a TAIS pass, produces AdvBench refusal 0.839 over 700 AdvBench completions with 95% Wilson CI [0.809, 0.864]. We make no claim about sampling temperatures, untested frameworks, untested model families, or tree-speculation variants such as EAGLE and Medusa.

• The paper demonstrates that speculative decoding at T=0 maintains byte-level identical outputs under diverse experimental conditions, ensuring no safety divergence.
• It introduces the TAIS protocol, which rigorously tests paired outputs using byte identity, TOST equivalence, and calibrated Cohen’s h thresholds.
• Empirical results across over 48,000 samples, including adversarial conditions, confirm that draft misalignment does not compromise safety in T=0 scenarios.

Safety and Behavioral Invariance of Speculative Decoding at Temperature Zero

Introduction and Context

This paper investigates the safety invariance of speculative decoding at temperature zero ($T=0$) in LLM inference pipelines. Speculative decoding accelerates LLM generation by allowing a smaller draft model to propose tokens, which a larger target model then verifies before committing these to output. While the mathematical guarantee of output equivalence under strict greedy (i.e., $T=0$) rejection sampling has been established theoretically, the possibility of a behavioral safety leak—namely, the risk that a misaligned or adversarial draft could alter the safety properties of the verified output—has not been systematically audited at scale. This study provides a large-sample, empirical safety-invariance screen across multiple axes: model families, serving configurations, draft perturbations, and safety benchmarks.

Experimental Protocol: Typical-Acceptance Invariance Screen (TAIS)

The core experimental protocol, the Typical-Acceptance Invariance Screen (TAIS), is defined as a multi-criteria paired behavioral equivalence test for speculative decoding stacks. TAIS requires:

1. Byte-for-Byte Output Identity: A strong null is indicated by $\geq 99.5\%$ byte identity between target-only and speculative outputs on paired prompts.
2. TOST Equivalence: Two One-Sided Tests (TOST) for binomial safety rates on each task, with a $\pm 3$ percentage points equivalence bound. Passing requires both one-sided intervals to reject non-equivalence.
3. Cohen’s $h$ Effect Size: For each task, absolute Cohen’s $h$ must fall below a calibrated threshold $h^\star=0.1$. This margin is set both above the maximum null-consistent contrast observed in extensive expansion experiments (max $|h|=0.024$) and well below the trivial-effect floor of 0.2.

TAIS is systematically applied to a factorial design spanning model families (Llama, Qwen), acceptance policies (strict rejection sampling, typical acceptance), multiple draft-target pairs, quantizations, draft perturbations (including explicit adversarial DPO fine-tuning), seeds, and accumulator precision (fp16, bf16). The aggregate evidence consists of a 48,072-sample expansion atop a large-sample main core, plus an additional 70B parameter production-scale probe.

Empirical Results

Null Results Across All Safety Benchmarks

The empirical results show no detectable safety divergence between speculative and target-only decoding under TAIS, across all tested configurations at $T=0$. The strongest observed absolute Cohen’s $h$ on critical safety contrasts is 0.024, an order of magnitude below conventional triviality thresholds. Out of 27 per-task TOST contrasts on the main core, 25 pass; the two non-passing cases are exact-match ceiling effects on capability tasks (not safety tasks), with both arms at perfect scores.

Adversarial Draft and Quantization Robustness

A particularly stringent test involves substituting the draft with a DPO adversarial variant, trained to prefer harmful completions. In this condition, byte-identical outputs are observed on all 4,006 paired samples. In other words, even with a strongly misaligned adversarial draft, the verified output of the target at $T=0$0 is empirically invariant up to the limits of floating-point non-associativity. This result is consistent across the core, fp16/bf16 accumulator types, quantized drafts (GPTQ-4bit), and seed replications.

Production-Scale Expansion and Pooling

A large-scale probe at the 70B-parameter model level confirms output metrics in range; however, as no target-only baseline was run at 70B, this is reported solely as a plausibility sanity check and officially excluded from TAIS pass/fail judgments.

Robustness and Statistical Power

Per-cell minimum detectable effects (MDE) are in the 7.4–8.3pp range; pooled analyses tighten this to about 4.3pp. Matched expansion across all configurations yields consistent null results. The “moderate” null TAIS flag correctly triggers when numerical accumulator changes (fp16 to bf16) shift output bytes without affecting safety rates.

Theoretical Justification and Limits

The theoretical basis for the null result derives from the properties of $T=0$1 speculative decoding under greedy or strict acceptance: the verified output must match the target distribution, and the only variations arise from floating-point artifacts uncorrelated to classification boundaries. Importantly, at higher temperature, with stochasticity in sampling, these distributional guarantees collapse and the attack surface likely increases; this is out of scope for the present study.

The empirical evidence decisively supports the claim that, within the studied scope—$T=0$2, vLLM v0.19, Llama and Qwen models, standard and adversarial drafts, rejection/typical acceptance, canonical safety and capability benchmarks—speculative decoding cannot be distinguished from target-only decoding by any of byte identity, TOST, or per-task effect size.

Practical and Theoretical Implications

Practically, the null result substantially reduces audit burden for deployments of $T=0$3 speculative decoding: under TAIS, draft selection can be driven by throughput and latency considerations without necessitating an independent behavioral safety audit, provided that model, temperature, and framework remain within the studied boundaries.

Theoretically, the results operationalize and validate prior distributional guarantees from the literature [Leviathan et al., 2023; Chen et al., 2023], confirming their real-world relevance in practical inference stacks. The introduction of TAIS—a protocol with explicit, calibrated thresholds—provides a reusable tool for both vendors and independent evaluators.

Caution is warranted in extrapolation: the results do not extend to $T=0$4 (temperature-sampled) decoding, other frameworks, multi-turn/agentic settings, or tree-based speculative variants (e.g., EAGLE, Medusa). The authors recommend re-running TAIS (with appropriate recalibration of $T=0$5) whenever leaving this regime.

Pathways for Future Research

The primary direction for future work is to invert each of the current study’s scope boundaries: relaxing the temperature constraint, exploring cross-framework robustness, extending to tree speculation, and incorporating multi-turn dialogues or agentic tasks. A further imperative is establishing held-out calibration sets for TAIS or standardizing effect-size reporting across speculative-decoding deployments to enable cross-benchmark/method comparison.

Conclusion

This comprehensive, factorial study demonstrates—across $T=0$6 samples, two model families, multiple draft perturbation regimes, and six safety/capability benchmarks—that speculative decoding at $T=0$7 is behaviorally invariant to draft alignment when the target’s verification is greedy. Maximum observed safety effect sizes are negligible ($T=0$8); the strongest adversarial construction tested has zero byte impact. The TAIS protocol operationalizes these findings and is recommended as a standard behavioral equivalence screen for $T=0$9 speculative decoding. The results provide strong evidence that, for temperature-zero inference pipelines under standard system implementations, draft alignment does not constitute a behavioral safety channel. Conservative adoption of TAIS is warranted when considering deployment outside these boundaries.

Reference: "Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion" (2606.25097).