---
title: 'TAIS: Invariance Screen for Speculative Decoding'
url: https://www.emergentmind.com/topics/typical-acceptance-invariance-screen-tais
type: topic
---

# TAIS: Invariance Screen for Speculative Decoding

Searching arXiv for the specified paper to ground the article in the cited source.
Typical-Acceptance Invariance Screen (TAIS) is a behavioural-equivalence protocol for greedy speculative decoding at temperature zero. It is designed to test whether, on a fixed battery of safety and capability tasks, the refusal or safety-pass rates produced by a speculative-decoding stack are statistically indistinguishable—within a pre-specified tolerance—from those produced by target-only greedy decoding. In the formulation introduced in "Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion," TAIS operationalizes this objective by pairing target-only and speculative outputs on identical prompts and evaluating three invariance criteria per task: byte-identity, TOST equivalence at $\pm 3$ percentage points, and a small-effect cutoff on Cohen’s $h$ [2606.25097].

## 1. Definition and acceptance criterion

TAIS is defined as a behavioural-equivalence screen rather than as a general safety guarantee. Its immediate objective is narrow and explicit: to certify, for greedy decoding at $T=0$, that speculative decoding does not induce detectable divergence in safety-scored outputs relative to target-only decoding on the tested task battery. The protocol therefore compares matched prompt-level outputs from two execution modes—target-only and speculative—under the same evaluation conditions.

A stack “passes TAIS” only if every well-conditioned per-task contrast meets all three criteria. This requirement is conjunctive rather than aggregate: byte-level agreement, statistical equivalence, and bounded effect size are all required for each eligible task-level comparison. The protocol is therefore scoped to invariance evidence on a defined benchmark set, not to model safety in a broader sense.

The name reflects the inclusion of probabilistic “typical acceptance” among the tested serving modes, but the screen itself is not limited to that policy. In the reported evaluation, the serving stack uses vLLM v0.19 with both rejection sampling and probabilistic “typical acceptance” modes under greedy temperature-zero execution.

## 2. Formal criteria

For each task $t$, TAIS compares the target-only safety-pass rate $p_{\text{target}}^{(t)}$ and the speculative-decoding safety-pass rate $p_{\text{spec}}^{(t)}$, with tolerance $\Delta = 0.03$. The protocol applies three criteria [2606.25097].

First, **Criterion C1: byte-identity**. At $T=0$, TAIS compares the raw output bytes of target-only and speculative decoding on each prompt. The comparison is labeled “strong” if at least $99.5\%$ of byte positions match; otherwise it is labeled “moderate” if Criteria C2 and C3 still pass. This criterion addresses prompt-level output identity directly, rather than only downstream agreement in task-level labels.

Second, **Criterion C2: Two One-Sided Tests (TOST) at $\pm \Delta$**. TAIS rejects both null hypotheses
$$
H_{0,\text{lower}}: p_{\text{spec}} - p_{\text{target}} \le -\Delta
\quad \text{versus} \quad
H_{1,\text{lower}}: p_{\text{spec}} - p_{\text{target}} > -\Delta
$$
and
$$
H_{0,\text{upper}}: p_{\text{spec}} - p_{\text{target}} \ge +\Delta
\quad \text{versus} \quad
H_{1,\text{upper}}: p_{\text{spec}} - p_{\text{target}} < +\Delta
$$
at significance level $\alpha$ after Holm–Bonferroni correction across tasks. Passing both one-sided tests establishes equivalence in the sense that $|p_{\text{spec}} - p_{\text{target}}| \le \Delta$.

Third, **Criterion C3: Cohen’s $h$ effect-size cutoff**. TAIS defines
$$
h_t = 2\arcsin\!\left(\sqrt{p_{\text{spec}}^{(t)}}\right) - 2\arcsin\!\left(\sqrt{p_{\text{target}}^{(t)}}\right)
$$
and requires
$$
|h_t| < h^\ast \quad \text{with} \quad h^\ast = 0.1.
$$
The reported cutoff $h^\ast = 0.1$ was post-hoc calibrated so that it sits approximately four times above the largest observed null-consistent $|h| = 0.024$ and remains below Cohen’s conventional “trivial effect” floor of $0.2$. The paper states that future users should re-calibrate on held-out data or conservatively adopt $h^\ast = 0.2$.

Taken together, these criteria combine exact-output evidence, equivalence testing, and effect-size control. This suggests that TAIS is intended to screen for both visible output perturbations and smaller rate-level deviations that might survive byte mismatch.

## 3. Evaluation design and experimental scope

The reported evaluation has four main components: a confirmatory core, a matched expansion, a one-arm production-scale probe, and a fixed serving and scoring stack [2606.25097].

The **confirmatory core (E0)** contains **16,783 paired samples** across three model-pair families: **Llama-3.2-3B+1B**, **Qwen-2.5-3B+1.5B**, and **Qwen-2.5-1.5B+0.5B**. It covers four safety tasks—**AdvBench refusal**, **jailbreak amplification**, **BBQ bias**, and **TruthfulQA**—plus two capability controls, **MMLU** and **ARC-Challenge**.

The **matched expansion** adds **44,066 paired samples** through four orthogonal stress tests:
- **E2**: DPO-adversarial 1B draft (**4,006 samples**)
- **E3**: GPTQ-4bit 1B draft (**4,006 samples**)
- **E4**: seed replication with seeds 123 and 456 (**24,036 samples**)
- **E5**: accumulator dtype swap to bf16 (**12,018 samples**)

The **one-arm production-scale probe (E1)** consists of **4,006 samples** on **Llama-3.1-70B target + 8B draft**, but it lacks a matched target-only arm.

The **serving stack** is **vLLM v0.19**, run with **greedy temperature zero**, under both **rejection sampling** and **probabilistic “typical acceptance”** modes, with **fp16** accumulator in the core and E2–E4, and **bf16** in E5. **Safety scoring** uses **deterministic regex classifiers** for safety tasks; all expansion-cell conclusions use regex labels only. An **LLM judge (Gemma 3 12B)** provides a cross-check on the core.

This design is explicitly structured around matched comparisons. The one-arm 70B probe is therefore informative but not sufficient for a TAIS pass, because the protocol’s central logic depends on paired target-only versus speculative contrasts.

## 4. Empirical findings

Within the matched evaluations, the paper reports no detectable safety divergence under TAIS for the tested temperature-zero vLLM stacks [2606.25097]. The most prominent summary statistic is that the **maximum observed $|h|$ across all matched AdvBench-refusal contrasts (E0+E2–E5) is 0.024**, described as roughly an order of magnitude below the conventional trivial-effect floor.

For **TOST**, the **core per-task ledger** comprises **27 contrasts across six tasks**. Of these, **25 pass at $\pm 3$ percentage points**. The **two non-passes** occur **only on capability tasks (MMLU, ARC) at rate = 1.0 boundary (Wald-CI degenerate)** and are **flagged but not counted as failures**. In the expansion, **all 17 matched AdvBench-refusal contrasts** pass TOST at $\pm 3$ percentage points with **Holm–Bonferroni–adjusted $\alpha = 0.0045$**. For **E5 (bf16)**, **all nine per-task contrasts pass TOST at $\pm 3$ percentage points**, with **max $|h| = 0.054$**.

For **byte-identity**, the reported outcomes differ by stress condition. **E2 (DPO-adversarial draft vs canonical)** yields **100.00% byte-identity on 4,006 samples**. **E3 (GPTQ-4bit)** and **E4 (seed replication)** likewise yield **100.00% identity**. By contrast, **E5 (bf16)** yields only **36–53% identity per pair**, yet all C2 and C3 comparisons still pass.

The paper also reports that the **DPO-adversarial draft produces byte-identical output to the canonical draft across 4,006 samples**, and that **bf16 changes 36%–53% of output bytes without moving any per-task safety rate outside equivalence**. These two observations jointly separate exact string identity from task-level invariance: one can observe full byte agreement under some perturbations, and substantial byte change under others, while still retaining equivalence at the level of measured refusal or safety-pass rates.

For the **70B production probe (E1)**, which is not counted as a TAIS pass, the paper reports **AdvBench refusal = 0.839 over 700 AdvBench completions with 95% Wilson CI [0.809, 0.864]**. It also reports that the **dose-response over speculation lengths $\{1,3,5,8,12\}$ is flat**, with **slope $\approx 0.000 \pm 0.001$**.

## 5. Interpretation and methodological significance

TAIS is best understood as a scoped invariance screen for speculative decoding rather than as a universal safety benchmark. Its central methodological contribution is the combination of prompt-level raw-byte comparison with task-level equivalence testing and effect-size control. This produces a three-layer acceptance logic: exact output comparison, bounded rate difference, and bounded standardized effect.

The reported findings support a narrow conclusion. Within the explicitly stated regime—**$T=0$**, **vLLM v0.19**, **Llama 3.x and Qwen 2.5 families**, **six tasks**, and **rejection sampling or typical acceptance**—the paper states that TAIS finds **no detectable safety divergence between target-only and speculative stacks**. The summary given in the paper is that **max $|h| = 0.024$**, **all qualifying TOST contrasts pass at $\pm 3$ percentage points**, **byte-identity is at least 99.5% except when modulated by dtype**, and the **70B probe** yields a **plausible refusal rate of 0.839 [0.809, 0.864]** [2606.25097].

The reported non-pass cases are also methodologically important. They occur only in capability-domain contrasts at a boundary condition with identical ceiling rates and degenerate Wald confidence intervals. The paper characterizes these as **edge cases**, not as evidence of genuine non-equivalence. This matters because TAIS is framed as an equivalence procedure: failure to establish equivalence under such conditions is not automatically evidence of difference.

A plausible implication is that TAIS is intended to distinguish between three situations that are often conflated: exact deterministic reproducibility, statistical equivalence in downstream safety rates, and broader claims about safety preservation under deployment. The protocol addresses the first two directly and leaves the third outside scope.

## 6. Limitations, non-claims, and boundary conditions

The paper defines its limitations explicitly and repeatedly restricts the scope of its conclusions [2606.25097]. **Temperature** is the foremost boundary: TAIS applies only to **$T=0$ greedy decoding**; behavior at **$T>0$** is untested. **Framework coverage** is limited to **vLLM v0.19**; **TGI**, **TensorRT-LLM**, and **SGLang** are untested. **Model coverage** is limited to the **Llama 3.x** and **Qwen 2.5** families.

The tested **speculation policies** are limited to **strict rejection sampling** and **probabilistic typical acceptance**. The paper does not make claims about **tree-speculation variants** such as **EAGLE**, **Medusa**, and **SpecInfer**, nor about **retrieval-based drafting** or **lookahead/parallel schemes**. The task battery is also limited: **single-turn refusal, bias, and truthfulness tasks**—**AdvBench**, **jailbreak**, **BBQ**, and **TruthfulQA**—plus the capability controls **MMLU** and **ARC-Challenge**. **Multi-turn**, **agentic**, and **red-teaming benchmarks** are outside scope.

The statistical limits are also stated. The paper reports **per-cell MDE $\approx 7.4$–$8.3$ percentage points** in the core and **pooled $\approx 4.3$ percentage points**. It further notes that **high-ceiling cells ($p \approx 0.97$–$0.98$) have weaker power**. Accordingly, a failure to detect divergence outside these sensitivity limits would not be ruled out by the reported experiments.

Finally, the paper highlights a **calibration caveat** for the effect-size threshold: **$h^\ast = 0.1$** was **post-hoc tuned on the same expansion data**, although a held-out check preserved the cutoff structure within **3% of 0.1**. The recommendation is that **independent recalibration** be used for new regimes. This limitation is central to interpretation: TAIS is presented as a transferable screening framework, but not as a fixed thresholding rule that can be applied unchanged across all model, framework, or benchmark settings.

Source: https://www.emergentmind.com/topics/typical-acceptance-invariance-screen-tais