Prefill-Time Intervention (PTI) Overview

• PTI is a prefill-stage intervention that modifies the initial KV cache to improve model grounding and computation efficiency in large-scale models.
• It enhances time-to-first-token by intervening once during prefill, making it orthogonal to decode-stage methods and adaptable to various serving scenarios.
• Empirical evaluations show that PTI reduces TTFT, boosts throughput, and mitigates issues like hallucinations and scheduling challenges in vision-language models and LLMs.

Searching arXiv for the supplied PTI-related papers to ground the article in current literature. Prefill-Time Intervention (PTI) denotes intervention performed during the prefill stage, before autoregressive decoding begins. In the strict sense introduced for large vision-LLMs (LVLMs), PTI is a training-free, plug-and-play steering paradigm that intervenes only once during prefill by modifying the initial Key-Value (KV) cache so that decoding starts from a better-grounded multimodal state (Zhang et al., 28 Apr 2026). In a broader systems sense used by several later LLM-serving papers, the same idea appears as a PTI-like design pattern: the intervention is applied to prefill scheduling, routing, prompt processing, or adapter application rather than to model weights or the decode algorithm, with the aim of improving time-to-first-token (TTFT), preserving decode-side quality-of-service, or reducing prefill-induced inefficiencies (Lee et al., 9 Oct 2025, Arun et al., 2 Jul 2026). This suggests that PTI is best understood as a stage-specific control principle centered on prefill as the leverage point for changing either model behavior or serving behavior.

1. Prefill as the decisive control point

Prefill and decode occupy sharply different roles in contemporary inference stacks. In standard autoregressive LLM serving, prefill processes the whole prompt to build the KV cache and produce the first output token, whereas decode generates one token per iteration thereafter. Multiple papers characterize prefill as compute-heavy or compute-bound, and decode as memory-bound, latency-sensitive, or both; this asymmetry is the core reason PTI is technically attractive (Lee et al., 9 Oct 2025, Liu et al., 5 Feb 2025, Hsieh et al., 18 Feb 2026). In LVLMs, the analogous claim is representational rather than purely systems-oriented: the prefill stage computes the initial multimodal KV cache, and hallucinations can be seeded there before any output token is generated (Zhang et al., 28 Apr 2026).

This asymmetry also determines the relevant service-level objectives. Serving papers repeatedly distinguish TTFT from time-between-token (TBT) or time-per-output-token (TPOT). One paper formalizes stall-free scheduling as the state where, “for each request, every TBT does not exceed a model-specific threshold” (Lee et al., 9 Oct 2025). Another defines goodput as the maximum sustainable request rate at 90% SLO attainment (Hsieh et al., 18 Feb 2026). A third shows that aggregation and disaggregation dominate in different SLO regimes, so prefill-side control becomes a first-class scheduling lever rather than a secondary optimization (Wang et al., 4 Aug 2025).

On the model side, the motivation is temporal and causal. The LVLM PTI paper argues that decoding-stage steering intervenes too late: if the initial multimodal interpretation formed during prefill is already wrong, later token-by-token steering is trying to correct an error after it has already entered the generation trajectory. The paper describes this as a “snowball hallucination” effect, in which residual hallucinations become more severe during autoregressive decoding because each incorrect token influences later context (Zhang et al., 28 Apr 2026). A plausible implication is that prefill is attractive not only because it is earlier, but because it is the stage at which both computational structure and initial semantic grounding are established.

2. KV-cache steering and modality-aware PTI

The canonical PTI formulation modifies the initial layer-wise KV caches computed during prefill. The prefill computation is written as

$K' = X'W_K, \quad V' = X'W_V.$

During decoding, new token key/value vectors are concatenated to the cache:

$$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$

PTI intervenes on the initial layer-wise caches $K^l, V^l$ computed during prefill, rather than repeatedly altering hidden states during generation (Zhang et al., 28 Apr 2026).

A central feature of this formulation is modality awareness. Rather than deriving a single steering vector, PTI extracts separate visual and textual directions from contrastive prefills. For visual direction extraction, positive samples are object-only images and negative samples are background-only images; the contrastive direction is

$$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$

The final visual directions are

$$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$

For the text side, the paper contrasts anchor words against non-anchor words or remaining caption context, and computes

$$\Delta C^i_l = (C^{pos} - C^{neg})[N_x - 1], \quad C \in \{K,V\},$$

with averaged textual directions

$$s_{k,\mathrm{txt}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{txt}} = \frac{1}{N}\sum_i \Delta V^i_l.$$

The downstream intervention is then applied separately to keys and values:

$$K'[\mathrm{img}] += A_{k,\mathrm{img}}\, s_{k,\mathrm{img}}, \quad V'[\mathrm{img}] += A_{v,\mathrm{img}}\, s_{v,\mathrm{img}}$$

and

$$K'[x] += A_{k,\mathrm{txt}}\, s_{k,\mathrm{txt}}, \quad V'[x] += A_{v,\mathrm{txt}}\, s_{v,\mathrm{txt}}.$$

The paper emphasizes that keys determine where to attend, while values determine what information gets aggregated; key steering is used to preserve and strengthen object-centric attention, and value steering is used to reduce the influence of background-only or non-object information (Zhang et al., 28 Apr 2026).

Experimentally, this PTI formulation is evaluated on LLaVA-1.5, Qwen-VL-Chat, and DeepSeek-VL-Chat under greedy decoding, beam search with beam size 5, and nucleus sampling with top-p = 1.0, using CHAIR, POPE, AMBER, MMHAL-Bench, and MME. On LLaVA-1.5 with greedy decoding, CHAIRS/CHAIRI move from 47.4 / 13.7 for Vanilla to 15.4 / 5.4 for PTI. The paper further states that PTI is orthogonal to decoding-stage methods such as PAI or VISTA, enabling plug-and-play integration, and that it has near-baseline latency and throughput because it modifies the cache only once (Zhang et al., 28 Apr 2026).

3. Token pruning, block sparsification, and prefill-only adaptation

Several later methods instantiate PTI by changing which prompt information survives prefill, rather than directly steering the KV cache. Speculative Prefill (SpecPrefill) uses a lightweight speculator model to estimate token importance from attention, keeps only the most important tokens in contiguous chunks, restores original positional indices, and feeds the reduced prompt to the main LLM. The method is training-free; its core assumption is that token importance transfers across model sizes within the same family. The paper defines attention scores from decoded token $j$ to prompt token $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$0 as

$$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$1

aggregates attention over heads, layers, and look-ahead tokens, applies 1D average pooling, groups context into contiguous chunks, and selects Top-K chunks. Preserving original position IDs is described as crucial; if tokens $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$2 are kept, the positions sent to the main model remain $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$3, and decoding positions continue from the original context length. The paper reports up to $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$4 maximal end-to-end QPS on real downstream tasks and up to $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$5 TTFT improvement on Llama-3.1-405B-Instruct-FP8, with the measured TTFT speedup close to the analyzed $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$6 (Liu et al., 5 Feb 2025).

UniPrefill shifts from attention sparsification to token survival. It estimates token importance from the last $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$7 query positions at a full-attention layer, groups tokens into blocks of size $$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$8, retains the minimum set of highest-scoring blocks whose cumulative attention mass reaches a top-$$K^{l} + [K^{l}, k^{txt}_{t}], \quad V^{l} + [V^{l}, v^{txt}_{t}].$$9 threshold, and then excludes dropped tokens from all subsequent layers in the same block. Tokens are partitioned as

$K^l, V^l$0

and the retained set is the smallest prefix of blocks reaching the threshold $K^l, V^l$1. Two token classes are always kept: the first $K^l, V^l$2 tokens, called attention sinks, and the last $K^l, V^l$3 tokens, i.e. the query window itself. Because sparsity propagates through later sublayers, the paper argues that UniPrefill saves both attention FLOPs and GEMM FLOPs, and is therefore applicable to full-attention models, linear/full hybrids, and sliding-window/full hybrids. The abstract reports up to 2.1x TTFT speedup, while the detailed table shows 2.26x on LLaMA-3.1-8B-Instruct at 128K context; the same evaluation reports 1.68x on Qwen3-Next-80B-A3B and 1.49x on Gemma-3-12B, with throughput improvements up to +109%, +68%, and +42% respectively (Fan et al., 7 May 2026).

PreFT (“prefill-only finetuning”) restricts adapter application to prompt positions. The general intervention is

$K^l, V^l$4

and PreFT sets

$K^l, V^l$5

where $K^l, V^l$6 is the prompt length, whereas standard all-position adapters use $K^l, V^l$7. The paper instantiates this idea for LoRA and DiReFT, yielding LoRAP and DiReFTP. Its systems motivation is explicit: prefill is compute-bound and can absorb extra adapter computation, while decode is memory-bound and highly sensitive to per-request adapter overhead. On Llama 3.1 70B with 512 adapters and 4×H100 tensor parallelism, prefill-only ReFT achieves 1.9× the throughput of multi-LoRA and prefill-only LoRA achieves 1.87×. On supervised finetuning, evaluation loss is higher than with all-position PEFTs but can be compensated by increasing rank with nearly no reduction in throughput; on reinforcement learning, PreFTs often approach parity with standard PEFTs, although GSM8K remains a weak spot (Lanpouthakoun et al., 14 May 2026).

4. Scheduler-level PTI in serving systems

A large branch of PTI-like work treats prefill as a scheduling and routing problem. The baseline motivation is that chunked prefill exists to preserve decode latency: a long prompt is split into smaller token chunks and interleaved with decode so that decode never stalls, but the chunk size creates a tradeoff between TBT protection and efficiency (Lee et al., 9 Oct 2025). Layered prefill redefines the scheduling unit from token chunks to contiguous layer groups. The transformer stack is vertically partitioned into $K^l, V^l$8 contiguous groups with

$K^l, V^l$9

where $$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$0 is prompt length. The central rule is “one-group-per-iteration”: in each iteration, exactly one designated layer group performs both prefill and decode for newly admitted requests, while all other groups perform decode only. This preserves stall-free decoding while eliminating chunk-induced MoE weight reloads. The paper states that layered prefill and chunked prefill are orthogonal and can be combined (Lee et al., 9 Oct 2025).

Kairos studies disaggregated LLM serving, where prefill and decode run on separate GPU pools. Its empirical claim is that in a production-style 2P2D A100 cluster, prefill execution itself accounts for only 2–23% of P95 TTFT; the rest is mostly prefill queueing delay and inter-node GPU-GPU KV-cache transfer. Kairos therefore introduces load-aware prefill deflection: for each queued request, it estimates TTFT on the prefill node and on each decode node under a TBT-safe chunked-prefill schedule, then deflects the request if the decode-side path is sufficiently favorable and respects the TBT SLO. Because the prefill of deflected requests runs in place on the decode node, inter-node KV-cache transfer is eliminated entirely for those requests. The implementation is built on vLLM 0.18.1, uses existing chunked prefill support on decode nodes, and adds a thin scheduling layer at the prefill dispatcher (Arun et al., 2 Jul 2026).

TaiChi unifies prefill-decode aggregation and disaggregation through differentiated-capability instances: P-heavy instances for fast prefill and low TTFT, and D-heavy instances for low-interference decode and low TPOT. Its PTI-like mechanism is length-aware prefill scheduling. For a new request, the scheduler first estimates feasibility under the TTFT SLO and then selects the least-loaded feasible instance, often a D-heavy instance, intentionally degrading the TTFT of requests that can tolerate it in order to preserve P-heavy resources for long or urgent requests. The paper writes TTFT as $$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$1 on D-heavy instances and $$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$2 on P-heavy instances, where $$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$3 is queuing time, $$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$4 is prefill execution time, and $$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$5 is KV-cache transfer time, and uses the feasibility criterion $$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$6. This prefill scheduler is paired with flowing decode scheduling, which migrates decode work between instance types to manage TPOT (Wang et al., 4 Aug 2025).

FlowPrefill addresses head-of-line blocking on the prefill side of PD-disaggregated systems. Its key claim is that chunked prefill couples preemption granularity to scheduling frequency, whereas the needed design is to decouple them. The method combines Operator-Level Preemption, with checks at operator boundaries such as qkv_proj, attn, o_proj, gate_up_proj, and down_proj, and Event-Driven Scheduling, which runs only when a request arrives or completes. Its slack-aware priority is

$$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$7

with

$$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$8

The scheduler sorts requests by priority, performs SLO-aware batching, and can submit new execution, preempt current execution, or resume a preempted task. The paper presents this as a prefill-time intervention that reorders and interrupts ongoing prefill execution to mitigate TTFT head-of-line blocking without sacrificing throughput (Hsieh et al., 18 Feb 2026).

5. Metrics, cost models, and reported performance

Across PTI research, the dominant metrics are TTFT, TBT or TPOT, goodput, SLO attainment, throughput, and, in some systems papers, energy and traffic. One serving paper defines TTFT as time from request arrival to first generated token, TBT as time between subsequent output tokens, and reports mean and p99 TTFT/TBT together with SLO attainment (Lee et al., 9 Oct 2025). Another defines throughput as

$$\Delta C^i_l = AP(C^{pos} - C^{neg})[\mathrm{img}], \quad C \in \{K,V\}.$$9

and emphasizes that prefill-only methods can improve decode-side latency by removing adapter overhead there (Lanpouthakoun et al., 14 May 2026). SpecPrefill explicitly derives prefill FLOPs for MLPs, QKVO projections, self-attention, and LM head, and defines speculation overhead as

$$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$0

This cost model is used to explain why the overhead of an 8B speculator becomes negligible for a 405B base model (Liu et al., 5 Feb 2025).

In MoE serving, layered prefill adds traffic and energy accounting to the PTI picture. Expert weight load traffic is measured as the total number of expert weight load bytes over a fixed trace of 100 requests. On Qwen, the paper reports 28.5 TB for chunked versus 25.1 TB for layered on ShareGPT, a 12.0% reduction, and 35.6 TB versus 21.7 TB on arXiv, a 39.0% reduction. At SLO-compliant operating points on arXiv, energy per token drops from 56.6 to 44.2 mJ/tok for Qwen and from 37.4 to 29.8 mJ/tok for GPT; the paper also reports up to 70% lower TTFT, up to 41% lower end-to-end latency, and per-token energy reductions up to 22%. For Qwen on arXiv at 1.3 req/s, TTFT mean/p99 move from 2.803 s / 8.651 s under chunked prefill to 1.237 s / 4.098 s under layered prefill, while TBT mean/p99 move from 32.9 ms / 51.1 ms to 21.5 ms / 37.1 ms (Lee et al., 9 Oct 2025).

Kairos reports tail-latency gains in the disaggregated setting. Its headline results are P95 TTFT reduced by up to 81%, SLO attainment improved by up to 79% over state-of-the-art disaggregated schedulers, and routing overhead below 1 ms per request. On the bursty trace, it stays within the joint P95 TTFT ≤ 4s and P95 TBT ≤ 70ms SLO up to 9 RPS, while traditional PD disaggregation and TaiChi only manage that up to 7 RPS; Kairos maintains zero TBT violations by construction (Arun et al., 2 Jul 2026).

TaiChi measures performance in goodput space under dual TTFT/TPOT SLOs. The paper reports up to 77% goodput improvement over state-of-the-art systems under balanced TTFT and TPOT SLOs, TTFT reduced by up to $$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$1 relative to PD disaggregation, TPOT reduced by up to $$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$2 relative to PD aggregation, and an SLO-attainment increase from 66.6% to 91.2% in one breakdown experiment after adding hybrid architecture, flowing decode, and length-aware prefill scheduling. The prefill scheduler itself contributes a further 11.7% SLO-attainment improvement after flowing decode scheduling (Wang et al., 4 Aug 2025).

FlowPrefill evaluates PTI in terms of goodput under heterogeneous TTFT SLOs. It reports up to 5.6× higher goodput than DistServe, 4.7×–5.6× higher request rates at the same SLO attainment, 1.5×–2.3× tighter SLOs than DistServe-CP2K, and 2.1×–3.1× tighter SLOs than DistServe-CP8K. Operator-level preemption reduces preemption blocking time by 3.5×–4.2× versus layer-level preemption and keeps observed blocking latencies below 4.5 ms (Hsieh et al., 18 Feb 2026).

6. Orthogonality, limitations, and prefill awareness

PTI is not a single unified framework across the literature. In the LVLM hallucination paper, PTI is a named steering paradigm with a specific intervention target—the initial multimodal KV cache—and a specific design—modality-aware, key/value-decoupled steering (Zhang et al., 28 Apr 2026). In several serving papers, by contrast, PTI is best understood as a PTI-like prefill-time intervention: layered prefill is described as a specific scheduling paradigm and implementation rather than a general PTI framework, Kairos embodies PTI through load-aware prefill deflection, and TaiChi’s PTI flavor is concentrated in length-aware prefill scheduling and latency shifting (Lee et al., 9 Oct 2025, Arun et al., 2 Jul 2026, Wang et al., 4 Aug 2025). A common misconception is therefore to treat PTI as referring only to steering vectors or only to serving schedulers; the surveyed literature supports neither restriction.

A second recurrent theme is orthogonality. The LVLM PTI paper states that PTI is orthogonal to existing decoding-stage methods and can be combined with methods like PAI or VISTA (Zhang et al., 28 Apr 2026). The layered prefill paper similarly states that layered prefill and chunked prefill are orthogonal and can be combined (Lee et al., 9 Oct 2025). This suggests that PTI should often be interpreted not as a wholesale replacement of decoding-time control, but as a prefill-stage control surface that can be composed with decode-stage techniques when the objectives are compatible.

The limitations reported across papers are correspondingly diverse. Steering-based PTI requires tuning intervention strengths; too much visual steering can hurt fluency or F1, and too much steering can shorten outputs (Zhang et al., 28 Apr 2026). SpecPrefill is strongest when prompts are redundant, while short, information-dense tasks and aggregation tasks are less compressible (Liu et al., 5 Feb 2025). UniPrefill depends on the quality of its importance estimate and on the choice of $$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$3, $$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$4, and $$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$5, with strongest gains in long-context, high-concurrency settings (Fan et al., 7 May 2026). PreFT improves throughput substantially, but supervised-finetuning loss is typically higher than for all-position PEFTs at matched parameter count, and DiReFTP exhibits failure modes on LongBench-Write / Long Writer that LoRAP does not (Lanpouthakoun et al., 14 May 2026).

The most direct controversy around PTI is prefill awareness: the possibility that models can recognize that assistant-side context was externally inserted or edited, and then change behavior because of that awareness. “Prefill Awareness in LLMs” defines prefill awareness as the capability to distinguish between assistant-side context naturally produced by the current model and assistant-side context that was externally supplied or modified. The paper separates this into prefill detection, where the model explicitly identifies tampering, and prefill resistance, where its continuation shifts away from the tamper’s trajectory and back toward baseline behavior even without saying the prefill was foreign. On the controlled preference benchmark, Claude Opus 4.5 reaches 67.6% balanced detection accuracy under thinking tampering, 48.6% resistance under thinking tampering, and 0% false positives across mechanisms in the main setting; with stronger prompting it can detect 95% of thinking tampers. The same paper reports that detection and resistance are only weakly correlated overall ($$s_{k,\mathrm{img}} = \frac{1}{N}\sum_i \Delta K^i_l, \quad s_{v,\mathrm{img}} = \frac{1}{N}\sum_i \Delta V^i_l.$$6), that stylistic mismatch mainly affects whether the model flags a prefill as foreign, while preference mismatch mainly affects whether the model reverts toward baseline, and that in misalignment-continuation evaluations Claude Opus 4.5 flags 88% of tampered trajectories (Wang et al., 10 Jun 2026). This establishes prefill awareness as a substantial confound for any PTI method that assumes inserted assistant-side context will be treated as if it were self-authored.

Taken together, the literature presents PTI as a broad research program centered on a single systems and modeling insight: the prefill phase is not merely a preparatory stage, but a high-leverage intervention point. Whether the goal is to correct hallucination-prone multimodal representations, prune or sparsify prompt computation, restrict adapters to prompt positions, eliminate MoE weight-reload pathologies, deflect queued prefills, or preempt head-of-line blocking, the technical move is the same in outline: intervene before decoding, and use the asymmetry between prefill and decode to alter the attainable operating envelope.