KV-Passthrough: Direct KV State Management

• KV-Passthrough is a design paradigm for moving, reusing, and preserving transformer key–value state by bypassing filesystem overhead and traditional cache handling.
• It encompasses two main implementations: an I/O-path mechanism using direct SSD access via SPDK and a reuse operator that enables training-free, position-invariant multimodal cache restoration.
• The approach mitigates latency and memory constraints by coupling efficient retrieval methods with low-rank conditioning patches and controlled resource trade-offs.

KV-Passthrough denotes a family of mechanisms for moving, reusing, or preserving transformer key–value state without re-encoding or incurring the full cost of conventional cache handling. The term is not standardized across the literature. In "Unified KV Pooling to Accelerate Long-Context LLM Serving" (Kang et al., 10 Jun 2026), KV-passthrough is the SSD access fast path that bypasses the Linux kernel filesystem and directly accesses SSD-resident KV caches from user space via SPDK. In "Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse" (Ma et al., 22 Jun 2026), KV-Passthrough is the authors’ name for a training-free, position-invariant multimodal KV-cache reuse operator that combines exact RoPE re-rotation with a low-rank conditioning patch. Other papers describe closely related continuity, recurrence, retention, or bypass behaviors without using the term explicitly, which indicates that KV-Passthrough is better understood as a cross-cutting design pattern than as a single canonical algorithm (Bai et al., 14 Apr 2026, Nadali et al., 12 May 2026, Liu et al., 2 Jun 2026, Liu et al., 13 May 2026, Cai et al., 30 May 2025).

1. Terminological scope and main usages

The literature uses KV-Passthrough in at least two distinct technical senses. The first is an I/O-path sense: KV state already offloaded to storage is retrieved through a direct user-space path rather than a filesystem-mediated file read. This is the meaning used in unified KV pooling, where KV-passthrough is the SSD-side mechanism that removes the filesystem bottleneck for SSD-backed KV retrieval while the KV orchestrator decides placement across host memory and SSD devices (Kang et al., 10 Jun 2026).

The second is a reuse-operator sense: cached KV is stored in a form that can be moved, re-instantiated, or reused under changed positional or contextual conditions without full recomputation. This is the meaning used by Kamera, where KV-Passthrough separates a chunk’s position from the conditioning it absorbed from antecedent context, then reconstructs reuse as exact relocation plus a low-rank conditioning restoration (Ma et al., 22 Jun 2026).

A broader survey of adjacent work shows that several papers instantiate passthrough-like behavior without naming it as such. PipeLive describes a live KV continuity path during pipeline-parallel reconfiguration through non-contiguous KV access, incremental KV patching, and atomic commit after convergence; the paper explicitly states that it does not use the term KV-Passthrough (Bai et al., 14 Apr 2026). KV-Fold passes prior KV forward unchanged as prefix state across chunk boundaries in a recurrent left fold over chunks (Nadali et al., 12 May 2026). TGV-KV defines a text-prioritised retention policy that effectively lets text KV “pass through” aggressive vision-side eviction (Liu et al., 2 Jun 2026). KVServe can effectively bypass compression when compression is non-beneficial, but does not introduce a dedicated passthrough primitive (Liu et al., 13 May 2026). R-KV is a redundancy-aware retention policy rather than a passthrough method in the architectural sense (Cai et al., 30 May 2025).

This distribution of meanings suggests that KV-Passthrough is not a single settled term. A plausible implication is that current usage clusters around three technical objectives: eliminating mediation overhead in the KV data path, preserving KV identity across execution boundaries, and retaining useful KV under strict memory or communication budgets.

2. KV-Passthrough as filesystem-bypassing SSD access

In unified KV pooling, the need for KV-passthrough arises from profiling long-context serving. The paper reports high serving latency of approximately $30.7$ s at $128$K context on the Qwen3-30B-A3B setup, exceeding the typical TTFT requirement of $10$ s by more than $3\times$. It attributes this to two compounding issues: retrieval is serialized through a narrow host-memory/SSD path, and SSD-based KV retrieval spends $84\%$ of its time in the kernel filesystem rather than actual device access (Kang et al., 10 Jun 2026).

KV-passthrough changes the retrieval path from filesystem-mediated file read to direct LBA access from user space. In the conventional path, evicted KV caches go from GPU to host memory and then get written to SSD via the kernel filesystem, which maps them to LBAs; retrieval reverses that path as SSD $\rightarrow$ filesystem $\rightarrow$ host memory $\rightarrow$ GPU. KV-passthrough removes the filesystem middle layer by using SPDK, a user-space NVMe framework that bypasses the kernel filesystem entirely. After the KV orchestrator assigns a KV shard to an SSD pool device, KV-passthrough allocates a contiguous LBA range on that SSD and writes the KV caches there using asynchronous NVMe writes. The resulting LBA is recorded in the global lookup table (GLT). Retrieval later uses the GLT’s device index and location fields to issue an asynchronous NVMe read directly against the SSD, with completion handled by SPDK polling on a dedicated CPU core (Kang et al., 10 Jun 2026).

The broader design couples this SSD fast path to a bandwidth-aware placement scheme. The KV orchestrator computes placement ratios

$p_i = \frac{b_i}{\sum_{j=1}^{N} b_j},$

where $b_i$ is the bandwidth of pool device $128$0. For decoding step $128$1 with offload set $128$2, the target per-device offload size is

$128$3

KV caches are partitioned in layerwise order, then written in parallel to the assigned pool devices. Host-memory devices use direct memory store, while SSD devices use KV-passthrough. The GLT stores, for each KV cache, a lookup key, device index, device type, and in-device location; retrieval groups requests by device and reads them in parallel, again routing SSD reads through KV-passthrough instead of the filesystem (Kang et al., 10 Jun 2026).

The paper reports that unified KV pooling reduces blocked I/O time by up to $128$4, and for SSD-heavy cases this improvement is directly tied to bypassing filesystem overhead. It further reports TTFTs at $128$5K context of $128$6 s on LLaMA 3.1-8B, $128$7 s on GPT-OSS-20B, and $128$8 s on Qwen3-30B-A3B. The evaluation also shows that the full unified KV pooling design outperforms both “orchestrator only” and “passthrough only,” which establishes that KV-passthrough alone is insufficient if retrieval remains concentrated on one device, while pooling alone is insufficient if SSD access still pays filesystem overhead (Kang et al., 10 Jun 2026).

The implementation assumptions are explicit. SPDK-based passthrough relies on polling, which reduces latency but increases CPU usage; the design dedicates a CPU core to polling. The approach also assumes that the serving engine can maintain and use its own metadata, so it can address KV objects by internal identifiers rather than relying on filesystem services. More advanced completion mechanisms such as user-level interrupts or hybrid polling are mentioned as future work for reducing CPU overhead further (Kang et al., 10 Jun 2026).

3. KV-Passthrough as position-invariant multimodal KV reuse

Kamera uses KV-Passthrough to denote a training-free, position-invariant multimodal KV-cache reuse operator. The central claim is that standard prefix caches entangle two distinct factors: the position of a chunk in the prompt or window, and the conditioning the chunk has absorbed from what came before it. The method separates these two factors so that multimodal chunks can be reused across sliding windows, reordered inputs, and later recalls without re-encoding the vision or video backbone (Ma et al., 22 Jun 2026).

For the positional component, the method relies on exact RoPE composition: $128$9 If a chunk was cached at position $10$0 and must be reused at $10$1, a rotation by $10$2 is applied to its cached keys. The paper emphasizes that keys need RoPE re-rotation to move them to the new target position, whereas values are reused directly in the content channel. This operator is formulated to work across MLA, GQA, and MHA once each architecture is split into a content channel and a RoPE or positional channel (Ma et al., 22 Jun 2026).

Naive reuse, however, loses the cross-chunk conditioning that a chunk has already absorbed from its antecedent context. Kamera distinguishes readout, which is recovered exactly by standard attention state-merge,

$10$3

from conditioning, which is the antecedent-dependent portion already embedded inside the KV state. The missing signal is written as

$10$4

The paper identifies this term as the cross-chunk conditioning deficit or cross-chunk conditioning residue. It reports that blind reuse leaves single-hop recall intact while sharply degrading tasks that require cross-chunk binding (Ma et al., 22 Jun 2026).

The repair is a low-rank conditioning patch stored alongside each canonical chunk. At compile time, one conditioned forward on $10$5 is used to extract $10$6, subtract the relocated solo chunk, and factorize the residual: $10$7 Serve-time reuse is then

$10$8

This is the paper’s fundamental KV-Passthrough operator: exact relocation plus low-rank conditioning restoration (Ma et al., 22 Jun 2026).

The empirical motivation for this decomposition is that the lost conditioning signal is reported as low-rank in feature space, diffuse over tokens, and concentrated in middle or deep layers. The paper states that the conditioning deficit has a knee around rank $10$9 and saturates around $3\times$0; rank-32 usually reaches the KL plateau, while rank-16 already recovers much of the gap. It also reports that the patch can optionally be stored only for the deepest roughly half of layers, yielding a “deep-half” variant that preserves about $3\times$1 of full fidelity with reduced storage (Ma et al., 22 Jun 2026).

Kamera frames three operations that this operator makes cheap. In reorder, the same chunks can be reused in a different order because the positional part is re-rotated and the conditioning patch is stored per chunk or orbit. In sliding-window survival, surviving chunks often require RoPE re-rotation only, with no conditioning patch in many cases. In recall, an evicted chunk is rehydrated from the canonical chunk plus a fresh patch computed on the then-valid antecedent context (Ma et al., 22 Jun 2026).

4. Continuity and recurrence mechanisms that resemble passthrough

PipeLive and KV-Fold do not define KV-Passthrough as a named component, but both describe mechanisms in which KV state remains continuously usable across an execution boundary. PipeLive addresses live in-place pipeline-parallel reconfiguration. The paper states that GPUs are already saturated by model weights plus KV cache, leaving little room for new layer placements, and that in-place reconfiguration therefore requires KV resizing and KV synchronization without interrupting inference (Bai et al., 14 Apr 2026).

PipeLive’s first enabling mechanism is a redesigned KV cache layout with non-contiguous block access. Instead of a single contiguous GPU buffer per layer, KV is represented as a list of independently allocated GPU blocks. PipeLive modifies the PageAttention block table so that it stores resolved block addresses directly, enabling access to non-contiguous physical KV blocks on the fly and preserving PageAttention-like access efficiency “with no measurable performance degradation in practice.” Shrinking is done by compacting live KV blocks toward the front of the block list and releasing blocks in batches, with overhead “less than 1ms,” while expansion is done by appending newly allocated blocks (Bai et al., 14 Apr 2026).

The second mechanism is incremental KV patching, inspired by live virtual machine migration. Sender and receiver threads are spawned for each source–destination GPU pair; the sender maintains a dirty bitmap for newly written KV slots, periodically drains it, gathers the corresponding KV data, and sends a KV patch to the destination. The receiver applies each incoming patch directly into the local layer KV cache. Convergence is tracked by a scheduler-side cumulative token counter $3\times$2 and a receiver-side applied counter $3\times$3, with commit gated by

$3\times$4

for all destination GPUs, where the testbed sets $3\times$5 tokens (Bai et al., 14 Apr 2026). The paper explicitly says that it does not use the term KV-Passthrough, yet functionally the design forms a continuous pathway for KV state movement while inference remains live.

KV-Fold presents a different analogue: KV cache itself becomes the accumulator in a recurrent left fold over chunks. A sequence is divided into chunks $3\times$6, and the transformer performs

$3\times$7

equivalently

$3\times$8

At every layer, the keys and values from chunk $3\times$9 are passed into chunk $84\%$0 unchanged as prefix state with continuous position IDs across the boundary, and chunk $84\%$1’s KV is appended and carried forward with “no copy, no transformation” (Nadali et al., 12 May 2026).

The paper directly interprets this as preserving the entire KV history rather than using bounded-memory streaming or sliding-window approximations. It reports stable recurrence dynamics, including a drift plateau whose change between depths $84\%$2 and $84\%$3 is $84\%$4 nats on Qwen2.5-7B-Instruct with $84\%$5 and $84\%$6, and $84\%$7 exact retrieval overall on needle-in-a-haystack trials spanning contexts from $84\%$8K to $84\%$9K tokens and chain depths up to $\rightarrow$0 on Llama-3.1-8B-Instruct (Nadali et al., 12 May 2026). This is not passthrough in the storage or reuse-operator sense, but it is a direct form of KV state continuity across chunk boundaries.

These systems motivate a broader interpretation: passthrough-like design often means that KV is neither recomputed wholesale nor collapsed into a lossy summary at the boundary where it must move.

5. Retention, compression, and adaptive bypass as neighboring paradigms

Several papers address the same bottleneck—KV state becoming the dominant memory, bandwidth, or quality constraint—by deciding what should remain in the cache rather than by forwarding the cache unchanged. These are adjacent to KV-Passthrough but are not equivalent to it.

TGV-KV is a text-grounded KV cache eviction framework for vision-LLMs. Its retention policy, Text-Prioritised Retention (TPR), keeps text KV first and fills remaining budget with top-scoring vision KV: $\rightarrow$1 The paper does not introduce KV-Passthrough as a separate module name, but the retention behavior is its closest analogue: text KV are protected by design because text tokens are described as highly sensitive to eviction, whereas vision tokens are highly redundant (Liu et al., 2 Jun 2026).

R-KV similarly focuses on selective retention during long reasoning traces. It scores tokens by

$\rightarrow$2

combining attention-derived importance and key-similarity-derived redundancy. The paper reports near-100% full-cache performance with only $\rightarrow$3 KV cache, $\rightarrow$4 of full-cache performance with $\rightarrow$5 KV cache, $\rightarrow$6 memory saving, and $\rightarrow$7 throughput over standard chain-of-thought reasoning inference (Cai et al., 30 May 2025). This is not a passthrough operator; it is a redundancy-aware retention policy.

KVServe addresses disaggregated serving, where KV becomes an explicit payload crossing network and storage boundaries. Its controller decides whether to compress, how to compress, and whether compression is worthwhile. Compression is beneficial only below a profile-specific threshold

$\rightarrow$8

When compression is non-beneficial, the system effectively falls back to the default uncompressed path. The paper is explicit that this is not a dedicated passthrough transport path, but it yields passthrough-like behavior through adaptive bypass of compression (Liu et al., 13 May 2026).

Together, these works delimit the conceptual boundary of KV-Passthrough. Passthrough preserves or reuses KV with minimal semantic alteration; eviction and compression instead decide which fraction of KV should survive. The technical problems are closely related, but the mechanisms are distinct.

6. Performance characteristics, trade-offs, and misconceptions

A recurring misconception is that KV-Passthrough denotes a universal optimization that can be applied independently of system context. The papers instead show that its effect depends on the bottleneck being addressed. In unified KV pooling, passthrough is beneficial because long-context TTFT is dominated by storage stalls once the KV footprint spills to SSD; filesystem traversal, not raw device bandwidth, is the residual bottleneck (Kang et al., 10 Jun 2026). In Kamera, KV-Passthrough is beneficial because naive position-independent reuse preserves direct readout but loses cross-chunk conditioning, which selectively damages multi-hop reasoning rather than single-hop recall (Ma et al., 22 Jun 2026).

Another misconception is that direct reuse of cached KV is always semantically exact. Kamera explicitly shows that the direct readout of a cached chunk is recovered exactly and for free by the standard state-merge, but blind reuse leaves a cross-chunk conditioning residue inside the KV state, so single-hop recall may remain intact while multi-hop accuracy drops sharply. On two-page document QA, the paper reports single-hop accuracy staying around $\rightarrow$9 while multi-hop accuracy drops from $\rightarrow$0 to $\rightarrow$1 for MLA and from $\rightarrow$2 to $\rightarrow$3 for GQA under blind reuse (Ma et al., 22 Jun 2026).

Conversely, another misconception is that bypassing the filesystem or forwarding KV state is by itself sufficient to restore service-level performance. Unified KV pooling reports that “passthrough only” underperforms the full design because retrieval can still be concentrated on one device, while “orchestrator only” underperforms because SSD access still pays filesystem overhead (Kang et al., 10 Jun 2026). PipeLive makes an analogous point in a different context: non-contiguous access without controlled synchronization would not solve KV consistency during live reconfiguration, and synchronization without resize-friendly layout would not solve the memory-pressure problem (Bai et al., 14 Apr 2026).

The trade-offs are implementation-specific. Unified KV pooling dedicates a CPU core to SPDK polling and notes the latency-versus-CPU trade-off of polling-based filesystem bypass (Kang et al., 10 Jun 2026). Kamera trades patch rank against storage footprint: rank-16 uses about $\rightarrow$4 of the segment KV bytes, rank-64 about $\rightarrow$5, and deep-half storage reduces bytes further with only a small accuracy drop (Ma et al., 22 Jun 2026). PipeLive trades layer stacking factor $\rightarrow$6 between memory utilization and reconfiguration granularity (Bai et al., 14 Apr 2026). These patterns suggest that KV-Passthrough is best treated as a systems interface between KV semantics and the dominant hardware or serving bottleneck, not as a universally fixed primitive.

7. Relation to long-context and multimodal inference

KV-Passthrough becomes most salient when the cost of recomputation or mediated access grows faster than the model’s useful compute. Long-context LLM serving is one such regime. Unified KV pooling reports TTFT explosions at $\rightarrow$7K context and shows that filesystem-heavy SSD retrieval can dominate service time; KV-passthrough is introduced precisely because the serving engine already knows exactly which KV object it wants and therefore does not need the general-purpose file abstraction, metadata handling, and block management of a conventional filesystem (Kang et al., 10 Jun 2026).

Multimodal agents provide a second regime. Kamera begins from the observation that agents repeatedly re-examine the same video frames, UI screenshots, and rendered artifacts as their context window slides and reasoning iterates, yet every look-back re-encodes from scratch because prefix caches serve reuse only at a fixed leading position. KV-Passthrough addresses this by making three window operations cheap—reorder, sliding-window survival, and recall—through exact RoPE re-rotation plus conditioning restoration (Ma et al., 22 Jun 2026). TGV-KV reinforces the same systems pressure from a different angle: VLMs are especially exposed to KV growth because images and videos can contribute thousands of visual tokens, and text-grounded retention is needed because vision tokens are highly redundant while text tokens are semantically fragile under eviction (Liu et al., 2 Jun 2026).

Reasoning-heavy decoding provides a third regime. R-KV argues that reasoning models generate excessively long and repetitive chain-of-thought traces, producing very large KV caches not because every token is equally informative but because many tokens are redundant reflections, re-checks, and verbose self-dialogue (Cai et al., 30 May 2025). KV-Fold, by contrast, shows that when exact recall matters and linear cache growth is acceptable, a frozen pretrained transformer can reuse the entire prior KV history across hundreds of chunk transitions without retraining (Nadali et al., 12 May 2026). These papers do not share a common passthrough term, but they converge on the same systems insight: once KV becomes the state that determines memory footprint, retrieval latency, or cross-chunk fidelity, efficient inference depends on treating KV as a first-class object rather than as an incidental by-product of attention.

Taken together, the literature presents KV-Passthrough as an emerging umbrella concept for direct KV data paths, position-invariant KV reuse, and continuity-preserving movement of attention state. The exact mechanism differs by setting, but the underlying principle is consistent: avoid unnecessary mediation, preserve the part of KV state that already encodes useful computation, and expose that state to the next stage of serving with as little distortion and overhead as the deployment regime allows.