Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems

Abstract: Rowhammer on GPU DRAM has enabled adversarial bit flips in model weights; shared KV-cache blocks in LLM serving systems present an analogous but previously unexamined target. In vLLM's Prefix Caching, these blocks exist as a single physical copy without integrity protection. Using software fault injection under ideal bit targeting, we characterize worst-case severity and identify three properties: (1) Silent divergence - 13 of 16 BF16 bit positions produce coherent but altered outputs, indistinguishable from legitimate responses without a clean baseline. (2) Selective propagation - only requests sharing the targeted prefix are affected. (3) Persistent accumulation - no temporal decay occurs, so cumulative damage grows linearly with subsequent requests. Together, these constitute a threat profile distinct from weight corruption: silent divergence and selective propagation enable detection evasion; persistent accumulation then proceeds unchecked, yielding damage amplification bounded only by how long the block remains cached. A checksum-based countermeasure detects any single-bit corruption at scheduling time, bounding cumulative damage to one batch independent of the block's cache lifetime, with negligible overhead. These results argue for integrity protection of prefix blocks before end-to-end exploitation is demonstrated.

• The paper identifies that a single hardware bit flip in a KV-cache block can cause silent divergence in LLM outputs, impacting multiple requests.
• Experiments show that 13 of 16 BF16 bit positions, when flipped, maintain high output similarity (BERTScore ≥ 0.93) and selectively propagate errors.
• The study proposes a SHA-256 hash-based integrity verification that limits cumulative damage from unbounded propagation to a constant impact.

Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems

Introduction and Motivation

The paper "Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems" (2604.17249) addresses the integrity risks associated with shared key–value (KV) tensor blocks in state-of-the-art LLM serving architectures, with a focus on vLLM’s Prefix Caching implementation. The authors identify a new hardware-induced vulnerability surface in production LLM systems, where a single hardware bit flip in a cached prefix block residing in GPU memory can propagate across all concurrent and future requests sharing the affected prefix. Unlike static weight corruption, this dynamic exploitation vector is unprotected by common integrity mechanisms—such as checksums or ECC—and uniquely leverages the single-copy, pervasive nature of cached KV-blocks to maximize practical impact while avoiding immediate detection.

Threat Characterization and Properties

Through methodical software fault injection (SFI) under an idealized bit-targeting regime, the authors characterize the qualitative and quantitative severity of this vulnerability. Their results reveal three distinctive and compounding properties:

1. Silent Divergence: Systematic bitwise evaluation demonstrates that 13 out of 16 bit positions in BF16 encoding, if flipped in a prefix block’s value tensor, cause coherent but altered model outputs—indistinguishable from legitimate responses without a clean baseline, due to the subtlety of the perturbations.
2. Selective Propagation: Effects are strictly confined to requests referencing the corrupted cache block, i.e., those with an identical prompt prefix, thus avoiding detection through aggregate quality metrics monitored at the service level.
3. Persistent Accumulation: Cached prefix blocks are immutable and remain resident until evicted (often indefinitely for popular prefixes). As a result, cumulative damage from a single flip grows linearly with every subsequent request, meaning that an undetected fault can affect an unbounded number of users and outputs.

These empirical findings are visually summarized in the next figure, mapping mean token change rate (TCR) by bit position across all concurrency levels.

Figure 1: Mean TCR by BF16 bit position, demonstrating a strong hierarchy from mantissa LSB to exponent MSB.

By distinguishing silent divergence from catastrophic collapse, the study advances prior bit-flip research, indicating that the majority of practical bit-flip errors in production settings will be far more challenging to detect and remediate than previously assumed.

Experimental Methodology and Results

The authors utilize a robust SFI framework within vLLM (v0.18.1rc0), targeting only reliably shared prefix blocks in the value tensors of the KV-cache. Experiments span two leading open-weight BF16 LLMs (Qwen3-8B and DeepSeek-R1-8B), with carefully controlled concurrency and per-block cache residency. Within their main 2,400-trial sensitivity study, bit positions are systematically evaluated for in-batch and cross-batch divergence.

Strong numerical evidence is provided: 13/16 bit positions cause silent divergence, as measured by high ROUGE-L and BERTScore values maintained on changed outputs (often $\geq 0.93$ BERTScore, $\geq 0.71$ ROUGE-L), signifying that generated outputs remain linguistic and contextually plausible. Collapse occurs almost exclusively at bit 14 ($E_7$, upper exponent), producing highly unrepresentative, easily detectable outputs.

Selective propagation is validated by a two-group experiment, where only the group sharing the faulted prefix diverges, confirming strict locality of impact among affected workloads.

Temporal persistence is quantified via sequential queries: per-request corruption rates remain statistically constant over at least 100 subsequent requests, with cumulative affected counts growing linearly in the number of queries.

Figure 2: Per-request corruption rate $\bar{c}_i$ remains robustly constant over 100 sequential requests for all significant bit positions and across both tested models.

Figure 3: The mean cumulative affected count $\bar{C}_N$ demonstrates the linear, unbounded growth of affected outputs with continued cache residency.

Concurrency independence is also observed: within the tested range, changing batch size does not affect the per-request probability of corruption.

Countermeasure: Integrity Verification

The paper proposes and empirically validates a lightweight, scheduling-time, checksum-based integrity verification as a countermeasure. Upon cache block creation or reuse, a SHA-256 hash is computed and later checked on matched cache accesses. This mechanism detects and intervenes on any single-bit corruption before serving affected blocks, bounding damage to a single batch of requests, independent of block residency duration.

The security guarantee is that any single-bit error will be caught prior to further propagation, reducing worst-case cumulative damage from $\Theta(N)$ to $O(1)$, with measured throughput overhead being negligible and zero false positives across 3,000 monitored requests.

Implications for LLM Security, Reliability, and Future LLM System Design

The results have direct operational and theoretical implications:

• Integrity Monitoring Priority: KV-cache blocks, particularly popular system prompts or relevant shared prefixes, constitute a critical new security boundary. Relying on output quality monitoring (repetition, perplexity) alone is fundamentally insufficient for broad classes of faults. Hardware ECC, where available, is valuable but not universal, and currently cannot fully mitigate targeted bit flips on many cloud GPU configurations.
• Runtime Verification and TTL: Short-term practical mitigations include TTL-based (time-to-live) prefix invalidation or forced recomputation, but only runtime integrity checks can address silent divergence without loss of caching efficiency. Extensions to multi-bit protection schemes and lower-precision formats are a logical next step, as are GPU-resident hash kernels to close remaining TOCTOU windows.
• Research Directions: Generalization to multi-GPU, pipeline/model parallel LLM architectures and fine-grained analysis of physical bit targeting (i.e., bridging the SFI–Rowhammer gap) remain necessary to assess real-world exploit feasibility. Key-side sensitivity and hybrid attack vectors also merit systematic exploration. There are also reliability engineering concerns: non-adversarial single-event upsets in DRAM/GDDR memory follow the same vulnerability profile.

Conclusion

This investigation provides a rigorous, empirical basis for the previously overlooked threat profile associated with shared prefix block corruption in LLM serving systems. By demonstrating that a single, well-placed bit flip can result in unbounded, silent, selectively-propagating, and persistent inference divergence—unlike conventional weight corruption—the paper establishes both the urgency and the practical efficacy of integrity verification for dynamic cache structures. LLM system implementers and researchers should consider deploying such defenses as a baseline measure, especially as architecture-level supply chain and side-channel attacks continue to evolve. Extension to a broader range of hardware, model configurations, and empirical real-world deployments constitutes a clear path forward for future work.