KV-Cloak: Secure KV-Cache Defense

• KV-Cloak is a reversible obfuscation mechanism that secures the KV-cache by applying secret invertible transforms, one-time per-block permutations, and structured additive masks.
• It maintains efficient LLM inference and exact attention computation through operator fusion and RoPE-compatible modifications with minimal runtime overhead.
• The method effectively mitigates direct inversion, collision, and injection attacks, reducing prompt reconstruction quality to near-random levels.

Searching arXiv for the KV-Cloak paper and closely related KV-cache privacy work to ground the article. Search query: "KV-Cloak KV-cache privacy leakage LLM inference arXiv" KV-Cloak is a defense mechanism proposed to protect the Key-Value cache used during LLM inference from leaking users’ private prompts. Introduced in “Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference” (Luo et al., 13 Aug 2025), it starts from the observation that the KV-cache is not merely an efficiency artifact but a persistent, structured, token-aligned representation of the user’s context. KV-Cloak protects each KV block using secret invertible transforms, a per-block one-time permutation, and a structured additive mask, while operator fusion absorbs much of the cost into the model weights. The method is designed to preserve the efficiency benefits of KV caching, preserve exact attention computation and model outputs, and avoid the severe utility loss of noise-based defenses.

1. Problem setting and privacy exposure

Modern decoder-only LLMs generate text autoregressively: $$P(x_1,\ldots,x_n)=P(x_1)P(x_2|x_1)\cdots P(x_n|x_n,\ldots,x_{n-1}).$$ At each step, a Transformer maps token representations to query, key, and value vectors: $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$ The attention weights and output are written as

$a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$

Without caching, previous keys and values would be recomputed repeatedly; KV-cache stores them explicitly and reuses them in later decoding steps. This makes inference much faster, especially for long prompts and multi-turn conversations (Luo et al., 13 Aug 2025).

The same persistence that makes KV-cache useful also makes it dangerous. The paper treats the cache as a highly informative representation of the user’s input, one that may be read from memory, storage, or inter-node communication. The attacker model is gray-box: the attacker can read leaked KV-cache contents and has access to the foundation model weights. The objective is to recover the user’s private input prompt, ideally verbatim. This threat model is stronger than timing side channels and is aimed directly at raw cache disclosure.

A common misconception is that encrypted client-server traffic is sufficient. The paper’s point is narrower and more operational: even when communication is protected, the KV-cache is often handled in plaintext because encrypting huge caches would impose unacceptable latency. KV-Cloak therefore addresses a systems-level privacy surface created by the inference stack itself rather than by the tokenizer, transport channel, or output interface.

2. Leakage mechanisms and prompt-recovery attacks

The paper develops three attacks to show that plaintext KV-cache leakage can reveal both exact text and semantic content (Luo et al., 13 Aug 2025).

Direct inversion uses the projection equations directly. Because

$$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$

the paper gives the inversion formulas

$$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$

This attack is strongest in the first decoder layer and in architectures where the relevant projection is invertible or well-conditioned. On LLaMA-7B, first-layer inversion reaches BERTScore $=1.000$ and ROUGE-L $=1.000$. It is much weaker on deeper layers and generally weaker on modern GQA models, although LLaMA-3.2-1B remains a notable exception with first-layer BERTScore $=1.000$ and ROUGE-L $=0.994$.

Collision attack treats recovery as forward matching rather than algebraic inversion. The attacker reconstructs tokens one by one, generates local KV-cache segments for candidate continuations, and compares them to the leaked target using the Frobenius norm. Its success is formalized by

$$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$

where $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$0 is a threshold and $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$1 is the rank of the true token in the probability-sorted candidate list. The paper reports that the attack works across layers, on MHA and GQA models, and even against fine-tuned models using only the public base model. On LLaMA-3.2-1B, plain collision gives BERTScore $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$2 on first, middle, and last layers, with ROUGE-L $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$3. The enhanced form, “Collision+,” reaches near-perfect recovery in many settings.

Injection attack uses the model itself as an exfiltration mechanism. Given the full leaked KV-cache, the attacker appends an instruction such as “Repeat the previous content.” or “Summarize the previous content.” and resumes generation. This is less exact than collision but much faster and still leaks substantial semantic information. Using the prompt “Repeat the previous content.”, the paper reports an average BERTScore of about $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$4 and an average ROUGE-L of about $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$5.

Attack	Core mechanism	Representative severity
Direct inversion	Invert $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$6, $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$7, and RoPE structure	LLaMA-7B first layer: $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$8
Collision	Match candidate-generated KV segments to leaked KV	LLaMA-3.2-1B: BERTScore $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$9
Injection	Continue generation from leaked cache with crafted prompts	Average BERTScore about $a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$0

These attacks establish the central motivation for KV-Cloak: raw KV-cache is not merely correlated with the prompt; it can often be used to reconstruct the prompt with high or near-perfect fidelity.

3. Core construction of KV-Cloak

KV-Cloak is presented as a reversible matrix-based obfuscation scheme for KV-cache blocks (Luo et al., 13 Aug 2025). For a block of keys

$a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$1

a naive reversible transform is

$a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$2

with secret invertible matrices $a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$3 and $a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$4. The paper argues that this baseline is insecure under chosen-plaintext analysis because differences obey

$a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$5

A stable linear relation remains visible across observations.

The improved defense introduces a fresh per-block permutation matrix $a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$6, giving

$a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$7

This destroys stable row order while preserving correctness, since positional information is already encoded into $a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$8 and $a_{ij} = \frac{\exp\left(q_{i} {k_{j}^{\top}/\sqrt{d}\right)}{\sum_{t=1}^{i}\exp\left(q_{i} {k_{t}^{\top}/\sqrt{d}\right)},\quad o_{i} = \left(\sum_{j=1}^{i}a_{ij} v_{j}\right)W_o^\top.$9 via RoPE. The paper states that this increases brute-force matching complexity by a factor of $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$0, where $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$1 is the block size.

A further refinement addresses rank-collapse or low-entropy blocks by adding a structured mask: $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$2 Here $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$3 contains row-specific outlier values. The paper observes that KV entries are usually bounded by a small threshold $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$4, for example $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$5, and samples $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$6 values from $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$7, with padding values around $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$8. This lets the legitimate system identify and subtract $$k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top,$$9 from outlier structure without storing $$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$0 itself. The same protection is applied analogously to both K-cache and V-cache.

The implementation is blockwise and per-head. The paper tests block sizes 16, 32, and 64, and notes that block size 16 matches vLLM’s default PagedAttention block. It samples $$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$1 and $$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$2 from orthogonal matrices to reduce numerical instability in inversion. The defense is explicitly not standard encryption; it is a reversible obfuscation mechanism designed so that the stored cache is mathematically protected while legitimate attention remains exact.

4. Exact preservation of attention, operator fusion, and RoPE compatibility

A central claim of KV-Cloak is that it preserves exact attention computation and model outputs (Luo et al., 13 Aug 2025). The paper defines transformed quantities

$$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$3

The score invariance derivation is

$$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$4

The output invariance derivation is

$$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$5

These equalities are the basis for the paper’s claim of lossless accuracy.

The paper then uses operator fusion to move expensive matrix multiplications offline into the model weights. The fused weights are

$$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$6

After fusion, the online obfuscation becomes

$$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$7

with an analogous expression for $$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$8. This substantially lowers online cost.

RoPE compatibility is a nontrivial constraint. For $$x_i = k_i ({R}_{\Theta, i}^d)^{-1} (W_k^{\top})^{-1}, \quad x_i = v_i (W_v^{\top})^{-1}.$$9 and $=1.000$0, fusion requires

$=1.000$1

The paper derives that $=1.000$2 must take a specific block-diagonal rotation-scaling form. This is how KV-Cloak is made compatible with modern RoPE-based models. The paper explicitly contrasts this with KV-Shield on this point.

The theoretical online cost per $=1.000$3 block is reduced from

$=1.000$4

to

$=1.000$5

after fusion. For LLaMA-3.1-8B with $=1.000$6, $=1.000$7, and $=1.000$8, the paper estimates naive overhead at about $=1.000$9 of recomputing the block from hidden states, and fused overhead at about $=1.000$0.

5. Empirical security, utility, and efficiency

The empirical evaluation covers six models—LLaMA-7B, LLaMA-3.2-1B, LLaMA-3.2-3B-Instruct, LLaMA-3.1-8B, LLaMA-3.1-8B-Distilled, and Qwen2.5-Math-7B—using 1,000 sampled instances from lmsys-chat-1m for attack evaluation, and MMLU and SQuAD for utility evaluation (Luo et al., 13 Aug 2025).

The strongest security result is that KV-Cloak reduces reconstruction quality to near-random levels across all attacks and evaluated models. For LLaMA-7B under KV-Cloak, first-layer inversion BERTScore is $=1.000$1; collision on first, middle, and last layers gives BERTScore $=1.000$2; Collision+ gives $=1.000$3; injection gives $=1.000$4; and corresponding ROUGE-L values are essentially $=1.000$5. For LLaMA-3.2-1B under KV-Cloak, inversion is $=1.000$6, collision is $=1.000$7, Collision+ is $=1.000$8, injection is $=1.000$9, and ROUGE-L is almost all $=1.000$0. A distributional comparison shows that under KV-Cloak, $=1.000$1 and $=1.000$2 become completely overlapping or indistinguishable, yielding a reported $=1.000$3 per-token collision success.

The paper compares KV-Cloak to a Gaussian-noise differential privacy baseline on K and V independently, using clipping at the 50th percentile of observed norms and $=1.000$4-DP as a practical configuration. The DP baseline weakens some attacks but does not fully solve the leakage problem. For LLaMA-3.2-1B at $=1.000$5-DP, collision BERTScore remains $=1.000$6, and Collision+ remains near $=1.000$7 on some layers.

The utility comparison is equally central. KV-Cloak is described as empirically lossless. On LLaMA-7B, MMLU remains $=1.000$8 and SQuAD $=1.000$9. On LLaMA-3.1-8B, MMLU remains $=0.994$0 and SQuAD $=0.994$1. By contrast, the DP baseline causes severe collapse: for LLaMA-7B, MMLU drops to $=0.994$2 and SQuAD to $=0.994$3; for LLaMA-3.1-8B, the corresponding scores are $=0.994$4 and $=0.994$5.

Measured runtime overhead on MMLU is modest: $=0.994$6 for LLaMA-7B, $=0.994$7 for LLaMA-3.2-1B, $=0.994$8 for LLaMA-3.2-3B-Instruct, $=0.994$9 for LLaMA-3.1-8B, $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$0 for LLaMA-3.1-8B-Distilled, and $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$1 for Qwen2.5-Math-7B. The paper summarizes this as an average overhead of about $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$2, mostly under $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$3. Block sizes 16, 32, and 64 all preserve accuracy while keeping latency around 9–10% overhead, which supports compatibility with PagedAttention-style block management.

Setting	Plaintext / DP behavior	Under KV-Cloak
LLaMA-7B inversion	First layer $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$4	BERTScore $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$5, ROUGE-L essentially $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$6
LLaMA-3.2-1B collision	Plaintext $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$7	$$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$8
Utility on LLaMA-3.1-8B	Plaintext $$P(\text{success}|t) = P(dis_{other} > t)^{r-1} \times P(dis_{target} < t),$$9; DP $$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$00	$$q_i = x_i W_q^\top R_{\Theta, i}^d,\quad k_i = x_i W_k^\top R_{\Theta, i}^d,\quad v_i = x_i W_v^\top.$$01

These results support the paper’s characterization of KV-Cloak as a lossless, lightweight, reversible obfuscation mechanism for LLM KV-cache.

6. Interpretation, deployment scope, and limitations

KV-Cloak is best understood as a systems-oriented privacy defense for inference deployments where KV-cache may cross trust boundaries (Luo et al., 13 Aug 2025). The paper identifies several especially relevant settings: prefill and decode running on different nodes, caches swapped or persisted, multi-tenant GPU memory leakage, and deployments built on open-source or fingerprintable foundation models. It is also explicitly designed for modern inference engines such as vLLM and for PagedAttention because it operates blockwise and creates no cross-block dependencies.

Its security rationale is empirical and structural rather than framed as standard cryptographic indistinguishability. Fixed linear obfuscation alone is vulnerable; adding a fresh one-time permutation per block breaks stable algebraic relations; the additive mask preserves permutation entropy even in low-rank blocks; and semantic injection fails because the protected cache becomes unintelligible to a normal model. A plausible implication is that KV-Cloak occupies a middle ground between plaintext caching and full cryptographic protection: it is stronger than heuristic noise addition in the reported experiments, yet it is designed to retain exact attention and practical serving latency.

The paper is also explicit about boundaries. Secret invertible matrices and related parameters must remain protected in memory; privileged host compromise is out of scope. Better key management and rotation are future work. Integration with TEEs or confidential GPUs would strengthen the model. The current work focuses on floating-point models rather than quantized ones. These limitations do not negate the reported results, but they clarify that KV-Cloak protects the cache representation itself and does not by itself solve all memory-resident secret management problems.

The broader significance of KV-Cloak lies in how it reframes KV-cache. In conventional serving practice, the cache is treated primarily as a performance primitive. KV-Cloak instead treats it as a first-class privacy surface. That reframing is the paper’s most durable contribution: it shows that prompt confidentiality in LLM systems depends not only on transport security and model outputs, but also on how intermediate attention state is represented, stored, and moved within the inference pipeline.