Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

Abstract: We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.

• The paper identifies a novel attack surface in local LLM inference: CPU-side detokenization that generates semantically meaningful outputs through cache activity observation.
• The attack strategy combines Flush+Reload for synchronization and L1 Prime+Probe to measure token-related activity, achieving an impressive 97.66% true-positive rate on initial cache identification.
• Experiments showed that the reconstruction pipeline, utilizing an MLP encoder and Flan-T5-XL translation, yielded substantial semantic recovery, averaging around 56-87% content recovery, thereby highlighting the potential for privacy leaks in sensitive information rendering.

Attack surface and threat model

“Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces” identifies CPU-side detokenization as a previously underexamined leakage surface in local LLM inference (2609.06674). The central claim is that an attacker can reconstruct semantically meaningful portions of generated outputs by observing cache activity associated with token-to-text conversion, without accessing model weights, GPU memory, internal activations, or shared model data.

The attack targets long-lived local inference services such as Llama.cpp- or HuggingFace-based servers. These services commonly expose localhost APIs to desktop applications, IDE integrations, and agent frameworks. The threat model assumes an unprivileged adversarial process running on the same machine as the victim and scheduled on the SMT sibling of the physical core executing the tokenizer. The attacker can issue profiling prompts to the local service, observe its own responses, execute clflush and high-resolution timing instructions, and access shared tokenizer code pages. The model family, quantization, vocabulary, and inference framework need not be known in advance, although the attacker must identify the tokenizer implementation and its decode routine.

The attack therefore depends on SMT-enabled CPUs and co-residency. This is a consequential but explicit assumption: disabling SMT would obstruct the L1 Prime+Probe component, while process restarts or randomized allocations would invalidate a process-specific profile. Neither condition eliminates the underlying leakage channel, because an attacker may re-profile a new process or exploit another deployment in which SMT remains enabled.

The authors emphasize that the vulnerable component is widely reused.

Figure 1: The vulnerable tokenizer implementations extend the attack surface to local LLM servers and dependent applications.

The practical implication is broader than an attack against a particular model. If a local application invokes a vulnerable tokenizer library, its generated text may become observable through a microarchitectural side channel, including outputs produced by agents operating on files, credentials, communications, or other local resources.

Detokenization as a leakage source

Autoregressive inference produces token IDs, which must be converted into UTF-8 text for streaming output. In Llama.cpp, this conversion accesses an array-like decode table indexed by token ID; HuggingFace Tokenizers uses an equivalent mapping implemented through a hash table. Because the decode-table layout remains stable after process initialization, repeated decoding of the same token induces repeatable memory accesses.

Those accesses influence L1 cache-set occupancy. Under the paper’s simplified x86 model, a token-dependent lookup maps to a cache set according to the memory location of its decode-table entry. The mapping is many-to-one: a vocabulary containing tens of thousands or hundreds of thousands of tokens is projected onto approximately 64 L1 cache sets. Consequently, an individual cache trace does not identify a token reliably. The leakage is instead a noisy, collision-prone representation of the ordered token sequence.

This observation distinguishes the attack from prior LLM side channels. Embedding-based attacks require shared model pages or CPU-resident embedding lookups, while MoE attacks require architecture-specific routing behavior. The present attack targets a post-generation operation that is executed by dense and MoE models alike, provided that the tokenizer performs the relevant CPU-side lookup. The resulting channel is weaker at the individual-token level but potentially more deployment-independent.

Synchronization and cache-trace acquisition

The attack combines two cache primitives for different purposes. Flush+Reload monitors shared executable pages belonging to the tokenizer library and detects when the decode routine executes. Prime+Probe then measures token-dependent activity in the private L1 data cache at the time indicated by the trigger.

This synchronization mechanism is the principal systems contribution. Prime+Probe applied continuously would mix the decode-table access with unrelated activity, scheduling effects, and other cache traffic. Flush+Reload supplies a high-resolution temporal signal because shared library instructions are available across processes even when the decode table itself is private. In a controlled experiment involving more than 1.2 million simulated decode invocations, the detector achieved a 97.66% true-positive rate with zero false positives.

After a trigger, the attacker probes all 64 L1 cache sets. On the evaluated desktop configuration, the optimal window began approximately $8\,\mu\mathrm{s}$ after the trigger and lasted $15\,\mu\mathrm{s}$. The authors found LLC monitoring impractical because its much larger set space and slower probing cannot be covered within the short decode interval.

Figure 2: Token-dependent decode operations produce distinct but noisy latency patterns across L1 cache sets.

The resulting trace is a 64-dimensional latency vector for each generated token. The paper reports that direct MLP classification of these vectors achieves below 1% top-1 token accuracy, confirming that the raw side channel is insufficient for direct detokenization. The attack succeeds by exploiting stability across traces and linguistic redundancy across sequences rather than by recovering token IDs independently.

From cache traces to text

The reconstruction pipeline has two stages. First, a supervised-contrastive MLP encoder maps normalized latency vectors into an embedding space. Traces associated with the same token are encouraged to cluster, while traces from different tokens are separated. The authors compute token centroids from profiling traces and cluster those centroids with cosine K-means. Each cluster is represented by a synthetic symbol, producing a symbolic sequence from the cache-trace sequence.

The default configuration uses $K=64$ clusters. An ablation shows that $K=1$ yields almost no successful reconstruction, performance rises sharply through $K=48$–$64$, and larger values reduce performance because the training signal becomes fragmented. The reported mean clustering accuracy is approximately 75% on held-out traces. The encoder-plus-clustering design also improves the worst-performing 10% of tokens by approximately $2\times$ relative to direct clustering on raw traces.

Second, Flan-T5-XL translates the symbolic sequence into natural language. Responses are divided into 32-token segments. A first model, $\mathrm{LLM}_A$, reconstructs the first segment from symbols alone. A second model, $\mathrm{LLM}_B$, reconstructs subsequent segments conditioned on both the current symbolic sequence and the preceding reconstructed text. This staged design exploits the fact that early response content often identifies the topic and constrains later wording.

Figure 3: The reconstruction pipeline encodes cache traces, assigns cluster symbols, and uses staged sequence-to-sequence models to recover text.

The design makes a strong and somewhat counterintuitive claim: severe token-level ambiguity does not prevent useful output reconstruction when the symbolic observations are combined with a LLM. The LLM is not merely correcting minor errors; it supplies substantial missing information under a many-to-one observation mapping. This also means that the evaluation measures semantic recovery rather than cryptographic-style plaintext recovery in most cases.

Evaluation across models, frameworks, and tasks

The evaluation covers Phi-3-mini and Llama-3, Llama.cpp and HuggingFace Transformers, an Intel Core i7-8650U laptop, an Intel Core i7-11700K desktop, and three datasets: UltraChat, ChatDoctor, and Code-Alpaca. Each token has 30 profiling traces and 20 held-out traces. The reconstruction models are trained on disjoint data splits, and the evaluation includes both first-segment and full-response reconstruction.

The main results show substantial semantic recovery across configurations:

Dataset and output scope	Reported performance
UltraChat, full responses	ASR approximately 56–87%; $\phi$ approximately 0.76–0.87
ChatDoctor, full responses	ASR approximately 78–93%; $15\,\mu\mathrm{s}$0 approximately 0.61–0.76
UltraChat, first segment	ASR approximately 26–62%, depending on framework and model
ChatDoctor, first segment	ASR approximately 11–85%, depending on configuration
Code-Alpaca, full output	ASR approximately 49–96%
OpenClaw end-to-end first segments	ASR 30.12%; mean $15\,\mu\mathrm{s}$1

The strongest controlled textual result is a 92.85% full-response ASR for ChatDoctor with Phi-3-mini, HuggingFace tokenization, and laptop hardware. UltraChat reaches an 87.42% full-response ASR with Phi-3-mini, Llama.cpp, and desktop hardware. These values should be interpreted alongside the different metrics: ChatDoctor has higher judge-based success but lower mean embedding similarity in some configurations, reflecting the distinction between preserving medically relevant content and reproducing lexical or sentence-level detail.

The paper attributes the consistent degradation on Llama-3 primarily to its approximately four-times larger vocabulary. A larger vocabulary increases cache-set collisions and makes cluster assignments less informative. This result supports the proposed mechanism: performance is not determined solely by CPU generation speed or framework choice, but by the relationship between vocabulary size, token frequency, cache aliasing, and the language distribution of the target responses.

Structured domains are easier to reconstruct than heterogeneous conversational data. On Code-Alpaca, full-output ASR reaches 95.87% for Phi-3-mini with Llama.cpp on the laptop and 90.56% on the desktop. Llama-3 performs more variably, from 49.43% to 86.86% across the tested settings. Because the code judge accepts functionally or structurally similar snippets, these numbers indicate recovery of APIs, frameworks, algorithms, and program structure rather than exact source-code recovery.

The paper’s high-fidelity results establish that the leakage can exceed topic identification. For ChatDoctor first segments, one configuration achieves $15\,\mu\mathrm{s}$2 for 62.92% of samples, with 36.06% exact matches under normalized edit distance. For UltraChat first segments, the strongest configuration reaches 44.62% at $15\,\mu\mathrm{s}$3 and 18.36% exact matches. Full-response recovery is weaker in exact terms, but up to 58.48% of UltraChat outputs still reach $15\,\mu\mathrm{s}$4 under the reported semantic metric.

These results imply that the most sensitive information may be exposed even when complete responses are not recovered. The first segment often contains diagnoses, topics, intentions, or key entities, so partial reconstruction can be operationally sufficient for privacy compromise.

Profiling cost and robustness

The profiling phase is not a minor implementation detail; it determines whether the attack is practical against a long-lived service. The authors evaluate targeted prompts that force selected token sequences and ordinary conversational prompts that collect naturally occurring tokens.

Targeted profiling exceeds 55% ASR with only 250 queries, completing in approximately six minutes in the tested Phi-3-mini configuration. An equivalent coverage profile requires approximately 7,650 benign conversational queries and roughly three hours, producing about 53% ASR. With 10,950 benign queries, ASR reaches approximately 56%, close to the 57.51% unlimited-profile baseline.

Figure 4: Targeted prompting substantially reduces the query budget required to obtain a useful token-trace profile.

This tradeoff has a direct security implication. The attacker need not issue conspicuous adversarial prompts if ordinary interaction with a persistent local server supplies sufficient token coverage. Conversely, targeted profiling increases efficiency but may be detectable through unusual repetition requests.

The reconstruction model also tolerates additional symbolic noise. The authors inject deletions, insertions, and substitutions corresponding respectively to missed decode triggers, false positives, and clustering errors. At corruption probability $15\,\mu\mathrm{s}$5, ASR remains approximately 30%, and about 10% of reconstructions retain $15\,\mu\mathrm{s}$6.

Figure 5: Reconstruction degrades gradually under synthetic insertion, deletion, and substitution noise.

The robustness result is important but bounded. The evaluation uses a model trained on clean traces and synthetic corruption applied after symbolization; it does not establish equivalent robustness against every source of system-level interference. In particular, an active defender could introduce workload patterns or scheduling interference unlike the injected noise model.

End-to-end deployment and mitigations

The OpenClaw experiment tests profiling, trace collection, model training, and reconstruction on a live local agent rather than on dataset-aligned traces. Using 25,000 UltraChat prompts and a local Phi-3-mini backend, the attack achieves 30.12% ASR and mean semantic similarity of $15\,\mu\mathrm{s}$7 on first response segments. This is substantially below the corresponding controlled Llama.cpp desktop result of 61.55% ASR and $15\,\mu\mathrm{s}$8.

The gap demonstrates both practical transfer and deployment sensitivity. The attack survives the additional timing variability and alignment errors of a real agent, but its accuracy is materially reduced. The paper therefore supports the claim that the vulnerability is exploitable in an end-to-end system, while not showing that controlled-dataset performance directly predicts operational performance.

The proposed mitigations are imperfect. Randomizing or shuffling decode-table placement would invalidate an existing profile, but an attacker could re-profile after each randomization event. Short-lived workers and periodic restarts limit the lifetime of a profile, at the cost of repeated model initialization and latency. Disabling SMT directly blocks the assumed L1 channel but may incur a reported 25–35% performance penalty. Cache partitioning, page coloring, and randomized cache indexing could reduce observability, although these mechanisms are generally unavailable or impractical on consumer systems.

A more fundamental mitigation question remains open: whether tokenizer implementations can remove or decorrelate token-dependent memory-access patterns without imposing unacceptable inference overhead. The paper establishes that layout randomization increases attack cost, but it does not demonstrate a defense that eliminates leakage under an adaptive attacker.

Limitations and open questions

The attack’s universality is conditional rather than absolute. It requires a vulnerable tokenizer implementation, shared executable code pages, SMT co-residency, and stable execution on a shared physical core. Systems without SMT, systems with effective core isolation, nonstandard tokenizer implementations, or hardware with stronger cache isolation may not satisfy the threat model.

The evaluation is also concentrated on Intel CPUs and on two principal tokenizer frameworks. A 13th-generation Intel Core i9-13950HX experiment shows degraded but nonzero performance: first-segment ASR is 38.40% for Phi-3-mini and 31.97% for Llama-3. This indicates persistence under newer hardware prefetch behavior, but does not establish performance across AMD, ARM, Apple Silicon, heterogeneous scheduling, or future cache designs.

The metrics require careful interpretation. ASR is produced by an LLM judge with prompts that explicitly count topic, intent, structure, style, and key points as success. That criterion is appropriate for measuring privacy leakage, but it is less stringent than exact reconstruction and may be sensitive to judge bias. The paper reports exact-match and edit-distance statistics separately, yet the highest ASR values should not be read as plaintext recovery rates.

Finally, the reconstruction model is process- and deployment-specific. ASLR, allocation differences, and randomized hash-table layouts prevent straightforward transfer across independent instances. The attacker must profile the same long-lived process or re-profile after a restart. How much profiling is required when prompts are unknown, outputs are highly domain-specific, decoding uses temperature or nucleus sampling, or the server multiplexes many concurrent requests remains incompletely characterized.

Conclusion

The paper demonstrates that CPU-side detokenization can expose a practical cache side channel in local LLM deployments. Flush+Reload provides token-event synchronization, L1 Prime+Probe captures noisy decode-table activity, and contrastive clustering followed by language-model reconstruction converts ambiguous traces into semantically informative outputs. Controlled experiments achieve high semantic recovery across models, frameworks, hardware, natural-language domains, and code, while an OpenClaw experiment confirms end-to-end feasibility with lower accuracy.

The principal security result is that a default, model-agnostic inference component can leak generated content even when model data and GPU memory are inaccessible. The remaining technical question is whether tokenizer and runtime designs can preserve efficient streaming detokenization while preventing token-dependent cache behavior from becoming a stable, profileable signal.