- The paper demonstrates cross-account prompt-cache sharing through OpenRouter’s default pooled credentials at all three tested providers, including a 100% hit rate at Groq and highly significant timing evidence at Fireworks (p = 4.1 × 10⁻¹⁵).
- The study combines interleaved time-to-first-token measurements with cached-token metadata across 18 configurations and approximately 9,000 requests, distinguishing gateway credential pooling from provider-side isolation failures.
- The findings show that direct provider access and OpenRouter’s BYOK mode preserve isolation, while gateways should implement per-tenant cache namespaces and users should avoid sending sensitive prompts through shared-credential routing.
CacheProbe is a systematic audit of whether OpenRouter, a widely used LLM API gateway, inadvertently defeats the per-organization prompt cache isolation guarantees that inference providers have implemented. Building on the timing-based cache auditing methodology of Gu et al. (Gu et al., 11 Feb 2025), the paper demonstrates that routing requests through OpenRouter's default credential architecture produces detectable cross-account cache sharing at all three providers tested, while direct API access and OpenRouter's BYOK mode preserve isolation.
Background and motivation
Prompt caching exploits the observation that many LLM requests share long common prefixes, particularly in multi-turn chat workloads. Providers store the KV attention states for matched prefixes and compute attention only for novel suffix tokens, yielding speedups of 1.5–10x on GPU inference and 20–70x on CPU inference (Gim et al., 2023). Because cached input tokens are substantially cheaper — OpenAI and Google both offer 90% discounts on cached tokens — caching is now ubiquitous across inference APIs.
The security concern, formalized by Gu et al., is that TTFT differences between cache hits and misses constitute a timing side-channel: an attacker can probe candidate prefixes and reconstruct cached prompts token-by-token via binary search. In response, major providers implemented per-account or per-organization cache isolation. The paper identifies an architectural gap in this mitigation: API gateways such as OpenRouter authenticate to upstream providers using their own shared organizational credentials. From the provider's perspective, all OpenRouter traffic originates from a single organization, so provider-level isolation does not separate individual OpenRouter users from one another.
Threat model
The adversary is any authenticated OpenRouter user with credits, who can force routing to a specific upstream provider via the provider.order parameter and measure TTFT for arbitrary prompts. No internal access to OpenRouter infrastructure or victim credentials is required. Detection relies on two complementary signals: TTFT distribution differences between hit and miss conditions, and explicit metadata disclosure via the cached_tokens field in usage responses.
Six scenarios isolate the source of sharing by crossing three routing paths (direct API, OpenRouter default credentials, OpenRouter BYOK) with same-account versus cross-account configurations. This design allows the paper to attribute any observed sharing to the gateway's credential pooling rather than to provider-side failures.
Methodology
The experimental procedure uses interleaved sampling to control for network latency and load fluctuations: each sample pair consists of a unique random-prompt miss baseline, a simulated victim priming the cache with a different random prompt, and an attacker probe sharing a prefix with the victim's prompt. Prompts are 4,096 tokens of space-separated ASCII letters (one token per letter), with a 95% prefix fraction — the probe shares 3,891 tokens and differs in the final 205, modeling partial prefix reconstruction rather than exact duplication. A 500 ms inter-request delay balances cache persistence against rate limiting.
Timing analysis uses two-sample Kolmogorov–Smirnov tests on TTFT distributions with a stringent significance threshold of p<10−8. Metadata analysis flags detection when more than 50% of cache-primed requests report cached tokens exceeding 90% of the expected prefix length. The combined criterion treats either signal as positive. Experiments ran between November 12 and December 8, 2025 on an Oracle Cloud VM in US-East, covering three providers (OpenAI/gpt-4o-mini, Groq/openai-gpt-oss-20b, Fireworks/qwen3-8b), six scenarios each, 250 sample pairs per configuration — approximately 9,000 API requests total.
Results
Direct API baseline
All three providers demonstrate correct cross-account isolation when accessed directly. Same-account tests confirm caching functions: Groq and Fireworks show statistically significant timing differences (p=3.8×10−11 for both), while OpenAI discloses cached tokens via metadata despite non-significant timing (p=0.13). Cross-account tests show no evidence of sharing anywhere, with p-values from 0.13 to 0.69 and no cross-account cached-token reports. An anomalous detail worth noting: Groq's same-account hits exhibited higher mean TTFT than misses (0.766 s vs. 0.460 s) even though metadata confirmed functional caching, suggesting timing alone can be an unreliable detector for some providers.
OpenRouter default mode
Cross-account cache sharing is detected for all three providers under shared organizational credentials:
| Provider |
Signal |
Cross-account finding |
| Groq |
Metadata |
100% cache hit rate; all reported samples exceeded the 90% prefix threshold |
| Fireworks |
Timing |
p=4.1×10−15, the strongest statistical evidence in the study |
| OpenAI |
Metadata |
4.8% of cross-account requests report cached tokens above threshold |
Groq's result is the most severe: every cross-account probe that reported cached tokens exceeded the prefix threshold, meaning prompts submitted by one OpenRouter user were served from cache to unrelated users. Fireworks provides no cached-token metadata, making the highly significant KS statistic the sole but decisive evidence. OpenAI's leakage is minor by comparison, but any cross-account hits on randomly generated prompts indicate incomplete isolation.
BYOK mode restores isolation
When users supply their own provider keys via BYOK, cross-account tests show no significant timing differences (p>0.16 everywhere) and no metadata-based sharing. This attribution is the paper's central causal claim: the vulnerability stems specifically from OpenRouter's pooled organizational credentials, not from its routing layer or from provider implementations.
Security implications
Because the attack requires only a standard OpenRouter account with credits, any user can probe the shared cache namespace. Practical exploitation includes system prompt extraction — reconstructing proprietary instructions, RAG retrieval patterns, or business logic of applications built on OpenRouter — as well as usage-pattern detection enabling competitive intelligence without full reconstruction, and exposure of sensitive data embedded in prompts such as PII or proprietary documents.
The paper frames this as a shared responsibility gap: providers correctly implemented organization-level isolation (validated by the direct baseline), but that isolation assumes organizational credentials constitute a trust boundary — an assumption gateways violate by construction. Neither party may recognize the resulting vulnerability without explicit testing, since providers see intentional intra-organizational sharing and gateways may lack visibility into provider caching behavior. The pattern generalizes beyond OpenRouter to any aggregation service proxying upstream requests with shared credentials, including enterprise API management platforms and custom proxies.
Mitigations
The paper proposes true per-tenant cache namespace isolation, where gateways forward a unique per-user identifier that providers use to partition cache entries — preserving caching benefits while restoring isolation. Partial mitigations exist: OpenAI's prompt_cache_key parameter influences cache-server assignment, so distinct keys per user probabilistically reduce collisions, but it is a routing hint rather than a namespace guarantee, and collisions remain possible with a finite server pool. Users seeking immediate protection can adopt BYOK mode, confirmed effective here, at the cost of OpenRouter's convenience and pricing advantages.
Limitations
Several constraints bound the findings. Only three providers and one model each were tested; isolation behavior may differ elsewhere on OpenRouter. Prompts were randomly generated rather than drawn from realistic application traffic, so production cache-hit patterns may differ. Cache eviction policies and timing vary with provider load and time of day, factors not controlled for. Finally, the conservative statistical thresholds may miss partial or intermittent leakage. A further practical caveat is the disclosure record: OpenRouter acknowledged the initial report but did not respond after receiving repository access, so remediation status at publication time is unclear.
Conclusion
This paper establishes, through an 18-configuration, ~9,000-request audit, that OpenRouter's default shared-credential architecture enables cross-account prompt cache sharing at all three providers tested — up to a 100% cross-account hit rate at Groq and p=4.1×10−15 timing evidence at Fireworks — while direct access and BYOK mode preserve isolation. The results reframe cache isolation as a property that must hold end-to-end across intermediary services, not merely within providers, and give OpenRouter users concrete guidance: treat prompts routed through default-mode gateways as potentially observable by other users, or switch to BYOK.