SenCache: Sensitivity-Aware Caching

• SenCache is a design pattern that modulates caching by using explicit sensitivity signals, replacing traditional uniform reuse heuristics.
• It applies structured sensitivity models across diverse domains such as diffusion inference, LLM serving, and runtime memory management to determine when cached information is safe and valuable.
• Empirical results demonstrate significant improvements in compute efficiency, error reduction, and privacy protection compared to conventional caching methods.

Searching arXiv for the cited papers and related uses of “Sensitivity-Aware Caching” to ground the article. Sensitivity-Aware Caching (SenCache) denotes a class of caching strategies that modulate cache reuse, retention, or eviction according to an explicit sensitivity signal rather than treating cached states as uniformly reusable or uniformly valuable. In recent arXiv literature, the term is used in several technically distinct ways: as a dynamic per-sample reuse policy for diffusion and video denoisers based on output sensitivity to perturbations in the noisy latent and timestep (Haghighi et al., 27 Feb 2026); as a unified Diffusion Transformer framework that models temporal, depth, and feature sensitivity through Timestep-Aware Dynamic Scheduling, Cumulative Error Budgets, and Frequency-Decomposed Caching (Li, 5 Mar 2026); as a token-level, privacy-aware Key-Value cache sharing mechanism in LLM serving that reuses only non-sensitive KV entries across users (Wu et al., 22 May 2026); and, in the context of prefix caches, as semantic-adaptive eviction that learns how sensitive cache value is to token type, position, and session timing (Fang et al., 12 May 2026). A plausible antecedent appears in prioritized garbage collection, where caches become sensitive to memory availability and application-assigned priorities through GC-enforced bounds and priority references (Nunez et al., 2016).

1. Terminological scope and common design principle

The literature suggests that SenCache is best understood as a design pattern rather than a single algorithm. Across diffusion sampling, LLM serving, prefix-cache eviction, and managed-runtime software caches, the shared departure from conventional caching is explicit modeling of non-uniformity. Existing baselines in these domains typically assume uniform reuse eligibility, uniform approximation tolerance, or uniform eviction value. SenCache variants instead introduce structured signals that determine when cached state is safe to reuse, which entries are worth retaining, and which reuse decisions are too risky.

A common misconception is that “sensitivity” refers only to privacy. In fact, the cited works use the term for several different quantities. In diffusion inference, sensitivity refers to output response to perturbations in $\mathbf{x}_t$ and $t$, or to temporal, depth, and spectral error accumulation (Haghighi et al., 27 Feb 2026); (Li, 5 Mar 2026). In LLM serving, sensitivity refers either to privacy labels over prompt tokens that gate cross-user KV reuse, or to semantic, positional, and timing cues that determine eviction priority under GPU memory pressure (Wu et al., 22 May 2026); (Fang et al., 12 May 2026). In managed runtimes, caches are sensitive to memory pressure and programmer-defined priority values, with the collector enforcing a target bound in bytes or as a heap fraction (Nunez et al., 2016).

This diversity also clarifies why SenCache papers differ in what is being cached. The cached object may be a denoiser output tensor, a whole-backbone residual, token-level KV entries, fixed-size KV blocks, or ordinary heap objects reachable through reference types. The unifying idea is not the cache substrate but the use of an explicit sensitivity model to replace coarse heuristics such as unrestricted sharing, uniform thresholds, entry-count caps, LRU, or LFU.

2. Dynamic sensitivity control in diffusion and video denoising

In the video diffusion formulation of SenCache, the denoiser is treated generically as $f_\theta(\mathbf{x}_t,t,c)$, and caching means storing a previously computed output $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$ and reusing it at nearby timesteps instead of recomputing the model (Haghighi et al., 27 Feb 2026). The central quantity is the local output discrepancy

$$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$

Using a first-order Taylor expansion, the paper defines sensitivities

$$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$

and derives the bound

$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$

The operational decision score is

$$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$

If $S(t;t') \le \tau$, the cached output is reused; otherwise the denoiser is recomputed and the cache is refreshed. To prevent drift, the method also limits the maximum consecutive reuses by $n$.

This formulation is explicitly per-sample and trajectory-aware. Rather than relying on a fixed set of cache timesteps, it uses the actual latent drift produced by the sampler and a precomputed timestep-indexed sensitivity schedule $t$0. Direct Jacobian computation is avoided through projector-free finite-difference surrogates calibrated offline. The paper reports that as few as 8 diverse videos suffice to produce sensitivity profiles comparable to those obtained with 4096 samples, which implies low calibration overhead. It also recommends strict thresholds for early timesteps, including setting $t$1 to allow $t$2 error for the first 20% of timesteps, and constraining $t$3 to $t$4.

Empirically, the method is evaluated on Wan 2.1, CogVideoX, and LTX-Video with $t$5. Under matched compute budgets, SenCache preserves or improves visual quality relative to TeaCache and MagCache. On Wan 2.1 in the fast regime, SenCache uses NFE 21 and cache ratio 58%, with LPIPS 0.0540, PSNR 29.1400, and SSIM 0.9219, compared with MagCache-fast at the same NFE/cache ratio with LPIPS 0.0603, PSNR 28.3684, and SSIM 0.9143, and TeaCache-fast at NFE 25 and cache ratio 50% with LPIPS 0.0966, PSNR 25.0661, and SSIM 0.8697 (Haghighi et al., 27 Feb 2026). On Wan 2.1 running on GH200, vanilla inference takes 182.3 s, SenCache 107.3 s, and MagCache 110.6 s; total compute is reduced from 8,244,043.09 to 3,482,412.58 GFLOPs. The paper characterizes the method as sampler-agnostic, model-agnostic, and training-free.

3. Tri-axis sensitivity in Diffusion Transformers

A second diffusion-oriented formulation appears in "Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers," where SpectralCache is presented as a realization of SenCache for DiTs (Li, 5 Mar 2026). Here the cached object is the whole-block residual at timestep $t$6,

$t$7

and on a cache hit the model approximates the stack output by

$t$8

This bypasses repeated attention, FFN, and normalization computations across all blocks.

The framework isolates three orthogonal sensitivity axes. Temporal sensitivity is measured by forcing reuse at a single timestep while fully computing other steps; the resulting $t$9 error curve follows an asymmetric U-shape, with large error at early steps, a minimum in the middle, and a moderate rise near the end. Depth sensitivity captures residual compounding under consecutive reuse; the paper reports that consecutive cached blocks yield 20–30% higher error than randomly distributed caching at matched rates. Feature sensitivity is established by a 2D DCT analysis of hidden states, which shows that low-frequency bands have about 30% higher volatility than high-frequency bands, with $f_\theta(\mathbf{x}_t,t,c)$0 versus $f_\theta(\mathbf{x}_t,t,c)$1.

These observations are operationalized by three coupled mechanisms. Timestep-Aware Dynamic Scheduling modulates a base threshold $f_\theta(\mathbf{x}_t,t,c)$2 using a cosine bell schedule,

$f_\theta(\mathbf{x}_t,t,c)$3

making caching conservative near endpoints and aggressive in the middle. Cumulative Error Budgets maintain a consecutive-cache counter $f_\theta(\mathbf{x}_t,t,c)$4 and accumulated distance budget $f_\theta(\mathbf{x}_t,t,c)$5, enforcing both $f_\theta(\mathbf{x}_t,t,c)$6 and $f_\theta(\mathbf{x}_t,t,c)$7. Frequency-Decomposed Caching splits the modulated input into low- and high-band channel groups and applies asymmetric thresholds, with stricter gating on the low-frequency band and looser gating on the high-frequency band. The reported defaults for FLUX.1-schnell at $f_\theta(\mathbf{x}_t,t,c)$8 are $f_\theta(\mathbf{x}_t,t,c)$9, $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$0, $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$1, $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$2, $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$3, and $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$4.

The main quantitative result is on FLUX.1-schnell at $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$5 and 20 steps on A100. No Cache requires 4.24 s. TeaCache runs in 2.00 s with 2.12$y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$6 speedup, LPIPS 0.215, and SSIM 0.734. SpectralCache runs in 1.72 s with 2.46$y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$7 speedup, LPIPS 0.217, and SSIM 0.727. The paper states that this is 16% higher speed than TeaCache with comparable quality, with LPIPS difference below 1% (Li, 5 Mar 2026). Threshold sweeps also show a smooth speed–quality trade-off: at $y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$8, speedup is 1.53$y_{t'} = f_\theta(\mathbf{x}_{t'},t',c)$9 with LPIPS 0.139 and SSIM 0.808; at $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$0, speedup is 2.46$$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$1 with LPIPS 0.217 and SSIM 0.727. The control overhead is described as negligible relative to transformer compute and below 0.5% runtime.

4. Privacy-aware token-level reuse in LLM KV caches

In LLM serving, SenCache refers to the sensitivity-aware caching mechanism inside CachePrune, a token-level privacy-aware KV cache sharing system for cross-user inference (Wu et al., 22 May 2026). The problem is a reuse-based side channel: if a probing request matches previously cached KV entries, the adversary can observe lower latency or scheduling benefits. The paper formalizes this with a binary reuse oracle $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$2, where $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$3 iff a probe prompt $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$4 triggers reuse, and models a prompt as $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$5 with binary sensitivity mask $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$6, where $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$7 iff token $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$8 is sensitive.

The selective sharing rule is exact: “KV entries for token $$E(t,t') = \big\| f_\theta(\mathbf{x}_t,t,c) - f_\theta(\mathbf{x}_{t'},t',c) \big\|_2.$$9 are reused across users if and only if $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$0, while tokens with $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$1 are excluded from cross-user sharing.” Sensitive tokens therefore act as separators that partition the prompt into variable-length non-sensitive segments. CachePrune does not stop at sentence- or chunk-level masking. It derives reusable substrings within these segments using the self-contextualization criterion from CacheCraft, reusing a substring $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$2 when its intra-attention exceeds inter-attention. A naive exploration of all substrings requires $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$3 total time. The paper reduces this to $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$4 by constructing an in-place summed area table on the final-layer attention matrix, enabling $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$5 evaluation of both intra- and inter-attention for any candidate span. It then marks the top $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$6 of tokens by inter-minus-intra difference for recomputation, with default $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$7, and discards segments shorter than a default minimum of 128 tokens.

Retrieval is handled by a two-phase rolling-hash pipeline over variable-length spans. First, a 128-token prefix hash is precomputed for each cached segment, and a rolling polynomial hash modulo the 64-bit Mersenne prime $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$8 is slid over the new request in $$S_x(t') = \big\| J_x(\mathbf{x}_{t'},t',c) \big\|_{\mathrm{op}}, \qquad S_t(t') = \big\| \partial_t f_\theta(\mathbf{x}_{t'},t',c) \big\|_2,$$9. Second, candidate matches are verified with substring rolling hashes and then by SHA-256, eliminating collisions without token-by-token comparison. CPU-side metadata stores span indices, hashes, and GPU references; GPU-side KV tensors remain resident. The implementation is layered over vLLM, leaves the LLM Engine unmodified, and remains compatible with paged attention and batching abstractions.

The security claim is formulated within the reuse-oracle threat model: because tokens with $$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$0 never generate reuse signals, direct leakage through KV cache reuse side channels is eliminated by design. Empirically, the “Defense effectiveness of CachePrune” table reports Direct Recovery of 0% across QASPER, NarrativeQA, and QMSum. Contextual Leakage remains nonzero but limited: QASPER 3.21% exact and 9.48% semantic, NarrativeQA 2.90% and 5.88%, and QMSum 0.50% and 3.98%, averaging about 2.2% exact and 6.4% semantic. End-to-end evaluation with Mistral-7B shows hit rates of 82.88% on QASPER, 56.45% on NarrativeQA, and 94.57% on QMSum, with TTFT reductions from 797 ms to 273 ms, from 1675 ms to 705 ms, and from 1554 ms to 342 ms respectively. The paper summarizes the overall effect as eliminating direct leakage while reducing TTFT by up to 4.5$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$1 and increasing cache hit rates by 44% over fixed-chunk baselines (Wu et al., 22 May 2026).

5. Semantic- and timing-sensitive eviction in LLM prefix caches

In "Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches," the paper states that “Sensitivity-Aware Caching (SenCache)” is not explicitly named, but that SAECache embodies sensitivity-aware caching through semantic sensitivity, structural sensitivity, and timing sensitivity (Fang et al., 12 May 2026). The setting is prefix caching for LLM prefill, where fixed-size KV blocks are reused across requests with shared prefixes. TTFT is linked to the effective prefill length through

$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$2

The paper argues that conventional LRU, LFU, and session-level scoring are inadequate because token types differ radically in reuse value. It reports system prompts at 92.3% reuse versus chain-of-thought at 2.2% in one table, and a figure caption states “up to 756$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$3 variation” between system prompts and chain-of-thought.

SAECache addresses this with a multi-queue architecture. The evict-first queue contains decode-phase blocks and low-reuse untemplated prefill blocks. The structural queue contains templated single-turn prefill blocks and scores them by position decay,

$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$4

The chat and agentic queues model multi-turn session reuse through a log-normal survival probability,

$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$5

Token-type sensitivity is represented by online-learned weights $$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$6, and queue-level efficiency by learned weights $$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$7. The global retention score is

$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$8

Eviction removes the minimum-$$E(t,t') \le S_x(t')\,\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2 + S_t(t')\,|t - t'| + \mathcal{O}\!\left(\big\| \mathbf{x}_t - \mathbf{x}_{t'} \big\|_2^2 + |t - t'|^2\right).$$9 block after draining the evict-first queue.

The timing model is itself workload-sensitive. For chat traffic in Qwen-Bailian, the paper reports heavy-tailed pauses with $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$0 s, $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$1 s, and $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$2 s; the log-normal fit has $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$3 and $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$4-$$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$5 $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$6. For agentic traffic in CC-Bench, it reports $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$7 s and $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$8 s, with $$S(t;t') = S_x(t')\,\big\|\mathbf{x}_t - \mathbf{x}_{t'}\big\|_2 + S_t(t')\,|t - t'|.$$9 and $S(t;t') \le \tau$0-$S(t;t') \le \tau$1 $S(t;t') \le \tau$2. All parameters are updated online by EMA-style rules with clamps, including $S(t;t') \le \tau$3, $S(t;t') \le \tau$4, $S(t;t') \le \tau$5, $S(t;t') \le \tau$6, and the token-type weights. The paper emphasizes that this avoids the fixed-parameter brittleness observed when chat-fitted parameters are transferred to single-turn-dominated workloads.

The evaluation uses ShareGPT, LMSys, and Chatbot-Arena on vLLM v0.8.5 with Qwen2.5-1.5B-Instruct on NVIDIA A40. SAECache achieves the highest hit ratio in all 12 dataset-interval cells, improving hit ratio by 4.8 to 5.9 percentage points over the strongest baseline. The abstract reports 1.4$S(t;t') \le \tau$7–2.7$S(t;t') \le \tau$8 TTFT improvement over production-style baselines, while fixed-parameter alternatives can degrade by up to 2.7$S(t;t') \le \tau$9 under workload mismatch. Ablations attribute +23% improvement to token-type weight learning, +14% to log-normal timing learning, +12% to position-decay learning, and +39% to queue-weight learning, with +88% combined improvement over the Fixed-Param multi-queue baseline. The learned token ordering converges to $n$0, with example values 4.13, 2.00, 1.86, 1.80, and 0.77. A notable caveat is that in Chatbot-Arena, where only 12% of requests are multi-turn, SAECache has the highest hit ratio but not always the lowest TTFT because multi-queue bookkeeping and predictor overhead can dominate in single-turn-dominant regimes (Fang et al., 12 May 2026).

6. Memory-sensitive cache management in managed runtimes

A broader systems interpretation of sensitivity-aware caching appears in prioritized garbage collection, which makes software caches sensitive to memory availability and application priorities through explicit runtime support (Nunez et al., 2016). The mechanism introduces PrioReference, a reference type with a programmable priority, and PrioSpace, a grouping abstraction with a target bound $n$1 specified either in bytes or as a fraction of heap or free space. At GC time, the collector processes references in descending priority order, marking reachable objects and computing a running footprint. Once the bound is reached, remaining lower-priority references are cleared and their referents are reclaimed during sweep.

The paper formalizes the bounded keeping policy over a space $n$2 by referents $n$3 with priorities $n$4 and sizes $n$5, where sizes are computed over the transitive closure reachable only through the space. Operationally, the collector marks in nonincreasing $n$6 until the cumulative size reaches $n$7, then clears the rest. An optional adaptive reserve computes

$n$8

where $n$9 is heap size, $t$00 is live bytes not in the cache space, and $t$01 is the desired free-memory reserve. This converts a cache into a memory-pressure-aware object whose size automatically expands or contracts at GC time.

The accompanying Sache cache uses HashMap<K, PrioReference<V>> and maps LRU recency to priority through a monotonically increasing counter. Unlike manual eviction, this keeps reclamation synchronized with GC and avoids the mismatch between mutator-side evictions and actual memory recovery. The paper contrasts this with soft references, whose policy is global and hard-wired, and with entry-count-based caches such as Guava, whose provisioning is brittle when entry sizes vary. It reports that configuring Sache to 40% of heap delivers robust performance across small, medium, and large object workloads, whereas Guava requires workload-specific entry caps and can encounter out-of-memory failures on larger values. In a two-cache experiment with a 10:1 access-frequency ratio, HotSpot soft references reduce the slower cache’s hit rate to about 25% of its potential, while two independent PrioSpaces keep the hit-rate difference within about 5%. Under increasing non-cache memory pressure, Guava crashes at about 77MB, whereas Sache maintains stable GC time and preserves a 50% free reserve (Nunez et al., 2016).

Taken together, these mechanisms show that SenCache is not confined to approximation-aware reuse. The same design logic appears when a cache is made sensitive to privacy labels, denoising error, semantic reuse value, session timing, spectral volatility, or GC-visible memory pressure. The term therefore denotes a family of non-uniform caching policies whose central claim is that efficient caching requires explicit modeling of what is safe, valuable, or stable enough to retain or reuse.