Adaptive Sparse Attention Module (ASAM)

• ASAM is a sparse attention mechanism that uses cumulative attention mass to dynamically select tokens in long-context LLM inference.
• It employs a two-stage pipeline with clustering-based sorting and distribution fitting to approximate token rankings efficiently.
• ASAM adapts per head, layer, and query during decoding, enhancing inference speed while maintaining output quality.

Searching arXiv for the cited papers and closely related ASAM terminology. {"^^^^^^^^2query2 OR title:\2^^^^^^^^"Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs\"","max_results":5} {"^^^^^^^^2query2 Sparse Attention Module\" arXiv long-context attention ASAM","max_results":^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2query2^^^^} {"^^^^^^^^2query2 OR title:\2^^^^&&&) AdaSplash adaptive sparse flash attention arXiv","max_results":5} Adaptive Sparse Attention Module (ASAM) denotes a class of sparse-attention mechanisms that replace dense, fixed-pattern, or fixed-budget attention with input-dependent selection rules. In the specific formulation described in "Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs," ASAM is a decode-phase mechanism for long-context LLM inference that selects tokens by cumulative attention mass rather than by a predetermined token count, and approximates that selection through clustering-based sorting and distribution fitting (&&&^^^^2query2^^^^&&&). More broadly, the same label, or closely related “ASAM-like” formulations, has been used for distinct modules in long-context reasoning, long-video MLLMs, video diffusion, and efficient entmax-based transformers, with the common objective of making sparsity adaptive to heads, layers, queries, contexts, or motion regimes rather than statically imposed (You et al., 2 Oct 2025).

^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^. Definition and problem setting

In Tactic, ASAM is introduced for the decode phase of long-context LLM inference, where repeatedly attending over a very large KV cache becomes the dominant systems bottleneck. The motivating observation is that in long contexts, loading K/V vectors can take over 5^^^^2query2^^^^% of decode time, while prior sparse-attention or KV-compression methods typically enforce a fixed token budget per head or layer (&&&^^^^2query2^^^^&&&). Tactic argues that this assumption is mismatched to actual attention behavior because sparsity varies across heads, across layers, and across ^^^^^^^^2query2^^^^^^^^ tokens and contexts. Some heads are described as “retrieval” heads with broader attention, whereas others are “streaming” heads with very peaky attention; similarly, earlier and later layers exhibit different sparsity levels, and the same head can become denser or sparser depending on the token being generated.

Within that framing, ASAM replaces the fixed-budget objective with a target cumulative attention score. This shifts the question from “how many tokens should be kept?” to “how much of the attention probability mass should be preserved?” The module is explicitly described as sparsity-adaptive and calibration-free, and its adaptation is performed online from the current ^^^^^^^^2query2 attention pattern rather than from manual layer/head budget assignment or calibration data (&&&^^^^2query2^^^^&&&).

A related but broader use of the name appears in ReSSFormer, where ASAM is one of three architectural components and combines token-level sparsity, activation sparsity through sparsemax or entmax, and expert-level sparsity through sparse routing. There the emphasis is not KV-cache loading during autoregressive decode, but replacing dense self-attention with focused, sparse computation for long-context reasoning (You et al., 2 Oct 2025). This suggests that “ASAM” has become a general label for adaptive sparse attention subsystems, but the Tactic formulation is specifically an online, ^^^^^^^^2query2 attention-mass-controlled selection mechanism for long-context LLM decoding.

2. Attention-mass criterion and approximation guarantee

For one decode step with ^^^^^^^^2query2^^^^^^^^ PRESERVED_PLACEHOLDER_^^^^2query2^^^^ and previous tokens PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^, Tactic writes full attention as

$$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$

Given a selected token set $I$, the cumulative attention score is defined as

$$p(I)=\sum_{i\in I}s_i =\frac{\sum_{i\in I}\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)} {\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$

ASAM then selects the smallest set of tokens whose cumulative attention mass reaches a target fraction $P$, i.e. the minimal prefix $I$ in descending attention order such that

$p(I)\ge P.$

The target $P$ is a fraction of total attention probability mass, not a fraction of tokens. If attention is concentrated, only a few tokens are required; if it is spread out, more tokens are selected. In that sense, the token count is a consequence of the attention distribution rather than an externally fixed prior (&&&^^^^2query2^^^^&&&).

The paper also states an approximation bound under the assumption that value vectors have similar norms across tokens: $$\epsilon(I)=\|o-\tilde{o}(I)\| \leq 2(1-p(I))\max_i\|v_i\|.$$ Accordingly, when PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2query2^^^^ is close to ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^, the sparse output is provably close to full attention. This is the main formal argument for preferring cumulative attention mass over fixed token budgets: the criterion is directly tied to approximation quality rather than indirectly controlled through a budget heuristic (&&&^^^^2query2^^^^&&&).

A conceptually related criterion appears in OmniSparse, where KV budget selection is also expressed through retained attention mass, though in a different architecture and with a different allocation rule. There, the budget PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^ is chosen from the flattest head as the minimal number of keys whose cumulative accumulated attention reaches PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\22^^^^, and the paper cites the theoretical error upper bound PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^3 (Chen et al., 15 Nov 2025). The shared theme is that adaptive sparse attention can be parameterized by retained mass or recall instead of by a fixed top-PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^4.

3. Tactic ASAM pipeline: clustering-based sorting and distribution fitting

The core difficulty in Tactic is that identifying the minimal set satisfying PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^5 would ordinarily require both the exact ranking of token importance and the cumulative sum of exact attention scores. ASAM approximates both with a two-stage pipeline: clustering-based sorting followed by distribution fitting (&&&^^^^2query2^^^^&&&).

During prefill, for each head in each layer, Tactic runs K-means clustering on key vectors PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^6, grouping tokens by key-vector similarity rather than position. The K-means objective is

PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^7

where PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^8 is a cluster centroid. The motivation is that positional grouping is unreliable because consecutive tokens can have very different key vectors while far-apart tokens can be semantically similar. At decode time, the ^^^^^^^^2query2^^^^^^^^ PRESERVED_PLACEHOLDER_^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^9 is scored against each cluster centroid using $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$^^^^2query2^^^^, clusters are sorted by that score, and tokens are unpacked in cluster order to form a partially sorted list. The paper reports an average cluster size of about 32 and states that clustering overhead stays below 6% of prefill time (&&&^^^^2query2^^^^&&&).

Clustering only approximates the order; it does not recover the absolute attention values needed for cumulative-mass stopping. Tactic therefore fits a simple long-tail curve after sorting: $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^ where $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$2 is token rank and $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$3 estimates attention score magnitude. The parameters $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$4 are fit using two sampled windows in the middle of the sorted sequence, for example around ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2query2^^^^% and 6^^^^2query2^^^^% of tokens, by computing mean exponential attention values $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$5. The first ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^–2% of tokens are treated as outliers: their exact $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$6 values are computed directly, and the fitted curve is used for the remainder (&&&^^^^2query2^^^^&&&).

The appendix procedure summarized in the paper is: fit $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$7; define

$$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$8

and choose the smallest $$o = \sum_{i=1}^{n}s_i v_i,\qquad s_i=\frac{\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)}{\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$9 such that

$I$^^^^2query2^^^^

The returned $I$^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^ is the token budget for that head. In operational terms, ASAM estimates the entire descending attention curve, accumulates estimated mass, and stops at the prescribed attention fraction $I$2 rather than at a fixed top-$I$3 (&&&^^^^2query2^^^^&&&).

4. Adaptation across heads, layers, contexts, and grouped-^^^^^^^^2query2^^^^^^^^ attention

The adaptivity of Tactic’s ASAM is explicitly multi-level. Token selection is performed per head using that head’s ^^^^^^^^2query2^^^^^^^^ and keys, so different heads can produce different budgets and selected sets. Clustering is also done for each head in each layer during prefill, which captures layer-specific key structure. Because the ^^^^^^^^2query2^^^^^^^^ token changes at every decode step, the ranking and estimated cumulative mass are re-evaluated online, so the selected tokens can also change with context and generation state (&&&^^^^2query2^^^^&&&).

This per-head, per-layer, per-step adaptivity is what the paper contrasts with static allocation methods such as PyramidKV or Ada-KV. The central claim is not merely that ASAM is sparse, but that the sparsity level is allowed to vary with the actual attention distribution instead of being prescribed by calibration tables or manual budget schedules (&&&^^^^2query2^^^^&&&).

For Grouped-Query Attention, where multiple ^^^^^^^^2query2^^^^^^^^ heads share one KV head, Tactic computes selected tokens per ^^^^^^^^2query2^^^^^^^^ head and then takes the union of selected tokens for the shared KV head. The paper notes that finding the exact minimum union satisfying all heads is NP-hard, so the union is used as a practical approximation. It further reports that taking the union across GQA heads can yield up to ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^.65× attention speedup versus processing ^^^^^^^^2query2^^^^^^^^ heads separately (&&&^^^^2query2^^^^&&&).

The importance of head heterogeneity appears across several later adaptive sparse-attention papers. OmniSparse determines a shared KV budget by analyzing the flattest head via kurtosis, then lets each head select its own top-$I$4 keys under that shared budget (Chen et al., 15 Nov 2025). HASTE assigns per-head top-$I$5 thresholds offline through Error-guided Budgeted Calibration and decides per-head mask reuse online through ^^^^^^^^2query2 drift, arguing that mask stability and threshold-response curves vary substantially across heads (&&&^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^8&&&). AdaSpa likewise states that DiT sparse patterns vary with inputs, layers, and heads, motivating head-adaptive hierarchical block sparse attention (&&&^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^9&&&). Taken together, these formulations support a broader interpretation of adaptive sparse attention as an answer to head- and layer-level heterogeneity rather than a single universal sparsity pattern.

5. Empirical behavior in Tactic

Tactic reports that ASAM consistently outperforms fixed-budget sparse methods including Quest, PyramidKV, and Ada-KV / Ada-SnapKV. On LongBench, it achieves the best scores across HotpotQA, TriviaQA, MultiFieldQA, Qasper, NarrativeQA, and Musique, and at both 7^^^^2query2^^^^% and 9^^^^2query2^^^^% cumulative-attention thresholds it is better than the baselines, often much better at the same approximate token budget. On RULER, it again outperforms baselines across context lengths ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^6K–96K, and at 9^^^^2query2^^^^% threshold its accuracy is described as close to full attention and clearly better than other sparse methods (&&&^^^^2query2^^^^&&&).

The paper also evaluates attention approximation quality by comparing the global optimal token selection order, the cluster optimal order, and Tactic. It states that Tactic gets very close to the cluster-optimal token count and reaches the desired cumulative attention threshold with high success rates. The reported success rates are 92% at 5^^^^2query2^^^^% threshold, 89% at 6^^^^2query2^^^^%, 86% at 7^^^^2query2^^^^%, 84% at 8^^^^2query2^^^^%, and 86% at 9^^^^2query2^^^^% (&&&^^^^2query2^^^^&&&).

For output-distribution fidelity, the paper reports that on PG^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^9, Tactic has the lowest KL-divergence from full attention among the compared methods, indicating that the sparse output distribution is closest to full attention. For efficiency, it reports up to 7.29× decode attention speedup and up to ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^.58× end-to-end inference speedup. These numbers are paired with the claim that clustering overhead is small relative to prefill and that distribution fitting and cluster sorting are lightweight in decode (&&&^^^^2query2^^^^&&&).

The broader sparse-attention literature represented in the supplied papers shows analogous but task-specific efficiency claims. Adamas reports up to 4.4× self-attention and ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^.5× end-to-end speedups on 32K-length sequences while matching full attention at a 64-token budget and being near-lossless at ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\22^^^^8 (Yan et al., 21 Oct 2025). OmniSparse reports comparable performance to full attention together with up to 2.7× speedup during prefill and 2.4× memory reduction during decoding (Chen et al., 15 Nov 2025). These are not directly comparable experimental settings, but they reinforce the recurring empirical pattern that adaptive selection is used to preserve output quality under aggressive sparsity.

6. Related ASAM formulations and conceptual boundaries

The label “Adaptive Sparse Attention Module” is not confined to Tactic. In ReSSFormer, ASAM is a module inside a larger recurrent architecture and combines hard top-$I$6 key selection, sparse activation mappings $I$7, and expert-level sparsity with $I$8 experts and $I$9 active experts per head. There the stated computational transition is $$p(I)=\sum_{i\in I}s_i =\frac{\sum_{i\in I}\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)} {\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$^^^^2query2^^^^, and ablation shows that removing ASAM changes Wikitext PPL from ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^7.4 to ^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^8.6 and HotpotQA EM from 7^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^.2 to 66.^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^ (You et al., 2 Oct 2025). This use of ASAM is architectural and end-to-end rather than an inference-only KV-selection mechanism.

OmniSparse presents an “ASAM-like adaptive sparse attention” in long-video MLLMs that operates in both training and inference. Its three mechanisms are ^^^^^^^^2query2^^^^^^^^ selection via lazy-active classification, KV selection with head-level dynamic budget allocation based on the flattest head, and KV cache slimming during decoding by skipping visual KV fetches for lazy heads. The paper explicitly frames its contribution as bridging the training–inference gap seen in inference-only sparsification (Chen et al., 15 Nov 2025). By contrast, Tactic is explicitly post-training and calibration-free, with a lightweight prefill-time clustering step that is not treated as training or calibration in the usual sense (&&&^^^^2query2^^^^&&&).

Other related mechanisms occupy additional points in the design space. AdaSplash makes $$p(I)=\sum_{i\in I}s_i =\frac{\sum_{i\in I}\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)} {\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^-entmax attention practical by a hybrid Halley-bisection solver and custom Triton kernels, using adaptive sparsity from the probability map itself rather than from explicit top-$$p(I)=\sum_{i\in I}s_i =\frac{\sum_{i\in I}\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)} {\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$2 token budgeting (&&&^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^&&&). Adamas is a training-free, token-level sparse attention method that applies the Hadamard transform, bucketization, 2-bit compression, Manhattan-distance estimation, and top-$$p(I)=\sum_{i\in I}s_i =\frac{\sum_{i\in I}\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)} {\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$3 token retrieval, and its adaptivity is ^^^^^^^^2query2 rather than learned by a gating network (Yan et al., 21 Oct 2025). AdaSpa, DynamicRad, and HASTE focus on video diffusion, where sparse patterns are described respectively as dynamic blockified patterns with online precise search, content-adaptive radial candidate sets with dual operating modes, and head-wise adaptive top-$$p(I)=\sum_{i\in I}s_i =\frac{\sum_{i\in I}\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)} {\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$4 control through Temporal Mask Reuse and Error-guided Budgeted Calibration (&&&^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^9&&&, &&&3^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^&&&, &&&^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^8&&&).

A common misconception is to treat all such modules as interchangeable under the single term ASAM. The supplied papers do not support that simplification. Tactic’s ASAM is centered on cumulative attention mass and clustering-plus-curve-fitting for long-context LLM decode (&&&^^^^2query2^^^^&&&); ReSSFormer’s ASAM is a sparse-attention submodule integrated with recurrent reasoning and structure induction (You et al., 2 Oct 2025); OmniSparse’s ASAM-like framework is training-aware and multi-dimensional (Chen et al., 15 Nov 2025). A plausible implication is that “ASAM” is best understood as a family resemblance term for adaptive sparse-attention designs rather than as a single canonical algorithm.

7. Strengths, limitations, and preferred operating regimes

For Tactic, the reported strengths are explicit: it is adaptive, calibration-free, theoretically grounded through the cumulative-attention approximation bound, efficient because clustering and curve fitting are cheap enough for decoding, accurate relative to fixed-budget methods, and compatible with GQA through the union strategy (&&&^^^^2query2^^^^&&&). The method is positioned for settings where long-context decode latency is a concern, some approximation is acceptable, and accuracy matters more than aggressive compression.

The limitations stated in the paper are equally specific. ASAM remains approximate because it relies on clustering and curve fitting rather than exact full attention. Prefill includes clustering overhead, though it is reported as small. The union strategy for GQA is a practical approximation rather than the exact global optimum. The method is also best suited to cases where attention score distributions are reasonably smooth so that curve fitting works well (&&&^^^^2query2^^^^&&&).

Across the related literature, analogous trade-offs recur in different forms. AdaSplash notes that adaptive but unstructured sparsity depends on the amount of learned sparsity and that the forward pass can be slower than FlashAttention-2 before sparsity is high enough to skip blocks (&&&^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^&&&). HASTE emphasizes that even training-free top-$$p(I)=\sum_{i\in I}s_i =\frac{\sum_{i\in I}\exp\left(\frac{qk_i^\top}{\sqrt{d}}\right)} {\sum_{j=1}^{n}\exp\left(\frac{qk_j^\top}{\sqrt{d}}\right)}.$$5 sparse attention can suffer from non-negligible mask-prediction cost and from suboptimal shared thresholds across heterogeneous heads, which motivates its plug-in head-wise control layer (&&&^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^8&&&). DynamicRad distinguishes a speed-optimized static-ratio mode from a quality-first dynamic-threshold mode, making the speed–quality trade-off an explicit part of the module definition (&&&3^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\2^^^^&&&).

Within the Tactic formulation, the preferred regime is therefore clear from the supplied description: long contexts, decode-time KV loading that dominates latency, accuracy-sensitive applications, and a requirement for a dynamic sparse method that does not depend on calibration. The examples listed are long-document QA, long-context reasoning, summarization or generation over 32K–5^^^^2id:(Zhu et al., 17 Feb 2025) OR title:\22^^^^K contexts, and serving scenarios where full attention is too slow but fixed budgets are too brittle (&&&^^^^2query2^^^^&&&).