SelfCompact Scaffold: Adaptive LM Compaction

• The paper introduces SelfCompact Scaffold, an inference-time framework that lets LM agents decide when to compact context via summarization and rubric cues.
• It employs a lightweight rubric and compaction tool to trigger summarization at natural task boundaries, outperforming fixed-interval strategies in accuracy and efficiency.
• Empirical evaluations reveal up to +18.1 percentage points improvement in math tasks and significant token reduction, underscoring its cost-effective performance gains.

SelfCompact Scaffold is an adaptive, training-free framework for context compaction in long-horizon LLM (LM) agents. It enables the agent itself to decide, at inference time, when and how accumulated context should be summarized and replaced, in contrast to prior fixed-interval strategies that rely on hard-coded token or turn thresholds. SelfCompact employs two inference-time mechanisms: an explicitly invoked compaction tool (summarizer interface) and a lightweight rubric that prompts the model to make timing decisions based on explicit task-structural criteria. This approach mitigates “context rot”—the degradation of generation quality resulting from the accumulation of stale or erroneous tokens—by aligning compaction with resolution of subtasks or natural points of convergence, thereby optimizing both performance and compute cost (Li et al., 22 Jun 2026).

1. Motivation and Theoretical Foundation

Long-horizon LM agents, such as those engaged in stepwise mathematical reasoning or agentic web search, generate sequences (trajectories) of tokens denoted as $y_{1:t} = (y_1, y_2, \dots, y_t)$. The complete context at time $t$ is $\mathcal C = (x, y_{1:t})$ with $x$ as the initial prompt. As the token sequence grows, model outputs increasingly reflect the influence of outdated, redundant, or inaccurate intermediate content, a phenomenon termed “context rot.” Empirically, both the likelihood of degraded reasoning and the compute burden scale with trajectory length, with cost $O(t^2)$ in the absence of compaction.

Fixed-interval compaction—triggering summarization at every $N$ tokens or when a budget threshold is approached—fails to account for task-local structure. These content-agnostic policies may compact mid-derivation (discarding in-progress reasoning) or too late, allowing errors to persist and anchor subsequent outputs. SelfCompact addresses this misalignment by proposing that off-the-shelf LMs can determine suitable compaction points if provided (i) a tool for context summarization and (ii) a rubric prompt enumerating explicit, verifiable criteria for compaction decisions (Li et al., 22 Jun 2026).

2. Architecture and Mechanisms

SelfCompact operates entirely at inference time and consists of two core functional elements:

1. Compaction Tool (Summarizer): An inline mechanism that allows the model to emit a special control token or keyword (e.g., <summarize>) to invoke summarization, typically triggered by the rubric’s judgment.
2. Lightweight Rubric: A short, structured prompt periodically appended to the context that requests the model to decide between “compress” or “continue,” based on explicit checks. The rubric may ask about the completion of subtasks (“Have you just completed a sub-step?”), the presence of unresolved derivations, or other task-specific indicators such as open case splits.

The operational algorithm can be abstracted as follows:

• At each generation step, following the emission of model tokens, if the step count $t \equiv 0 \pmod{N}$, the rubric prompt $P_R$ is appended.
• The model samples from $\{\text{compress}, \text{continue}\}$, guided by $P_R$.
• If “compress” is selected, the summarization prompt $t$0 is appended, and the produced summary $t$1 replaces the prior trajectory, resetting the context to $t$2.
• Otherwise, the rubric and its verdict are removed, preserving the full context.

Prompt construction is both task- and role-aware, with distinct summarizer instructions for mathematical derivations (preserving equations and verified facts) and search tool trajectories (summarizing tool calls and verified information). The summarizer prompt is always inserted as a user-role message to maximize key-value (KV) cache reuse, optimizing inference cost (Li et al., 22 Jun 2026).

3. Formalization and Cost Analysis

Let $t$3 denote the length of the pre-compaction trajectory and $t$4 the length of the summary. Compaction is triggered every $t$5 tokens or tool calls, as determined by rubric probes. The cost for a question is given by:

$t$6

where $t$7 is the cached-read rate, $t$8 is the output token rate, $t$9 is the number of cached reads, and $\mathcal C = (x, y_{1:t})$0 the number of output tokens (see Appendix C of (Li et al., 22 Jun 2026) for the full formulation with prefill rate).

Summarization is cost-effective when the break-even condition holds:

$\mathcal C = (x, y_{1:t})$1

This is empirically satisfied for the summarizer designs presented, with observed compression ratios well above this threshold.

4. Example Agentic Workflow

A concrete agentic-search example illustrates the operation of SelfCompact. A user query prompts the agent to search and record verified facts:

• Turn 1: The agent performs a tool call, retrieves evidence, and asserts a verified fact.
• Probe $\mathcal C = (x, y_{1:t})$2: The rubric prompt assesses completion and recommends compressing, as a new verified fact is established.
• Summarizer Call: The agent compacts to a bullet-point summary of facts.
• This cycle repeats, ensuring only relevant, consolidated information persists as context grows.

This approach preserves necessary evidence while purging obsolete or redundant intermediate content, allowing the agent to maintain trajectory coherence during long or branching search episodes (Li et al., 22 Jun 2026).

5. Empirical Results and Comparative Performance

SelfCompact was evaluated on six benchmarks spanning competition math (IMO-Answerbench, HMMT Nov ’25, HMMT Feb ’26) and agentic search (BrowseComp, BrowseComp-Plus, DeepSearch QA) using seven open-weight models. For math tasks, SelfCompact consistently matched or exceeded the accuracy of fixed-interval summarization at substantially reduced token usage. Gains reached up to +18.1 percentage points over no-compaction baselines.

Table: Competition Math Accuracy (%) and Tokens Used

Model	No Compaction	Fixed-Interval	SelfCompact
Qwen3-4B-Instruct	38.7 [16k]	41.5 [44k]	45.1 [48k]
Qwen3-30B-A3B-Instruct	50.6 [16k]	54.9 [26k]	56.4 [29k]
Qwen3.5-9B	32.5 [16k]	40.1 [90k]	47.3 [93k]
Qwen3.5-4B	21.9 [16k]	30.7 [64k]	33.8 [67k]

For agentic search tasks, SelfCompact outperformed both no-compaction and fixed-interval policies while reducing per-question cost by 30–70%. Improvements in accuracy ranged between 5 and 9 points compared to fixed-interval baselines for web search benchmarks, with per-question cost reductions between 46% and 58%.

6. Design Constraints, Extensions, and Limitations

SelfCompact was developed and tested exclusively on open-weight models; potential improvements in timing and compaction metacognition are hypothesized for larger, frontier models that may not require scaffolding. Notably, the system is entirely inference-time: no parameter updates, external verifiers, or RL-based fine-tuning are involved. This isolates the effect of the rubric and ensures compatibility with arbitrary models and downstream inference engines.

Rubric design is task-specific; mathematical reasoning prompts differ from agentic search prompts. Automating rubric generation or learning universal rubric templates remains an open research direction. Additionally, SelfCompact is complementary—it can layer atop model-internal compaction heuristics, and extension to black-box KV-cache eviction alongside human-readable summarization is proposed as a future avenue.

A plausible implication is that meta-cognitive abilities to detect context rot and to trigger efficient, timely compaction can be scaffolded in non-finetuned models by suitably crafted inference-time prompts—potentially redefining the boundaries between agent function and environment (Li et al., 22 Jun 2026).

7. Conclusion

SelfCompact demonstrates that context-compaction timing can be reframed as a local, meta-cognitive task for the agent, achievable through an inference-time rubric and a summarization tool. Across multiple open-weight models and challenging benchmarks, SelfCompact consistently matches or surpasses fixed-interval approaches in both task accuracy and per-question compute cost, all without requiring parameter updates or external supervision. This suggests a new class of adaptive scaffolds for LM agents operating in long-horizon or tool-augmented settings, with promising opportunities for further refinement and automation (Li et al., 22 Jun 2026).