SelfCompact: Inference-Time Trajectory Compaction

• SelfCompact is an inference-time scaffold for language-model agents that adaptively compacts lengthy reasoning trajectories to remove stale or irrelevant information.
• It pairs a compaction tool with a lightweight rubric to determine safe breakpoints, ensuring summaries occur only after a reasoning sub-goal is completed.
• Experimental results show significant improvements in long-horizon tasks, with gains in accuracy and up to 70% reduction in processing cost compared to fixed-interval summarization.

SelfCompact is an inference-time scaffold for language-model agents that allows the model itself to decide when and how to compact its own trajectory, using only prompting and no fine-tuning (Li et al., 22 Jun 2026). It is designed for long-horizon reasoning and tool-using settings in which chains of thought, tool calls, tool outputs, and exploratory branches accumulate into long traces containing stale or superseded content. The central claim is that long traces suffer from “context rot”: old, often flawed reasoning and irrelevant search results remain in context, anchor subsequent generations, and eventually outgrow the context window. SelfCompact addresses this by pairing a compaction tool with a lightweight rubric that specifies when compaction should fire and when it should be suppressed, thereby eliciting adaptive compaction without external supervision (Li et al., 22 Jun 2026).

1. Problem setting and motivating diagnosis

SelfCompact is motivated by the behavior of long agent traces in competitive mathematics and agentic web search. The traces considered in the paper include chains of thought, tool calls, search results, partial sub-plans, and abandoned branches. As these traces grow, they accumulate “junk,” including dead ends, obsolete hypotheses, and irrelevant search results (Li et al., 22 Jun 2026).

The paper identifies two related failures. The first is “context rot,” or “rotting context,” in which stale reasoning remains in the prompt and exerts an anchoring effect on later generations. The second is the finite context window and associated cost: trajectories eventually exceed the available window, and even before overflow, long contexts become expensive to process (Li et al., 22 Jun 2026). This diagnosis leads to the conclusion that some form of compaction is necessary for long-horizon agents.

A central contrast is drawn with fixed-interval and threshold-triggered summarization. The paper characterizes standard practice as content-agnostic compaction: summarization is triggered every $k$ turns, every $k$ tokens, or when context nears a maximum length. Such triggers do not distinguish mid-derivation from completed sub-goals, may fire mid-search, and may preserve stale material for many steps after a sub-goal has resolved (Li et al., 22 Jun 2026). The paper’s BrowseComp example illustrates this difference: a fixed-interval summary can erase verified facts if it fires mid-reasoning, whereas SelfCompact places summaries at “closed reasoning units” and preserves the facts needed for the final answer (Li et al., 22 Jun 2026).

The paper further frames the issue as a “meta-cognitive gap.” Off-the-shelf models, even when given access to a summarization tool, do not reliably recognize when their own context is rotting or when summarization would be beneficial. However, when given a short rubric describing appropriate breakpoints, the same models become able to gate compaction more effectively (Li et al., 22 Jun 2026). This suggests that the missing capability is not summarization as such, but state-aware timing.

2. Scaffold design and algorithmic structure

SelfCompact consists of two inference-time elements: a compaction tool and a lightweight rubric (Li et al., 22 Jun 2026). The compaction tool is a summarizer $\mathcal{S}$ that compresses the trajectory into a shorter natural-language summary. The rubric is a prompt that periodically asks the model whether the current point is suitable for compaction.

The method is formalized in the paper through a generation loop over a context $\mathcal{C}$, with original prompt $x$, base model $\pi$, probe interval $N$, step budget $T$, rubric prompt $P_R$, and summarizer prompt $P_S$ (Li et al., 22 Jun 2026). At each generation step, the model produces a message $k$0, which is appended to the context. Every $k$1 steps, a rubric probe is appended; the model then outputs either compress or continue. If it outputs compress, the scaffold appends the summarizer prompt, obtains a condensed trajectory $k$2, and hard-resets the context to $k$3. If it outputs continue, the rubric probe and verdict are removed and the trajectory proceeds unchanged (Li et al., 22 Jun 2026).

The paper emphasizes that both the rubric judge and the summarizer are the same model $k$4. No separate controller, verifier, or reward model is introduced. The approach therefore differs from training-based methods that embed the compaction policy in model weights. Here, the decision policy is implemented through prompting, while the compaction operation is another prompted generation by the same model (Li et al., 22 Jun 2026).

This separation of roles is structurally important. The scaffold distinguishes between the decision of whether to compact and the act of producing the compacted representation. A plausible implication is that SelfCompact treats context management as a compositional control problem rather than a monolithic learned behavior: the model is induced to classify its current trajectory state and, conditional on that classification, to summarize it.

3. Compaction tool and rubric semantics

The compaction tool $k$5 maps the original prompt and current trajectory to a condensed trajectory,

$k$6

The summary is intended to preserve the essential reasoning steps concluded so far, verified facts or partial proof state, and the high-level plan going forward (Li et al., 22 Jun 2026). For mathematics, the summarizer is instructed to produce a concise account of assumptions, established lemmas, and partial results. For agentic search, it is instructed to produce a compact log of what has been searched, which facts have been confirmed or ruled out, and what remains unresolved (Li et al., 22 Jun 2026).

The rubric is task-specific but lightweight. It is appended as a user message at probe times and asks the model to output one of two tokens: compress or continue (Li et al., 22 Jun 2026). The rubric operationalizes two classes of conditions.

The first class contains conditions under which compaction should fire. One is that a sub-task has resolved: in mathematics, a lemma, case analysis, intermediate variable, or consistent branch has been completed; in search, a sub-question has been answered or a clear intermediate conclusion has been reached (Li et al., 22 Jun 2026). The second is that the trajectory is converging: recent steps reflect refinement or confirmation rather than large shifts, and the agent is about to move to a new sub-goal (Li et al., 22 Jun 2026).

The second class contains conditions under which compaction should be suppressed. These include mid-derivation, where a proof or calculation has not yet closed; mid-search or mid-exploration, where multiple hypotheses remain live; and “stuck or thrashing,” where the model is oscillating or repeatedly re-fetching similar search results without genuine progress (Li et al., 22 Jun 2026). The paper states that in such states summarization can freeze an unhelpful state or delete still-needed information.

The rubric is described as only a paragraph of guidance, and the model is required to quote evidence from the trajectory that satisfies the stated conditions (Li et al., 22 Jun 2026). This requirement is noteworthy because it turns timing into a localized evidence-evaluation problem over the current transcript. This suggests that SelfCompact does not presume intrinsic self-monitoring; rather, it provides an explicit textual criterion for what counts as a safe breakpoint.

4. Baselines, ablations, and the role of timing

The paper evaluates SelfCompact against several baselines: no compaction, fixed-interval or fixed-threshold summarization, delete-all, keep-last-$k$7, and SelfCompact without rubrics (Li et al., 22 Jun 2026). The comparison is designed to isolate whether gains arise from summarization per se or from state-aware timing.

In mathematics, fixed-interval summarization always fires every 16k tokens of reasoning; in search agents, summarization is triggered when 30% of the maximum context window is consumed (Li et al., 22 Jun 2026). The delete-all baseline discards all past trajectory when the threshold is reached; keep-last-$k$8 retains only the last three turns. These baselines reduce context length, but they do not decide whether the current semantic state is safe to compress.

The ablation “SelfCompact without rubrics” is the most direct test of the paper’s central claim. In this setting, the model has access to the compaction tool but lacks the structured rubric that specifies when to use it. The reported results show that this rubric-free version performs similarly to fixed-interval methods rather than to full SelfCompact (Li et al., 22 Jun 2026). For GLM-4.7-Flash on agentic search, fixed-interval summarization reaches 41.5% average accuracy, SelfCompact without rubric reaches 41.0%, and full SelfCompact reaches 46.4% (Li et al., 22 Jun 2026). For Qwen3-4B-Instruct on IMO-Answerbench, no compaction reaches 38.9%, fixed interval 41.4%, SelfCompact without rubric 40.9%, and full SelfCompact 45.5% (Li et al., 22 Jun 2026).

These ablations support two negative claims made explicitly in the paper. First, the tool alone is insufficient: some models invoke it too frequently, including mid-derivation, while others do not invoke it when it would help (Li et al., 22 Jun 2026). Second, the rubric alone cannot act, because it supplies criteria but does not itself implement summarization (Li et al., 22 Jun 2026). The full method therefore depends on both components.

A broader implication is that SelfCompact locates the main source of failure in timing rather than in summarization quality. The evidence presented is consistent with the view that summarization can be actively harmful if invoked at the wrong moment, and beneficial if aligned with completed sub-goals.

5. Experimental domains and quantitative findings

The paper evaluates SelfCompact on six benchmarks spanning competitive mathematics and agentic search, using seven models (Li et al., 22 Jun 2026). The mathematics benchmarks are IMO-Answerbench, HMMT Nov 2025, and HMMT Feb 2026. The search benchmarks are BrowseComp, BrowseComp-Plus, and DeepSearch QA (Li et al., 22 Jun 2026).

For mathematics, the models are Qwen3-4B-Instruct-2507, Qwen3-30B-A3B-Instruct-2507, Qwen3.5-4B, and Qwen3.5-9B, with the Qwen3.5 models run with thinking disabled in the reported study (Li et al., 22 Jun 2026). For search, the models are GLM-4.7-Flash, MiniMax-M2.5, and MiMo-V2-Flash (Li et al., 22 Jun 2026). The paper states that it uses vLLM-based inference for Qwen and adopts the scaffolds and evaluation settings from the Agentic Aggregation work for the search agents (Li et al., 22 Jun 2026).

Quantitative results on competition math

The reported results show that SelfCompact outperforms both no compaction and fixed-interval summarization under approximately matched token budgets (Li et al., 22 Jun 2026).

Model	No compaction avg	Fixed interval avg	SelfCompact avg
Qwen3-4B-Instruct-2507	38.7	41.5	45.1
Qwen3-30B-A3B-Instruct-2507	50.6	54.9	56.4
Qwen3.5-9B	32.5	40.1	47.3
Qwen3.5-4B	21.9	30.7	33.8

For Qwen3.5-9B, the gains over no compaction are especially large: +16.4 points on IMO, +10.0 on HMMT Nov, and +18.1 on HMMT Feb (Li et al., 22 Jun 2026). The paper summarizes the mathematics results by stating that SelfCompact is best in 11 out of 12 model-by-benchmark cells (Li et al., 22 Jun 2026).

Quantitative results on agentic search

On search benchmarks, the method improves accuracy while lowering per-question cost relative to no compaction (Li et al., 22 Jun 2026).

Model	No compaction overall	SelfCompact overall	Cost reduction
GLM-4.7-Flash	36.6	46.4	–46%
MiniMax-M2.5	54.6	63.9	–58%
MiMo-V2-Flash	48.9	59.2	–48%

The paper reports that on BrowseComp-Plus the gains over no compaction are +8.5 for GLM, +9.2 for MiniMax, and +5.3 for MiMo (Li et al., 22 Jun 2026). Across search models, the accuracy ordering is reported as No compaction < Fixed-interval $k$9 SelfCompact (Li et al., 22 Jun 2026).

The headline summary in the abstract is that SelfCompact matches or exceeds fixed-interval summarization at a fraction of the token cost, improving over a no-summarization baseline by up to 18.1 points on math and 5–9 points on agentic search at 30–70% lower per-question cost (Li et al., 22 Jun 2026). Within the paper body, the search results concretize the lower-cost claim through reported reductions such as –29%, –67%, –46%, –53%, –63%, –58%, –59%, –33%, and –48% on individual benchmark/model combinations (Li et al., 22 Jun 2026).

6. Cost model, behavior analysis, and broader significance

The paper includes a cost analysis based on the length $\mathcal{S}$0 of the pre-compaction trajectory and the length $\mathcal{S}$1 of the resulting summary (Li et al., 22 Jun 2026). At each probe, the rubric probe adds only a short generation over a cached prefix. If the model decides to compress, the summary generation costs $\mathcal{S}$2, and the new shorter context incurs a one-time $\mathcal{S}$3 prefill cost (Li et al., 22 Jun 2026). Because KV-cache is reused, the method avoids an $\mathcal{S}$4 re-encoding that would arise in a naive separate summarization pass (Li et al., 22 Jun 2026). The paper reports that compaction pays off when $\mathcal{S}$5, and that the search summarizer achieves 20–80$\mathcal{S}$6 compression (Li et al., 22 Jun 2026).

Behavioral analysis reinforces the paper’s claim that timing is the dominant issue. For Qwen3-4B-Instruct on IMO-Answerbench, the authors track correctness transitions after each of 12 fixed-interval summarization calls and find 1486 wrong-to-correct transitions and 1009 correct-to-wrong transitions, so 40.4% of all transitions hurt performance (Li et al., 22 Jun 2026). They also define an oracle policy that summarizes only when the current answer is not already correct, yielding 52.9% accuracy at 44k tokens, compared with 38.9% for no compaction, 41.4% for fixed interval, and 45.5% for SelfCompact (Li et al., 22 Jun 2026). This is presented as evidence that adaptive timing has substantial headroom.

The search experiments further show that SelfCompact tends to trigger summaries earlier than the fixed 30% context-threshold baseline, aligning them with completed sub-questions rather than waiting for context saturation (Li et al., 22 Jun 2026). On the hardest difficulty bins, measured by the total tokens used by the no-compaction baseline, SelfCompact outperforms the threshold policy by 5–20 percentage points across all three search models (Li et al., 22 Jun 2026). This suggests that the scaffold is particularly valuable when trajectory structure becomes long and heterogeneous.

The paper’s broader claim is that “when to compact” can be supplied as an inference-time capability through a lightweight rubric rather than trained into the model (Li et al., 22 Jun 2026). It therefore positions SelfCompact not merely as a summarization mechanism, but as an example of scaffolded meta-cognitive control. The authors suggest extensions to coding agents, scientific research assistants, and long-horizon planning, and propose reinforcement learning or fine-tuning with rubrics as targets as a future direction (Li et al., 22 Jun 2026). A plausible implication is that SelfCompact exemplifies a more general pattern in agent design: control problems such as when to search, when to compress, or when to switch sub-goals may be partially solvable through explicit textual criteria plus tools, without modifying the base model.