Self-Compacting LM Agents

• Self-compacting LM agents are advanced scaffolding architectures that dynamically condense context, prune irrelevant details, and internalize memory to mitigate context rot in long-horizon tasks.
• They integrate inference-time compaction, parametric memory consolidation, and orchestration strategies to achieve significant performance gains and token cost savings.
• Empirical results show these agents can boost success rates by up to +18.1 pp while reducing computational latency and maintaining efficient multi-agent collaboration.

Self-compacting LLM (LM) agents are advanced scaffolding architectures for LLMs that enable agents to autonomously maintain high performance over long multi-step interactions by dynamically condensing, pruning, or internalizing context and memory. They address the combinatorial context growth, context rot, and efficiency bottlenecks inherent to standard prompt-chaining, agentic search, and multi-agent orchestration. These agents combine model-driven situational compaction, parametric memory consolidation, meta-cognitive orchestration, and targeted history pruning—leveraging both inference-time and learned mechanisms to preserve relevant information, control context window usage, and maximize downstream effectiveness (Li et al., 22 Jun 2026, Yu et al., 2 Feb 2026, Sampath et al., 10 Jan 2026).

1. Motivation: Challenges in Long-Horizon Agent Context Management

LLM agents performing long-horizon reasoning, tool use, or multi-agent collaboration face rapid and often uncontrollable growth of their working context. As reasoning traces accumulate—composed of intermediate thoughts, failed attempts, explored derivations, tool call results, and user feedback—they suffer from "context rot": irrelevant or redundant content begins to dominate the prompt, anchoring model generations and degrading agentic performance. Standard practices—such as fixed-interval summarization or naive trajectory replay—introduce either loss of salient partial results (by compacting at arbitrary token thresholds) or unsustainable context bloat, resulting in frequent loss of critical information and prompt overflow (Li et al., 22 Jun 2026, Yu et al., 2 Feb 2026).

The scalability bottleneck is further magnified in collaborative and multi-agent systems. Monolithic agents equipped with large toolkits experience context pollution and deteriorating attention, while static agent swarms suffer high latency and resource overhead (Sampath et al., 10 Jan 2026).

2. Inference-Time Compaction: Adaptive Abstractive Summarization with SelfCompact

SelfCompact is a scaffold that empowers the LM itself to make compaction decisions during inference. The key contributions are:

• Inline Abstractive Summarizer: The agent is exposed to a <summarize> tool in the prompt, enabling it to replace long context segments with condensed natural-language summaries—invoked at moments selected by the agent based on reasoning structure, not arbitrary token count.
• Lightweight Rubric-Guided Control: A configurable judge prompt encodes task-specific positive and negative triggers (e.g., "sub-task resolved," "mid-derivation"), which the model consults at regular intervals to decide "compress" vs. "continue."
• Integration into Generation Loop: At every N steps, the agent issues a rubric probe, evaluates (via the LM itself) whether to summarize or not, and replaces the prompt context accordingly.

Pseudocode (from (Li et al., 22 Jun 2026)):

Input: x (user prompt), model π, interval N, max steps T Initialize C ← [ x ] for t in 1..T: y_t ~ π( · | C ) C.append(y_t) if y_t is final_answer: return y_t if t mod N == 0: # Rubric probe C.append(P_rubric) r ~ π(· | C) # 'compress' or 'continue' C.pop(P_rubric) if r == 'compress': # Summarize C.append(P_summarizer) ŷ ~ π(· | C) C = [ x, ŷ ] # else do nothing return last y_T

Empirical results demonstrate substantial gains: up to +18.1 pp on competition math and +5–9 pp on agentic search over no compaction baselines, and outperforming fixed-interval summarization at similar or lower token cost (30–70% per-query cost savings). The approach requires no fine-tuning or additional models—the rubric and summarizer are powered by the base LM (Li et al., 22 Jun 2026).

3. Parametric Self-Consolidation: Lifelong Internalization via Prefix-Tuning

The self-consolidation mechanism (EvoSC) addresses the ill-posedness of lifelong agent memory by distilling the agent's full interaction history—including both success and failure—into learnable parametric memory:

• Contrastive Reflection: Summarizes insights from pairs of successful and failed trajectories into compact bullet points, capturing both error patterns (to avoid) and reusable success strategies.
• Prompt-Based Parametric Distillation: Periodically, a small set of fresh experiences (trajectories and insight bullets) is distilled into a fixed-size prefix $P_{\theta}$ via token-level cross-entropy minimization. The model thereafter leverages this prefix to access implicitly internalized historical experience without accreting ever-growing textual memory.
• Explicitly Bounded Prompt Length: At inference, the prompt consists of $P_{\theta}$, a small number of recent insights, and a fixed number of full examples—ensuring context length remains constant.

Formally, consolidation minimizes: $$\mathcal{L}_{\text{consolid}}(\theta) = -\sum_{s=1}^{r}\;\sum_{j} \log\,P_{\theta} \bigl(A^*_{k,s,j}\mid P_{\theta},\,\mathcal{I}_k,\,\mathcal{H}_{k,s-1},\,A^*_{k,s,<j}\bigr)$$ This process enables agents to scale to hundreds of tasks, outperforming memory-based baselines whose context grows linearly and eventually causes out-of-memory errors. Empirically, EvoSC achieves +6.7 pp over AWM, TER, SCM, and A-MEM on Database tasks, and remains stable as prompt size increases (Yu et al., 2 Feb 2026).

4. Dynamic Multi-Agent Orchestration and Compaction: DMoE, Meta-Cognition, and History Surgery

Self-compacting agents in multi-agent or tool-rich settings require runtime orchestration strategies to avoid context and attention collapse:

• Dynamic Mixture-of-Experts (DMoE): Maintains a bounded active set of specialized sub-agents, routing user queries based on embedding similarity. Just-in-time "hiring" of experts is triggered by capability gap detection; least-recently-used (LRU) eviction policies ensure that the token cost remains bounded.
• Meta-Cognition Engine: Asynchronously scans dialogue for evidence of competence gaps or excessive reliance on generic tools. Signals (e.g., $S_{\text{gap}}, S_{\text{opt}}$) trigger dynamic architectural adaptation—hiring new experts or reconfiguring the pool.
• Surgical History Pruning: Upon hydration of a new expert, context history is pruned to remove prior negative responses ("I lack the capability") for the corresponding domain, reducing refusal bias and context pollution.
• Token–Success Trade-off: The utility function $U = \alpha S_{\rm rate} - \beta T_{\rm avg}$ quantifies trade-offs between accuracy and context efficiency.

Empirically, these mechanisms yield 50–60% token savings and significant latency improvements compared to static agent swarms, while maintaining or improving success rates (Sampath et al., 10 Jan 2026).

5. Integration with Sample-and-Select: Efficient Trajectory Search via Self-Consistency

Self-compacting agents often leverage sample-and-select procedures to generate, rank, and select promising action trajectories:

• Soft Self-Consistency (SOFT-SC): Replaces brittle majority voting with token-likelihood-based trajectory scoring. Each trajectory $y_j$ is assigned a continuous score derived from its per-token probabilities (mean, min, or product aggregation).
• Sample Efficiency: SOFT-SC achieves target performance with roughly half the samples required by majority voting (SC). For interactive and combinatorial domains, SOFT-SC improves success rates while reducing inference cost by 30–50% (Wang et al., 2024).
• Adaptive Sampling: The number of samples taken is adaptively controlled by the agent’s confidence in high-scoring trajectories, further reducing computation.

This probabilistic ranking dovetails with self-compacting strategies by enabling robust selection under high action diversity and context change, vital for multi-step, memory-intensive operations.

6. Empirical Results and Performance Benchmarks

Method / Scaffold	Context Management Scope	Main Reported Gains
SelfCompact	Inference-time context	+18.1 pp (math); +5–9 pp (search);
	summarization w/ rubric	30–70% token cost savings
EvoSC (Self-Consolidation)	Lifelong agent memory	+6.7 pp (DB); solves OOM vs. replay
DMoE with Meta-Cognition	Multi-agent orchestration	50–60% token, 40% end-to-end latency↓
SOFT-SC sample-and-select	Action space search	1.3–9.6 pp success gain @½ samples

A key qualitative result is that self-compacting approaches consistently outperform both no-compaction and static/fixed-interval baselines while minimizing context and computation inefficiencies (Li et al., 22 Jun 2026, Yu et al., 2 Feb 2026, Sampath et al., 10 Jan 2026, Wang et al., 2024).

7. Limitations and Open Directions

• Rubric and policy automation: Manual rubric design is task-specific; automating its induction or learning optimal compaction policies without reliance on fixed templates is an open research problem (Li et al., 22 Jun 2026).
• Frontier closed models: To date, empirical validation on open-weight models dominates; applicability to closed-source, frontier models with possibly superior internal introspection remains uncertain.
• Adaptive schedules and multi-agent lifelong learning: Exploring adaptive consolidation frequencies, relevance-based experience selection, and continual-world streams are promising, unaddressed challenges (Yu et al., 2 Feb 2026).

A plausible implication is that unified architectures combining all above elements—abstractive compaction, parametric consolidation, orchestration, and probabilistic sample-and-select—will define the scaling frontier for robust, efficient, and self-maintaining LM agents across open-ended, real-world tasks.