Balancing memory compression with performance in LLM-based agents

Determine the appropriate balance between memory compression rates and downstream task performance in memory modules for large language model–based agents, by characterizing how textual or latent memory extraction and compression affect accuracy and cost and by identifying strategies that retain salient information while minimizing token usage and inference latency.

Background

The survey highlights that memory extraction and compression are central to improving agent efficiency by reducing token consumption and context-window saturation. However, aggressive compression can discard critical information and degrade task accuracy. Empirical results cited for systems such as LightMem show a clear trade-off: higher compression lowers cost but can harm performance, whereas milder compression preserves accuracy at greater cost.

AgentFold and other memory-centric methods demonstrate the potential of proactive context management and summarization, yet the precise compression level that optimally balances efficiency (tokens, latency) and effectiveness (task success, reasoning quality) is not established. The authors call for a principled understanding of this trade-off and methods that preserve key signals during compression.

References

Therefore, how to strike an appropriate balance between compression and performance remains an open question, and there may also be alternative approaches that aim to retain as much salient information as possible during the extraction or compression process.

— Toward Efficient Agents: Memory, Tool learning, and Planning  (2601.14192 - Yang et al., 20 Jan 2026) in Discussion: Trade-off Between Memory Compression and Performance, Section 3 (Efficient Memory)

Kernel-managed shared memory reliably outperforms a standard RAG pipeline and an unmanaged external memory backend across all three models, at lower latency than full unfiltered context; the naive_concat comparison is more nuanced, tying on two of three models with a Llama-3.1:8B deficit unexplained by scale, left to future work.

— Kernel-Managed Shared Memory for System-Wide Personalization  (2609.10144 - Lum et al., 9 Sep 2026) in Discussion and Conclusion

Total cost savings are \textbf{not established}.

— When Agent Metrics Measure Different Things: An Evidence-Grounded Audit of the Praxa AI Pipeline  (2609.12017 - Creadore et al., 10 Sep 2026) in Section 5.3, Compaction: follow-up savings and two-call accounting

Since probe-gated re-surfacing did not recover the static-plan loss (\S\ref{sec:compression}), whether such a self-refreshing scratchpad survives compression better than a static block is an open question we hope this workshop takes up.

— Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agents  (2606.22953 - Mehta et al., 22 Jun 2026) in Section 6, Discussion and Limitations, subsection “Implications for context management”

A dedicated survey reviews these methods in detail; from the lifecycle perspective, the open questions are whether folded trajectories should be written back to long-term memory, in what form, and whether information lost in folding can later be recovered.

— From Long Context to Selective Memory:A Survey of Memory-Centric Compression for Large Language Models  (2610.21165 - Gou, 15 Oct 2026) in Section 6.4, final sentence of the subsection “Agent-Side Activation Control”