AutoMem: Automated Learning of Memory as a Cognitive Skill

Abstract: Memory expertise is a learned skill: knowing what to encode, when to retrieve, and how to organize knowledge--a capacity known in cognitive science as metamemory. We bring this perspective to LLMs by treating memory management as a trainable skill. We promote file-system operations to first-class memory actions alongside task actions, letting the model itself decide how to manage its memory. This memory skill improves along two axes: the structure that supports it (prompts, file schemas, action vocabulary), and the proficiency of the model exercising it. Both axes resist manual optimization: episodes in long-horizon tasks run for thousands of steps, and a single memory mistake can hide long before it surfaces, making human review of full trajectories impractical. We introduce AutoMem, a framework that automates both axes. In the first loop, a strong LLM reviews complete agent trajectories and iteratively revises the memory structure that shapes how the agent interacts with its memory files. In the second loop, the agent's own good memory decisions are identified from many episodes and used as training signal to sharpen the model's memory proficiency directly. Across three procedurally generated long-horizon games (Crafter, MiniHack, and NetHack), optimizing memory alone--without modifying the model's task-action behavior--improved the base agent's performance ~2x-4x, bringing a 32B open-weight model competitive with frontier systems such as Claude Opus 4.5 and Gemini 3.1 Pro Thinking. Our results show that memory management is an independently learnable skill, and a high-leverage objective yielding large gains on long-horizon tasks.

• The paper presents a two-loop meta-optimization framework that separates memory scaffold revisions from memory proficiency training.
• It demonstrates a 2x–4x improvement in progression rates on procedurally-generated tasks by employing file-system-based memory management.
• The methodology showcases how autonomous, trainable memory operations serve as modular cognitive skills to enhance LLM agent performance.

AutoMem: Automated Learning of Memory as a Cognitive Skill

Motivation and Conceptual Framing

AutoMem approaches memory management in LLM-based agents from the perspective of metamemory: the trainable skill of controlling not only what information to encode and retrieve, but also how such information is structured. Existing memory-augmented LLMs commonly treat memory as a static architectural module—an external retriever, vector database, or scratchpad—where operation rules are set a priori. AutoMem diverges by promoting file-system operations (read, write, search, append, create) to first-class actions, granting the model full, explicit agency over its own memory management decisions. This construct views cognitive scaffolding as mutable, and memory proficiency as separable from core task behavior.

The framework formalizes memory skill along two orthogonal axes: (1) memory structure, that is, the design of prompts, file schemas, validation logic, and the vocabulary of memory actions; and (2) proficiency, or the model’s parametric ability to select among and execute these actions optimally. Both axes inherently resist static, hand-coded solutions, especially for long-horizon, stochastic environments where the repercussions of memory decisions surface only after hundreds or thousands of steps.

AutoMem: Two Automated Outer Loops

AutoMem’s central contribution is a two-loop meta-optimization framework (Figure 1), each leveraging large meta-LLMs to automate improvement of the two axes:

Figure 1: Two automated outer loops optimize an LLM agent using file-system memory: (1) A meta-LLM iteratively revises the memory scaffold; (2) a meta-LLM orchestrates curation and finetuning to train a memory specialist.

Outer Loop 1 (Structure Optimization):

A high-capacity meta-LLM reviews full agent trajectories—including code, prompts, and the entire file-system memory state—and diagnoses failures or inefficiencies in structure. In each iteration, it proposes precise revisions to the agent’s scaffold. These revisions are only accepted upon measured improvement over fixed evaluation seeds. Notably, this approach eliminates the need for human-in-the-loop trajectory review, which would be intractable at such episode lengths.

Outer Loop 2 (Memory Proficiency Training):

After scaffold convergence, a separate meta-LLM acts as an automated data curator and training orchestrator. It samples large pools of agent trajectories, selects high-value memory decision examples, and configures supervised LoRA-based finetuning. The separation of the memory specialist model from the unmodified base gameplay model enables targeted improvement of memory proficiency without negative transfer or catastrophic forgetting for world actions.

Experimental Setting and Evaluation

Experiments are conducted on the challenging, procedurally-generated task suite from BALROG—Crafter, MiniHack, and NetHack—which are characterized by long horizons ($10^3$–$10^5$ steps), stochasticity (minimizing the effect of LLM pretraining), and complex, non-trivial information management requirements (Figure 2).

Figure 2: Crafter, MiniHack, and NetHack represent procedurally-generated, stochastic, long-horizon environments unsuited for simple context-window management.

Performance is measured by episode progression rates, with fixed random seeds and standardized episode budgets, using Qwen2.5-32B-Instruct as the base agent.

Numerical Results and Analysis

The combination of scaffold optimization and memory specialist training achieves large gains on all environments, without modifying the model’s world-action weights. Starting from a base file-system memory agent (v0), AutoMem achieves a 2x–4x progression improvement by optimizing only memory usage (Figure 3).

Figure 3: Sequential improvement: memory scaffold optimization (v0 to vX), then memory proficiency training (+train), boosts Crafter, MiniHack, and NetHack performance.

Specifically, for Qwen2.5-32B-Instruct:

Environment	Base v0 (%)	+ Scaffold Opt (%)	+ Memory Training (%)
Crafter	25.0	47.3	51.4
MiniHack	7.5	27.5	30.0
NetHack	0.42	1.57	1.85

The optimized 32B open-weight agent surpasses Qwen2.5-72B-Instruct and closes the gap with proprietary systems such as Claude Opus 4.5 and Gemini 3.1 Pro-Thinking—indicating that memory is a higher-leverage optimization axis than model scaling in these domains.

Behavioral Analysis of Memory Skill Improvements

Scaffold optimization yields sharp improvements in both agent gameplay and memory actions, analyzed via four key behavioral metrics (Figure 4):

• Unproductive Game Action Rate: Drops by 32–65% (less stuck or oscillating behavior).
• Redundant Memory Writes: Reduced by 68–83%.
• Empty-search Rate: Falls by 13–50%, indicating more targeted retrievals.
• Per-step Input Context: Compressed by up to 30%, reflecting memory organization gains.

  Figure 4: Unproductive actions, redundant writes, empty search rates, and input token context decrease markedly with scaffold optimization.

Qualitatively, the agent evolves from low-level, repetitive behaviors (e.g., gathering wood in Crafter), to multi-stage goal pursuit with efficient resource use and navigation (Figure 5). The trained memory specialist further internalizes structured, consult-before-write discipline, as seen in a large decrease in memory writes per search operation.

Figure 5: Agent progression across environments: base agent (left) displays repetitive, low-skill behavior; scaffolded agent (middle) shows structured strategies; memory specialist (right) executes more robust plans and sustains progress.

Implementation and Architectural Details

The inner-loop agent models memory as an extensible file system: at each time step, it chooses whether to log new observations, consult memory, or execute a world action. The outer loops operate by meta-LLM-driven code review and data curation, facilitated by observable, explicit memory actions in the trajectory.

Concrete scaffold revisions include: deduplicating map entries by coordinate via upsert actions, auto-syncing inventories from environment observations, pre-populating strategy templates, and expanded auto-hinting to reduce search and navigation inefficiencies. Memory specialist training is driven by precise episode-segment selection and environment-specific LoRA configurations.

Implications and Theoretical Significance

AutoMem demonstrates that memory management is a separately learnable cognitive skill for LLM agents. Optimizing memory actions and their supporting scaffolds is at least as potent as scaling model size for long-horizon, procedural tasks. Furthermore, the results suggest that “meta-optimization” of scaffolding and proficiency—previously approached only by human designers or via architectural search—can be efficiently delegated to sufficiently strong meta-LLMs using trajectory-level feedback.

This approach operationalizes cognitive-scientific concepts like metamemory and the extended mind, serving as a technical template for learning other cognitive skills (e.g., self-reflection, planning) as modular, trainable agentic faculties. The formal separation between learned memory structure and model decision weights enables targeted advances in agent capabilities, mitigates negative transfer, and allows for compositional skill development.

Potential future extensions include persistent, cross-episode memory; generalization to non-gaming, real-world domains with complex temporal dependencies; and multi-environment or universal scaffolds. Additionally, this methodology may inspire similar automated learning loops for other facets of agentic cognition.

Conclusion

AutoMem provides compelling empirical and methodological evidence that automated, LLM-driven optimization of memory management—separating scaffold structure from proficiency—substantially improves the performance of LLM agents on challenging long-horizon tasks. The explicit promotion of file-system memory actions to first-class, observable policy decisions establishes a blueprint for systematic development of modular cognitive skills in autonomous agents. This work advances both practical agent competence and foundational understanding of skill acquisition and meta-learning in large-scale LLMs.