MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

Abstract: Most LLM agent memory systems rely on a small set of static, hand-designed operations for extracting memory. These fixed procedures hard-code human priors about what to store and how to revise memory, making them rigid under diverse interaction patterns and inefficient on long histories. To this end, we present \textbf{MemSkill}, which reframes these operations as learnable and evolvable memory skills, structured and reusable routines for extracting, consolidating, and pruning information from interaction traces. Inspired by the design philosophy of agent skills, MemSkill employs a \emph{controller} that learns to select a small set of relevant skills, paired with an LLM-based \emph{executor} that produces skill-guided memories. Beyond learning skill selection, MemSkill introduces a \emph{designer} that periodically reviews hard cases where selected skills yield incorrect or incomplete memories, and evolves the skill set by proposing refinements and new skills. Together, MemSkill forms a closed-loop procedure that improves both the skill-selection policy and the skill set itself. Experiments on LoCoMo, LongMemEval, HotpotQA, and ALFWorld demonstrate that MemSkill improves task performance over strong baselines and generalizes well across settings. Further analyses shed light on how skills evolve, offering insights toward more adaptive, self-evolving memory management for LLM agents.

• The paper introduces a learnable and evolvable memory framework that replaces fixed memory operations with a skill bank managed by a controller, executor, and designer.
• The paper employs reinforcement learning and LLM-guided evolution to optimize memory extraction and update, achieving state-of-the-art results on long-context and embodied benchmarks.
• The paper demonstrates that evolving memory skills enhances adaptive memory management and promotes robust transfer across varied tasks and distribution shifts.

MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

Motivation and Problem Formulation

Prevailing LLM agent memory architectures predominantly depend on static, manually-crafted primitives (e.g., add/update/delete/skip) and heuristic routines for memory extraction, consolidation, and pruning. Such designs encode strong human priors and fail to adapt to the diversity and scale of real-world, long-horizon LLM-agent interactions. These approaches are brittle under distribution shifts and interaction complexity, limiting both the efficiency and effectiveness of memory systems as agents encounter larger and more diverse histories.

MemSkill addresses this rigidity by reframing agent memory operations as a learnable and evolvable bank of memory skills. Memory skills are structured, reusable routines that specify when and how to extract, revise, or discard memory from interaction traces. This framework eliminates reliance on static procedural templates, instead elevating memory construction and management to a trainable abstraction governed by interaction data and task performance.

MemSkill Architecture

MemSkill comprises three integral modules:

• Controller: Learns to select a compact, context-sensitive subset of skills from the evolving skill bank. At each processing step (span-level), the controller encodes the current text span and retrieved memories, computes compatibility scores with each skill, and selects a Top-K set using Gumbel-Top-K sampling. This process is compatible with a dynamically changing skill repertoire.
• Executor: A fixed LLM-based module that receives the selected skills, current text span, and retrieved memories, generating skill-guided memory updates in a single forward pass. This formulation enables scalable memory construction, obviating repeated incremental operations typical of per-turn methods.
• Designer: Periodically reviews hard failure cases accumulated during trace processing. Utilizing LLM-based analysis, the designer identifies missing or suboptimal memory behaviors, refines existing skills, and proposes new ones, progressively enhancing the expressivity and utility of the skill bank. The designer triggers an exploration phase for new skills and retains only those updates that yield improved training rewards.

The overall architecture alternates between learning to use the current skill bank (controller+executor) and evolving the skill bank itself (designer), yielding a closed-loop adaptation process.

Optimization and Training Protocols

The controller is trained via reinforcement learning with downstream task performance as the reward signal (e.g., F1, success rate). For each interaction trace, constructed memories are evaluated, and the resulting reward is assigned to the sequence of Top-K skill selection actions using policy-gradient objectives (PPO). The joint action probability for Top-K selection without replacement is computed to enable effective policy optimization and exploration under a mutating action space.

Skill evolution is orchestrated by the designer, which aggregates recent hard cases, clusters them by failure patterns, and issues actionable LLM-guided modifications to the skill bank. Only INSERT and UPDATE operations are evolved in this process, with DELETE and NOOP remaining fixed. A rollback/early-stop mechanism ensures monotonic improvement in skill bank quality.

Empirical Results

Experiments were conducted on established long-context and embodied agent benchmarks, including LoCoMo, LongMemEval, HotpotQA, and ALFWorld. MemSkill is compared to strong baselines: MemoryBank, A-MEM, Mem0, MemoryOS, and more.

Key results:

• On conversational memory benchmarks (LoCoMo, LongMemEval), MemSkill achieves the highest LLM-judge (L-J) and F1 scores across models, demonstrating its superiority in memory extraction quality.
• In embodied settings (ALFWorld, both seen/unseen splits), MemSkill yields the highest success rates, substantiating the practical benefit of adaptable, skill-guided memory for long-horizon action planning.
• MemSkill exhibits strong generalization: skills learned on one dataset/model (e.g., LoCoMo with LLaMA) transfer robustly to others (e.g., LongMemEval, HotpotQA or Qwen), maintaining performance without retraining. This property underscores that the learned skills encode task-agnostic, reusable memory behaviors.
• Ablation studies confirm that both the RL-trained controller and the LLM-based skill designer are critical—removing either results in substantial performance degradation. Skill evolution (especially the addition of new skills) is necessary to generalize beyond primitive memory operations.
• Under pronounced distribution shift (dialogue → long-form QA, evaluated on HotpotQA), MemSkill continues to outperform MemoryOS and A-MEM, particularly as context length increases, indicating strong resilience to surface-form and task structure variation.
• Case analysis reveals MemSkill’s capacity for domain specialization: dialogue-focused skills (temporal organization, activity tracking) contrast with ALFWorld skills (action constraints, object states/movements), reflecting task-driven, data-grounded skill adaptation.

Practical and Theoretical Implications

MemSkill's closed-loop adaptation framework provides significant theoretical and engineering advances for LLM agent memory:

• Reduced Manual Priors: By learning both what to store and how to revise memory directly from interaction data, MemSkill eliminates the brittleness and rigidity of fixed procedural templates.
• Skill-conditioned Composition: Flexible skill selection and composition enables more expressive memory construction, facilitating operation at variable granularity and efficient handling of extremely long contexts.
• Evolvability and Self-Improvement: The skill bank evolves through explicit feedback on failure cases, mirroring realistic, lifelong agent learning. The explicit separation of controller (usage) and designer (structure evolution) suggests a pathway toward more interpretable, modular agent systems.
• Generalization and Reusability: The ability to transfer learned skills within and across domains, as well as across LLM backbones, suggests an emergent property of skill-based memory abstractions: independence from low-level surface forms and strong task-agnostic adaptation.

Future Directions

This framework opens several avenues for future work:

• Automated and continual skill evolution in open-ended, real-world deployment.
• Extension of self-evolving skill abstractions beyond memory—for example, to tool use and planning modules.
• Incorporation of adversarial and safety-aware designer protocols, enabling resilience against suboptimal or risky memory operations.
• Study of interpretability and user control interfaces for skill banks, enabling human-in-the-loop memory supervision and safe deployment in sensitive applications.

Conclusion

MemSkill introduces an agent memory architecture in which reusable, composable memory skills supplant static, hand-designed routines. By unifying RL-based skill selection with continual, LLM-guided skill evolution, it achieves substantial improvements across both conversational and embodied agent benchmarks. The explicit, editable skill bank anchors interpretability and transferability, moving toward robust, adaptive, and self-improving agent memory systems. This paradigm offers promising opportunities for advancing the adaptability, autonomy, and practicality of LLM-driven agents operating over long interaction horizons (2602.02474).