MemSkill Framework: Adaptive Memory for LLM Agents

• The paper demonstrates that using learnable memory skills with reinforcement learning optimizes structured memory operations in LLM agents.
• MemSkill defines memory skills as tuples with natural language summaries and structured templates to extract, consolidate, and prune information.
• Experimental results show MemSkill’s efficiency and robust transfer across long-context and embodied tasks, reducing LLM calls and enhancing performance.

The MemSkill framework is an adaptive memory management system for LLM agents, designed to replace static, hand-crafted memory extraction and revision procedures with a data-driven, continually evolving bank of learnable "memory skills." MemSkill situates these skills as reusable, structured operations for extracting, consolidating, and pruning information from interaction traces, and employs a closed-loop of skill selection and skill evolution leveraging reinforcement learning and LLM-powered specification synthesis. The primary design motivation is to achieve robust, generalizable, and efficient memory construction for LLM agents under diverse, long-horizon interaction scenarios, eliminating reliance on rigid human priors (Zhang et al., 2 Feb 2026).

1. Formal Structure of Memory Skills

A memory skill is parameterized as a tuple $s=(\desc(s),\spec(s))$, where:

• $\desc(s)$ is a short natural language summary, facilitating retrieval and embedding for skill selection.
• $\spec(s)$ is a structured instruction template detailing the skill's purpose, usage conditions, application protocol, constraints, and action type (e.g., "INSERT," "UPDATE," "DELETE," "NOOP").

At memory-construction step $t$, the controller operates on a context $x_t$—a segment of dialogue or narrative—and a current memory bank $M_t = \{m_{t,1},\ldots,m_{t,R}\}$. The application of a skill yields one or more structured actions (INSERT, UPDATE, DELETE, or NOOP), forming atomic memory operations. The actual implementation $A_s: (x, M) \rightarrow \text{Actions}$ is realized by prompting an LLM executor with $\spec(s)$, rather than via hand-coded procedures.

2. System Architecture and Loop Interactions

MemSkill operates as two mutually integrated loops—use and evolve—incorporating the following components:

• Controller: Encodes $(x_t, M_t)$, scores all skills $s \in S$, and samples a top-K subset of skills ("A_t") using learned selection distributions.
• Executor (LLM): Takes $x_t$, $M_t$, and $\spec(A_t)$ as prompt inputs, producing a sequence of structured memory actions in a single pass. Each memory operation conforms to a canonical block syntax (for INSERT, UPDATE, DELETE).
• Designer: Periodically reviews a buffer of "hard cases" where the current skill set fails (or yields low reward); clusters and analyzes these cases; then prompts the LLM to synthesize new skill specs or refine existing ones. A rollback protocol is employed if the evolved skill bank results in performance regression.

The controller leverages learned embeddings:

• $h_t = f_{\text{ctx}}(x_t, M_t) \in \mathbb{R}^d$ (context-memory encoding)
• $u_i = f_{\text{skill}}(\desc(s_i)) \in \mathbb{R}^d$ (skill encoding)

Skill–context compatibility is scored by $z_{t,i} = h_t \cdot u_i$, with the skill-selection policy $p_e(s_i|h_t)=\text{softmax}_i(z_{t,\cdot})$. Top-K selection utilizes Gumbel-TopK without replacement. The PPO-style policy objective is computed with end-of-trace reward, standard GAE, and includes value and entropy costs. The executor’s LLM is instructed to apply all selected skills jointly, allowing for compositional memory construction in a single inference call (Zhang et al., 2 Feb 2026).

3. Skill Evolution and Designer Mechanism

Skill evolution is driven by the designer, which activates every fixed interval (e.g., 100 training steps):

• Hard-case mining: Interactions yielding low reward are logged with metadata—$(q$, ground truth, retrieved $M$, prediction, reward $r$, fail count $c)$—and assigned a difficulty score $d(q) = (1-r(q)) c(q)$.
• Representative selection: Embeds queries, clusters by difficulty, and samples exemplars.
• LLM analysis and synthesis: First, the LLM proposes root causes for failure and suggests operations (add/refine). Second, synthesis of concrete skill specs is output in JSON format for instantiation or update.
• Exploration incentive: For $T_\text{explore}$ steps post-evolution, the policy is biased to encourage sampling of new or refined skills.
• Rollback and early stopping: If average reward $F_\text{tail}$ over most recent cycles fails to improve, or drops compared to best previous, the skill bank is rolled back.

Ablation results confirm that both controller (policy-guided skill selection) and designer (continual skill evolution) are critical to the system’s efficacy. Allowing only refinements, versus introducing new skills, produces smaller but still significant improvements.

4. Experimental Evaluation

MemSkill's empirical performance has been assessed on several benchmarks covering long-context QA, multihop QA, and embodied agent tasks:

• LoCoMo (long dialogue QA): LLM-judge ("L-J") accuracy of 51.0 versus 44.6 (MemoryOS), with ablations confirming >6-point delta due to active controller and designer.
• LongMemEval: L-J of 59.4 (MemSkill) versus 36.5 (MemoryOS).
• ALFWorld (embodied tasks): Seen split SR of 47.0 and Unseen split SR of 64.2, outperforming prior baselines.
• HotpotQA transfer: Significant gains, particularly as context size scales, e.g., L-J of 72.0 for 200 docs with K=7 (MemSkill), versus 65.5 (A-MEM).
• Efficiency and generalization: Span-level and joint execution reduces LLM calls compared to per-turn alternatives. Across benchmarks, MemSkill demonstrates robust transfer to new LLMs and tasks without retraining.

Ablations indicate that removing the designer or randomizing skill selection both degrade performance, and exploration bias accelerates skill adoption (Zhang et al., 2 Feb 2026).

5. Skill Bank Dynamics and Learned Operations

Skill evolution proceeds from a primitive set (INSERT, UPDATE, DELETE, SKIP) to a bank containing domain-specific or frequent-need operations, such as:

• "Capture Temporal Context" (LoCoMo): extract explicit date/time intervals.
• "Track Object Location" (ALFWorld): monitor object states in embodied environments.
• "Capture Action Constraints": encode precondition–object pairs from task instructions.

These learned skills emerge adaptively from data, without requiring manual specification or domain priors. The final skill bank tends to reflect high-level semantic memory operations surfaced by the agent’s task distribution and error patterns.

6. Comparative Results and Generalization

In direct comparison against state-of-the-art baselines (No-Memory, Chain-of-Notes, ReadAgent, MemoryBank, A-MEM, Mem0, LangMem, MemoryOS), MemSkill consistently yields higher F1, L-J, and success rates across all evaluated settings. The largest gains are observed in unseen or distribution-shift scenarios (e.g., transfer from LoCoMo training to LongMemEval or HotpotQA), corroborating the advantages of skill-driven, data-adaptive memory management for LLM agents.

The observed improvements—L-J of 51.0/59.4 on LoCoMo/LongMemEval, and unseen-ALF SR of 64.2—support the efficacy of the closed-loop skill evolution pipeline. Ablation-style analyses underscore the necessity of both skill evolution and controller-driven selection; removing either component reduces both raw accuracy and generalization to shifted distributions (Zhang et al., 2 Feb 2026).

7. Analysis, Limitations, and Implications

MemSkill demonstrates that structured, evolvable skills—curated via an LLM-powered designer and optimized with policy-gradient selection—can supplant brittle, hand-crafted memory routines in LLM agents. The approach offers three main advantages:

• Adaptivity: Actions for memory construction grow to match domain requirements, free from human preconceptions.
• Efficiency: One-pass, joint application of multiple skills reduces LLM invocation cost.
• Transferability: The framework retains efficacy across LLM architectures and task families, with no retraining required.

A plausible implication is that as skill banks expand and are clustered by emergent need, they will form robust, reusable components for future agentic systems, reducing engineering effort and increasing interpretability. However, skill bank growth may introduce retrieval bottlenecks or increase complexity in long-horizon deployments; managing such scaling effects remains an open problem.