From Memory to Skills: Evidence-Grounded Co-Evolution Governance for Long-Horizon LLM Agents

Abstract: Existing memory systems for long-horizon LLM agents often retrieve prior traces as passive context rather than converting them into executable capabilities. In this paper, we propose MSCE, a training-free Memory--Skill Co-Evolution framework that organizes agent experience into grounded step traces, reusable procedural policies, and declarative environmental cognition. MSCE crystallizes evidence-backed L2 policies with positive estimated gain into callable skills that retain evidence links, applicability boundaries, decision guidance, verification rules, and reliability estimates. It further introduces reflection-weighted value backfilling, which propagates sparse terminal feedback through dense local self-reflections to produce evidence-calibrated trace values for governing memory and skill evolution. Experiments on EvoAgentBench and LoCoMo demonstrate that MSCE significantly outperforms state-of-the-art skill-augmented and memory-driven agent baselines, exhibiting strong cross-domain transferability and lifelong-evolution capabilities.

• The paper introduces MSCE, a training-free governance framework that promotes grounded traces into verified, reusable skills through a three-level memory hierarchy, calibrated values, and lifecycle controls.
• The framework achieves best or tied-best Pass@1 across five EvoAgentBench domains, including a 15.39-point software-engineering improvement, while also improving LoCoMo memory scores and cross-domain transfer by an average of 3.93 points.
• The results show that skill crystallization and structured memory are critical for efficient reuse, but heuristic credit signals, LLM-dependent update operators, runtime specificity, and privacy risks remain important limitations.

Overview

MSCE (Memory–Skill Co-Evolution) is a training-free framework that converts long-horizon LLM agent experience from passive retrievable context into governed, executable skills (2607.16621). The central premise is that existing memory systems—whether factual stores of raw interactions or experiential stores of reflections and summaries—are consumed as context that the agent must re-reason over at every invocation. MSCE instead treats memory as a governed substrate from which procedural policies are induced, verified, and promoted into callable skill objects, with every promotion anchored to concrete interaction evidence.

Three-level memory hierarchy

The framework organizes experience into three levels that separate evidence, procedure, and environmental knowledge:

• L1 trace memory: grounded step-level records $(s_{i,t}, a_{i,t}, o_{i,t}, \rho_{i,t})$ augmented with a value $V$ backfilled after terminal feedback arrives. Storage is bounded: observations are truncated, deduplicated, and redacted; higher levels store evidence identifiers rather than raw content.
• L2 policy memory: reusable procedures $(\phi, \pi, \kappa, \mathcal{B})$—trigger, natural-language procedure, verification rule, applicability boundary—induced only when a candidate signature accumulates evidence from at least $n_{\min}$ distinct episodes, preventing a single trajectory from producing an over-specific policy.
• L3 environmental cognition: declarative facts about environment structure, action–response regularities, and constraints, abstracted from cohorts of active L2 policies and explicitly forbidden from containing imperative instructions.

This separation is what distinguishes MSCE from concurrent memory-skill hybrids such as MemSkill and ProcMEM: integration is framed as a governed promotion problem—what becomes a skill, when it applies, and how it is revised or retired.

Skill crystallization

An L2 policy is crystallized into a skill only after passing two gates: positive estimated gain $G(f^{(2)}) > \theta_G$, computed as the difference between softmax-weighted mean trace values on policy-linked traces and a shrunk baseline over non-linked traces; and stability, meaning recent evidence fits the current trigger, procedure, and boundary without substantial rewrite. The resulting skill inherits evidence anchors, decision guidance (preferences and anti-patterns), and a reliability estimate $\eta$ computed as a smoothed success rate $(n_{\text{pass}}+1)/(n_{\text{trial}}+2)$.

A deterministic verifier gates deployment: schema validation, evidence-identifier grounding against the support set, tool whitelisting from observed traces, and two coverage tests ensuring no unsupported commands. Failing drafts are discarded rather than exposed. Skills move through probationary → active → archived lifecycle states driven by $\eta$, user corrections, and source-policy gain. At inference, retrieved L3 cognitions instantiate skill parameters (e.g., choosing apk vs. apt-get based on detected OS) without overwriting the procedure.

Reflection-weighted value backfilling

The core credit-assignment mechanism couples dense but noisy step-level self-reflections with sparse but reliable terminal feedback $R_i$. Step values follow:

$$V(f_{i,t}^{(1)}) = \alpha_{i,t} R_i + (1-\alpha_{i,t})\gamma V(f_{i,t+1}^{(1)})$$

where $V$0 is scored by an LLM judge along faithfulness, causal insight, transferability, and concreteness dimensions. High-$V$1 steps retain more terminal reward directly; low-$V$2 steps inherit value from successors. When no numerical reward exists, textual feedback is quantified into $V$3 via weighted goal achievement (0.45), process quality (0.30), and user satisfaction (0.25) scores. These calibrated values serve as the unified signal for retrieval prioritization, L2 association/induction, L3 abstraction triggers, and skill promotion or retirement.

Main results

On EvoAgentBench (GPT-5.2 backbone, GPT-4o auxiliary operators, OpenClaw runtime), MSCE achieves best or tied-best Pass@1 across all five domains:

Domain	Best non-MSCE baseline	MSCE	Δ
Information Retrieval	21.54	26.15	+4.61
Mathematical Reasoning	43.00	47.00	+4.00
Software Engineering	38.46	53.85	+15.39
Code Implementation	61.54	61.54	0 (cost 3.9→2.0 turns)
Knowledge Work	48.28	53.45	+5.17

The SE gain of +15.39 points is the largest single improvement, though it comes with increased cost (37.3→40.8 turns), which the authors attribute to the additional navigation, editing, and testing inherent to successful software-engineering solutions. On LoCoMo long-dialogue memory, MSCE attains the best overall judge score (61.23) and F1 (49.89), exceeding SkillFlow-Evolve by 2.01 and 1.18 points respectively, with the strongest single-hop, multi-hop, and temporal reasoning performance.

Two generalization results stand out. Cross-domain transfer improves Pass@1 on all six evaluated transfer pairs (+2.56 to +5.13 points, average +3.93), indicating the framework learns transferable problem-solving structure rather than domain-specific artifacts. Under lifelong evolution, Pass@1 improves monotonically with accumulated experience scale, with gains up to 17.00 points on Math, while normalized cost eventually falls below the no-accumulation baseline—a "learning by using" effect where accuracy rises without proportional inference-cost growth.

Ablations

The ablation study isolates each component's contribution. Replacing the hierarchy with Flat Memory causes the largest degradation: −15.38 to −19.23 Pass@1 points across IR, Math, and SE, plus a Code cost increase from 2.0 to 5.3 turns. This supports the paper's claim that passive experience retrieval is fundamentally insufficient for reuse. Among component-level ablations, removing skill crystallization is most consistently damaging (−6.15 to −11.54 points, higher cost everywhere), followed by removing reflection weighting, L3 abstraction, and value calibration—the latter increasing cost in every domain, confirming that applicability-aware filtering prevents harmful skill injection.

A supplementary study injects LLM-simulated human feedback during training, improving Pass@1 on four of five domains (up to +13.79 on KW), indirectly validating the reward quantification pathway. Math shows zero change, suggesting its bottleneck lies in exact symbolic correctness rather than procedural learning.

Limitations and open questions

The authors concede several substantive limitations. First, the governance signals—trace values, policy gains, reliability estimates—are heuristic utility measures, not causal credit assignments; they support ranking and filtering but do not guarantee counterfactual efficacy of a promoted policy. Second, five update operators (reflection scoring, reward quantification, L2 induction, L3 abstraction, skill drafting) rely on prompted LLMs whose outputs, despite evidence-constrained verification, remain sensitive to model and prompt choices and add latency. Third, results depend on a specific runtime, tool interface, and backbone configuration, so absolute numbers may not transfer. Finally, truncation, deduplication, and rule-based redaction reduce but do not eliminate privacy risk from stored traces, secrets, or unsafe procedural knowledge. An open question the paper leaves unresolved is whether learned policies would survive counterfactual intervention—that is, whether positive observed gain reflects genuine causal contribution to task success.

Conclusion

MSCE demonstrates that treating memory-to-skill conversion as an evidence-governed promotion pipeline—rather than direct distillation from noisy trajectories—yields state-of-the-art self-evolution performance across heterogeneous agent domains, with measurable cross-domain transfer and lifelong-learning gains achieved training-free. Its principal contribution is architectural: separating evidence, procedure, and environmental cognition, then linking them through calibrated value signals and deterministic verification before any abstraction becomes callable.