Are Self-Evolving Agents Continual Learners?
Abstract: LLM agents are often deployed in environments where new knowledge, tools, and workflows keep arriving. Such an agent is therefore expected to adapt continually, learning new tasks as they arrive without forgetting earlier ones. This resembles the continual learning problem, but at a pace that conventional gradient updates cannot sustain. Nonparametric self-evolving agents are a natural alternative: they adapt by rewriting their own harness, such as prompts, memory, and skills, from task feedback while the model weights stay frozen. We study whether such agents are effective continual learners across four drifting task streams that shift in topic, workflow, or domain (built from BIRD, BFCL, and ScienceWorld). Evaluating three representative methods spanning experience retrieval and self-modification (ReasoningBank, GEPA, and Meta-Harness), we find that although self-evolution improves aggregate performance on most streams, higher plasticity consistently comes at the cost of stability. Specifically, self-modifying methods apply global edits to a shared prompt or workspace, interfering with unseen tasks and overwriting earlier knowledge as the stream shifts. Conversely, similarity-based memory retrieval localizes updates across distant domains, but still retrieves foreign memories on unseen tasks and suffers from retrieval collision when similar queries require conflicting workflows. These findings show that current self-evolving agents remain incomplete continual learners.
Summary
- The paper assesses three self-evolving methods (ReasoningBank, GEPA, and Meta-Harness) to determine if they can continually learn new tasks while preserving previously learned skills, using metrics like terminal advantage and average advantage.
- GEPA achieves the highest plasticity but incurs the most severe stability loss, indicating a trade-off between adapting to new tasks and retaining previous knowledge
- The study shows that self-evolution can improve performance, but current methods fail to balance plasticity and stability effectively, leading to significant interference and forgetting.
- Find the latest research on continual learning in self-evolving agents?
Research question and conceptual framework
“Are Self-Evolving Agents Continual Learners?” examines whether nonparametric self-evolving LLM agents satisfy the central requirement of continual learning: acquiring competence on incoming task families while preserving performance on previously learned and not-yet-seen families (2610.25858). The paper focuses on agents whose model parameters remain frozen and whose adaptation occurs exclusively through the harness: system prompts, external memories, skill files, and controller code.
This formulation is important because self-evolution operates at the timescale of deployment. Unlike gradient-based continual learning, which requires parameter updates and often substantial data or computation, harness-level adaptation can occur after individual episodes. The paper formalizes an evolver that maps the current harness and accumulated experience to a new harness. The resulting agent policy is therefore determined by the fixed LLM and a changing nonparametric state.
The authors distinguish two adaptation paradigms. Self-modification rewrites globally shared artifacts, as in GEPA and Meta-Harness. Experience augmentation stores reflections or trajectories externally and retrieves them at inference time, as in ReasoningBank. Both mechanisms can improve performance on the task that generated the update, but they expose different interference mechanisms when the task distribution changes.
The study defines continual learning over task families and evaluates the complete policy sequence rather than only the final checkpoint. This is a consequential methodological choice: a method that eventually relearns a task after damaging it may obtain a reasonable terminal score while performing poorly over much of the deployment horizon. Performance is therefore measured relative to the seed harness at multiple checkpoints using terminal advantage and average advantage. The paper further decomposes adaptation into within-block plasticity and cross-block stability. Plasticity measures improvement on the currently trained group; stability is the negative sum of pre-block interference on unseen groups and post-block forgetting on earlier groups.
Experimental design
The evaluation uses Gemini 3.6 Flash with a common ReAct-style seed harness and compares three self-evolving methods:
- ReasoningBank stores one memory item per episode and retrieves the top-$1$ item by query-embedding cosine similarity.
- GEPA rewrites a shared strategy section in the system prompt using reflective mutations over collected rollouts.
- Meta-Harness edits a persistent workspace containing the system prompt, memories, skill files, and controller code, accepting edits through a validation gate.
The task streams are constructed from BIRD text-to-SQL, BFCL function-calling, and ScienceWorld interactive science tasks. Each task group contributes 50 training instances, with disjoint evaluation sets of 50 instances, or 90 for ScienceWorld. The four schedules isolate progressively different distribution shifts:
| Stream | Ordered task groups | Shift represented | Horizon |
|---|---|---|---|
| Topic | BIRD-F1 → BIRD-Code → BIRD-TP | Database schema and topic | 150 |
| Workflow | BFCL-Miss → BFCL-Base | Conflicting interaction policies | 100 |
| Domain | ScienceWorld → BFCL-Base → BIRD-F1 | Environment and action space | 150 |
| Compound | BIRD-Code → BIRD-F1 → BFCL-Miss → BFCL-Base | Topic, domain, and workflow shifts | 200 |
The streams are block-structured, but the agent does not receive group labels or explicit block-boundary information. Consequently, every update must be generated from episode feedback and inferred environmental regularities.
Aggregate performance and the plasticity–stability trade-off
Self-evolution improves aggregate performance in most settings, but the gains are modest relative to the degradation incurred elsewhere. Averaged over the four drifting streams, GEPA achieves the highest terminal advantage, percentage points, followed by ReasoningBank at and Meta-Harness at . GEPA also has the highest average advantage, points, compared with for ReasoningBank and for Meta-Harness. Topic drift is the exception: terminal advantage is for GEPA, for ReasoningBank, and $1$0 for Meta-Harness.
These aggregate results conceal a systematic trade-off. GEPA obtains the highest average plasticity, $1$1 points, including $1$2 on workflow drift and $1$3 on compound drift. It also incurs the largest average stability loss, $1$4 points. ReasoningBank has substantially lower plasticity, $1$5 points on average, but the best stability, $1$6. Meta-Harness has $1$7 plasticity and $1$8 stability overall, although its behavior varies sharply across schedules.
Figure 1: Plasticity and stability exhibit a negative relationship across methods and drift scenarios.
The reported plasticity–stability frontier is not merely a difference between weak and strong optimizers. It reflects the scope of the mutable state. GEPA’s single strategy prompt is present in every rollout, so a rule learned from the active group immediately affects all subsequent groups. ReasoningBank localizes updates more effectively because memories are injected only when retrieved, although that localization depends on the quality of the embedding space. Meta-Harness has the largest editable surface and can modify the full execution substrate, but this freedom increases the risk that task-specific changes become global.
The apparent stability of Meta-Harness on topic and workflow drift is partly an artifact of its validation gate. The gate rejects at least half of candidate updates in these settings, effectively freezing the workspace and limiting plasticity. When the gate is removed and the best candidate is accepted at every checkpoint, stability deteriorates to $1$9 on topic drift and 0 on workflow drift. Thus, conservative update selection can suppress interference, but it does so by restricting learning rather than by resolving task conflict.
Figure 2: Per-group learning curves show that gains on the active block often coincide with degradation on groups outside that block.
The implication is that aggregate stream advantage is insufficient as an evaluation criterion. GEPA’s strong terminal and average gains coexist with the most severe cross-group damage, while ReasoningBank’s more conservative behavior yields better preservation with weaker adaptation. A continual learner must be evaluated on the joint trajectory of acquisition, interference, and forgetting.
Self-modification: global interference and overwriting
The central failure mode of GEPA and Meta-Harness is that their updates are globally applied. A task-specific instruction discovered during one block becomes part of the prompt or workspace used for every later task, including task groups that have not yet been encountered.
The effect is substantial under domain and compound drift. After training on ScienceWorld, GEPA reduces BFCL-Base performance by 1 points at the first domain-transition checkpoint; Meta-Harness reduces it by 2. In compound drift, training on the two BIRD blocks lowers BFCL-Miss performance by 3 points under GEPA and 4 under Meta-Harness before any BFCL-Miss training example has appeared. The evolved prompts contain mandatory SQLite exploration, file inspection, and interactive testing instructions. These instructions are appropriate for BIRD but bias the agent away from BFCL-Miss’s requirement to ask users for missing function arguments.
Even related task groups interfere. Training on BIRD-F1 and BIRD-Code reduces BIRD-TP performance by 5 points under GEPA and 6 under Meta-Harness before BIRD-TP training begins. This result contradicts the assumption that topic-local updates within a common task family are necessarily benign. Shared tools and task formats do not guarantee compatible local conventions.
Post-block forgetting arises when later updates compress or overwrite earlier instructions. On topic drift, GEPA’s BIRD-F1 rules account for 100% of the strategy prompt after the first block, 75.2% after BIRD-Code, and only 40.2% after BIRD-TP. BIRD-F1’s advantage correspondingly falls from 7 to 8 and then to 9. On compound drift, BIRD-Code’s prompt share falls from 100% after its own block to 14.7% at the end of the stream. The transition to BFCL-Miss condenses seven BIRD SQL sections totaling 3,386 characters into one 1,034-character section; BIRD-F1’s prompt share drops from 59.5% to 5.8% and its earlier 0 gain disappears.
Figure 3: GEPA’s prompt occupancy tracks the displacement of earlier task-specific rules as later blocks are optimized.
The paper also identifies an important conditional behavior. GEPA sometimes grows the prompt additively when task domains are sharply separated. In domain drift, its strategy expands from 4,372 characters after ScienceWorld to 7,621 after BIRD-F1 while preserving all six ScienceWorld sections verbatim; this yields zero measured post-block forgetting for ScienceWorld. The same mechanism fails in compound drift, where BIRD and BFCL transitions prompt compression rather than addition. Additivity is therefore not a stable architectural property. It emerges opportunistically when the evolver detects sufficiently distinct domains and fails when successive tasks share surface structure.
Conflicting workflows
The most damaging case involves BFCL-Miss and BFCL-Base. Both use similar APIs and language, but they require opposite policies. BFCL-Miss rewards asking for clarification when arguments are missing, whereas BFCL-Base penalizes unnecessary clarification because all required arguments are supplied.
Before training, the seed agent asks for clarification on only 3.8%–5.8% of BFCL-Miss episodes and makes speculative tool calls on 91%–93%. After BFCL-Miss training, GEPA raises clarification behavior to 29.8% and obtains a 1-point within-block gain. Subsequent BFCL-Base training reverses the policy: clarification falls to 11.2%, or 45 out of 400 rollouts, and BFCL-Miss accuracy drops by 13.0 points, from 2 to 3. Meta-Harness exhibits the same behavioral reversal on compound drift: clarification falls from 60.8% to 8.5%, and BFCL-Miss performance declines by 10.2 points.
This result establishes that the issue is not simply limited prompt capacity. The two workflows require incompatible conditional behavior under highly similar observations. A global instruction surface cannot represent both policies unless it also contains a reliable task discriminator. Neither GEPA nor Meta-Harness discovers such a discriminator in these experiments.
Similarity-based retrieval: localization without compatibility
ReasoningBank avoids direct overwriting of the shared system prompt and therefore performs better when task domains are well separated. On domain drift, ScienceWorld, BFCL-Base, and BIRD queries have within-group nearest-neighbor cosine similarities of approximately 4–5, compared with 6–7 across groups. No post-block evaluation episode retrieves a memory from another domain, and measured post-block forgetting is zero.
However, top-8 retrieval without a rejection criterion creates pre-block interference by construction. Before a task group has generated any memories, every retrieved item must originate from a foreign group. Consequently, unseen groups receive foreign memories in 100% of evaluation episodes before their training blocks. This causes losses of 9 points for BFCL-Base after ScienceWorld training, 0 points for BFCL-Miss after the BIRD blocks in compound drift, and 1 points for BIRD-TP after earlier BIRD topics.
Figure 4: Foreign retrieval is harmless when domains are well separated but becomes frequent when task groups occupy overlapping embedding regions.
A similarity threshold could suppress low-confidence retrieval across distant domains, but it cannot solve the workflow-conflict case. BFCL-Miss and BFCL-Base have mean pairwise cosine similarity of 2, while their within-group similarities are 3 and 4. Their mean nearest-match similarities are 5–6, essentially identical to within-group values of 7–8. The embedding space therefore provides no threshold that reliably separates the two workflows.
At the end of workflow drift and compound drift, 52% of BFCL-Miss episodes retrieve BFCL-Base memories, while 46% of BFCL-Base episodes retrieve BFCL-Miss memories. BFCL-Miss performance drops by 8.0 points on workflow drift, from a 9 gain at the end of its own block to 0 at the end of the stream. The retrieved memories are semantically similar at the query level but behaviorally incompatible at the workflow level.
Figure 5: ReasoningBank’s foreign-memory rate is zero across many separated domains but remains high for overlapping BFCL workflows.
The finding places a precise limitation on retrieval-based continual learning: semantic similarity is not equivalent to procedural compatibility. A memory can be topically relevant while encoding the wrong action policy. Retrieval therefore requires conditioning on task identity, workflow constraints, or expected action semantics rather than relying only on query embeddings.
Interleaved streams and the persistence of shared-state contention
The authors additionally evaluate a uniformly interleaved mixture of the four compound-stream groups. Interleaving removes the explicit blocked sequence and gives every group repeated opportunities to generate experience. It does not eliminate shared-state contention.
GEPA’s terminal advantage falls from 1 on the blocked compound stream to 2 on the interleaved mixture. BIRD-F1 declines from 3 to 4, and BIRD-Code from 5 to 6. Prompt composition shows that BIRD-Code’s share decreases from 37.0% at position 50 to 5.8% at position 200, while BIRD-F1 and BFCL-specific rules occupy more of the shared strategy.
Figure 6: Uniform interleaving does not prevent competing task-specific updates from displacing one another in GEPA’s shared prompt.
This experiment rules out a narrow explanation based solely on one-pass block ordering. Even when all groups are continually represented in the input stream, a shared prompt remains a contested resource. Repeated exposure supplies additional feedback but does not provide a mechanism for conditional storage or conflict resolution.
Limitations and open questions
The evaluation is controlled in several respects. It uses one frozen base model, Gemini 3.6 Flash, three benchmark sources, binary task-success feedback, and short streams containing 50 training tasks per group. The conclusions therefore characterize the tested evolvers and task configurations rather than all nonparametric self-evolving agents. In particular, larger memory budgets, richer task metadata, alternative retrievers, or stronger validation procedures could alter the observed balance.
The task streams are also constructed with known benchmark groupings, even though those labels and boundaries are withheld from the agent. This permits precise measurement of pre-block interference and post-block forgetting, but real deployments may involve gradual drift, recurring tasks, mixed task identities, or unknown numbers of latent groups. The paper’s results leave open whether a method can infer reusable task contexts without explicit labels while avoiding the pathological global updates observed here.
The study evaluates top-7 retrieval for ReasoningBank and a particular prompt-editing configuration for GEPA. It does not test hierarchical memories, task-conditioned routing, confidence-based abstention, structured workflow representations, or explicit compatibility models. It also does not compare against parameter-updating continual-learning baselines under the same agentic environments. These omissions do not weaken the demonstrated failure modes, but they limit claims about which architectural intervention would best resolve them.
Finally, the plasticity–stability relationship is reported across 12 method–scenario combinations. The negative relationship is consistent in the presented experiments, but the sample is too small to establish a general scaling law. A specific unresolved question is whether task-conditioned partitioning can preserve GEPA-level plasticity while eliminating the global interference and retrieval collision documented in this study.
Conclusion
The paper provides a systematic continual-learning evaluation of nonparametric self-evolving agents across topic, workflow, domain, and compound drift (2610.25858). Self-evolution improves aggregate performance in most streams, but the methods do not jointly achieve strong adaptation and preservation. GEPA attains the highest plasticity by rewriting a shared prompt, at the cost of interference and overwriting. Meta-Harness exposes an even broader mutable surface and exhibits similar failures, with validation gates trading learning capacity for stability. ReasoningBank avoids direct prompt overwriting and preserves performance across separated domains, but similarity-based retrieval injects foreign memories and cannot distinguish semantically similar queries that require incompatible workflows.
The paper’s main conclusion is therefore precise: current self-evolving agents are useful online adapters but remain incomplete continual learners. Their central deficiency is not the absence of adaptation, but the absence of reliable mechanisms for isolating, routing, and conditionally applying task-specific knowledge.
Whiteboard
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Explain it Like I'm 14
1. ¿De qué trata el artículo?
El artículo estudia si los agentes de inteligencia artificial basados en modelos de lenguaje pueden aprender continuamente cuando reciben tareas nuevas una detrás de otra.
Por ejemplo, imaginemos un asistente que primero aprende a consultar una base de datos, después aprende a usar una aplicación y luego realiza experimentos científicos. El asistente debería aprender cada tarea nueva sin olvidar las anteriores.
Los investigadores analizan tres métodos que permiten que un agente se mejore a sí mismo sin cambiar las partes principales del modelo de lenguaje. En vez de volver a entrenar todo el modelo, modifican elementos externos, como:
- las instrucciones del sistema;
- una memoria de experiencias anteriores;
- una colección de habilidades;
- el código que controla al agente.
Estos elementos forman lo que el artículo llama el “arnés” (harness) del agente.
2. ¿Qué preguntas intentan responder?
La pregunta principal es:
¿Los agentes que se modifican a sí mismos son realmente buenos aprendiendo continuamente?
Para responderla, los investigadores examinan varias preguntas más concretas:
- ¿Puede un agente aprender una tarea nueva después de otra?
- ¿Puede conservar lo que aprendió antes?
- ¿Qué ocurre cuando las tareas nuevas pertenecen a temas o entornos diferentes?
- ¿Qué método aprende mejor las tareas nuevas?
- ¿Qué método causa menos problemas en las tareas antiguas?
- ¿Existe una relación entre aprender rápidamente y olvidar rápidamente?
En el artículo aparecen dos ideas importantes:
- Plasticidad: qué tan bien puede el agente aprender una tarea nueva.
- Estabilidad: qué tan bien puede conservar sus conocimientos anteriores mientras aprende.
Es parecido a un estudiante que aprende mucho sobre un tema nuevo, pero quizá empieza a confundirlo con temas que ya había estudiado.
3. ¿Cómo realizaron la investigación?
Las tareas utilizadas
Los investigadores crearon cuatro tipos de secuencias de tareas. Cada secuencia tenía varios grupos de tareas, presentados en bloques. Primero aparecía un grupo, después otro y así sucesivamente.
Usaron tareas de tres pruebas conocidas:
- BIRD: escribir consultas para buscar información en bases de datos.
- BFCL: usar herramientas y aplicaciones siguiendo instrucciones de varios pasos.
- ScienceWorld: realizar experimentos científicos en un entorno simulado.
Los cuatro tipos de cambios fueron los siguientes:
| Tipo de cambio | Qué cambia | Ejemplo |
|---|---|---|
| Cambio de tema | La tarea es parecida, pero trata sobre otro tema | Consultar tres bases de datos diferentes |
| Cambio de procedimiento | Las herramientas son parecidas, pero las reglas cambian | A veces pedir aclaraciones y otras veces no |
| Cambio de dominio | Cambia completamente el entorno | Pasar de una base de datos a un experimento científico |
| Cambio combinado | Cambian el tema, el procedimiento y el entorno | Una mezcla de todos los cambios anteriores |
Cada grupo tenía aproximadamente 50 tareas para que el agente aprendiera. Después, los investigadores lo probaron con otras tareas que no había visto antes.
Los tres métodos comparados
ReasoningBank
Este método guarda experiencias anteriores en una memoria externa. Cuando aparece una tarea nueva, busca la experiencia que parece más parecida y se la muestra al modelo.
Es como un estudiante que busca en sus apuntes el ejercicio anterior que más se parece al nuevo.
GEPA
GEPA cambia una parte compartida de las instrucciones principales del agente. Después de obtener comentarios sobre sus resultados, intenta reescribir esas instrucciones para mejorar.
Es parecido a escribir nuevas reglas en una guía que el estudiante usa para resolver todos los ejercicios.
Meta-Harness
Meta-Harness puede cambiar más cosas: las instrucciones, la memoria, las habilidades guardadas y el código de control del agente.
Es como permitir que el estudiante no solo cambie sus apuntes, sino también el manual de estudio y las herramientas que utiliza.
Cómo midieron el rendimiento
Los investigadores evaluaron al agente después de cada bloque de tareas. Observaron:
- cuánto mejoraba en el grupo que acababa de estudiar;
- cuánto empeoraba en grupos anteriores;
- si las instrucciones aprendidas para una tarea afectaban a tareas que todavía no había visto.
El modelo de lenguaje principal se mantuvo igual durante todo el experimento. Solo cambiaron su memoria, sus instrucciones y otros componentes externos. Esto permitió estudiar específicamente los métodos de autoevolución.
4. ¿Cuáles fueron los principales resultados?
Los agentes suelen mejorar, pero no de manera perfecta
En general, los tres métodos mejoraron el rendimiento en la mayoría de las secuencias. Sin embargo, ninguno logró aprender tareas nuevas y conservar perfectamente las anteriores.
El resultado principal fue una especie de intercambio:
Cuanto más cambia el agente para adaptarse a una tarea nueva, más riesgo tiene de dañar su rendimiento en otras tareas.
Este es el conflicto entre plasticidad y estabilidad.
GEPA aprendió mucho, pero también causó más interferencias
GEPA obtuvo, en promedio, las mayores mejoras al aprender la tarea activa. Sin embargo, sus cambios se escribían en una instrucción global compartida por todas las tareas.
Esto produjo dos problemas:
- Interferencia: una regla útil para una tarea afectaba negativamente a otra tarea que todavía no había aparecido.
- Olvido: las reglas nuevas podían reemplazar o comprimir las reglas aprendidas anteriormente.
Por ejemplo, una instrucción útil para consultar bases de datos podía hacer que el agente actuara de forma incorrecta en una aplicación que necesitaba otro procedimiento.
Meta-Harness también tuvo problemas con los cambios globales
Meta-Harness podía modificar más partes del agente, por lo que tenía mucha libertad para adaptarse. Pero esa libertad también podía causar daños mayores.
Al cambiar instrucciones, memoria y código compartidos, podía sobrescribir conocimientos anteriores. En algunos casos, aprendía correctamente a pedir aclaraciones cuando faltaban datos, pero después aprendía otra regla que le indicaba actuar directamente. Como resultado, olvidaba cuándo debía pedir ayuda.
ReasoningBank protegió mejor los conocimientos antiguos, pero mezcló experiencias
ReasoningBank fue más estable porque no cambiaba las instrucciones principales del agente. En su lugar, guardaba las experiencias por separado y solo recuperaba algunas cuando parecían relevantes.
Esto ayudó especialmente cuando las tareas pertenecían a dominios muy diferentes, como ciencia y bases de datos.
Sin embargo, su sistema de búsqueda también tenía problemas:
- podía recuperar una experiencia de una tarea que el agente todavía no había aprendido;
- dos tareas podían parecer muy similares aunque necesitaran procedimientos opuestos;
- una memoria sobre “pedir aclaraciones” podía mezclarse con otra que indicaba “usar la herramienta inmediatamente”.
Este problema se llama colisión de recuperación: el sistema encuentra una memoria que parece adecuada por las palabras utilizadas, pero que en realidad contiene una estrategia equivocada.
El caso más difícil: tareas parecidas con reglas opuestas
El problema fue especialmente grave cuando dos tareas usaban las mismas herramientas y un lenguaje parecido, pero exigían comportamientos diferentes.
Por ejemplo:
- en una tarea, si faltaba un dato, el agente debía preguntar al usuario;
- en otra tarea muy parecida, todos los datos ya estaban disponibles y preguntar innecesariamente era un error.
Como ambas tareas se parecían mucho, los métodos podían confundir sus reglas. El agente aprendía una estrategia y luego la cambiaba por la opuesta, perjudicando la tarea anterior.
5. ¿Por qué son importantes estos resultados?
Estos resultados muestran que hacer que un agente se modifique a sí mismo no es suficiente para convertirlo en un buen aprendiz continuo.
Los métodos actuales pueden mejorar el rendimiento rápidamente, pero todavía tienen dificultades para:
- separar los conocimientos de tareas distintas;
- saber cuándo una experiencia antigua sigue siendo útil;
- reconocer que dos preguntas parecidas pueden necesitar respuestas diferentes;
- evitar que una nueva regla borre una regla importante;
- mantener varias estrategias al mismo tiempo.
Esto es importante porque los agentes de inteligencia artificial se utilizan cada vez más en situaciones cambiantes. Un asistente real podría tener que usar nuevas herramientas, atender distintos tipos de usuarios y trabajar en diferentes entornos durante meses o años.
Si aprende una nueva regla y esa regla arruina sus habilidades anteriores, puede cometer errores inesperados.
Conclusión: ¿qué impacto podría tener esta investigación?
El artículo concluye que los agentes autoevolutivos actuales son aprendices continuos incompletos. Sí pueden aprender de sus experiencias, pero todavía no saben equilibrar bien el aprendizaje nuevo con la conservación de lo aprendido.
Los investigadores sugieren que los futuros agentes deberían:
- guardar conocimientos específicos en compartimentos separados;
- identificar el tipo de tarea antes de elegir una estrategia;
- recuperar memorias usando más información que la simple semejanza entre preguntas;
- proteger reglas importantes para que no sean sobrescritas;
- comprobar si una modificación mejora una tarea sin perjudicar a las demás.
En pocas palabras, el desafío no es solo conseguir que una inteligencia artificial aprenda. También hay que conseguir que aprenda algo nuevo sin olvidar ni confundir lo que ya sabía.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
- Limited benchmark coverage: The study evaluates only three benchmarks—BIRD, BFCL, and ScienceWorld—and four manually constructed streams, leaving unclear whether the findings generalize to other domains such as coding, web navigation, robotics, multimodal interaction, or long-horizon planning.
- Single-model evaluation: All experiments use Gemini 3.6 Flash, so it remains unknown whether the plasticity–stability trade-off persists across different model sizes, model families, reasoning capabilities, instruction-tuning methods, and open-weight versus proprietary models.
- Small stream sizes: Each task group contains only 50 training tasks, and streams contain at most 200 tasks. The behavior of self-evolving agents over substantially longer deployments, with thousands or millions of interactions, is unresolved.
- Restricted drift structure: The streams use contiguous, one-hot task blocks with abrupt transitions. The methods are not tested under gradual drift, recurring tasks, mixed-task streams, irregular transitions, or unknown and evolving task mixtures.
- No systematic variation of stream schedules: The paper provides limited evidence from an interleaved stream in an appendix, but does not systematically study how block length, task frequency, transition order, recurrence, or drift rate affect performance.
- Task labels and latent task structure remain underexplored: Although agents are not given explicit group labels, the experiments do not determine whether automatically inferred task identities, environment metadata, tool signatures, or latent context could enable safer routing and reduce interference.
- No comparison with continual-learning baselines: The study compares three self-evolving methods but does not evaluate rehearsal, regularization, task-conditioned routing, parameter-efficient fine-tuning, test-time training, mixture-of-experts methods, or hybrid parametric–nonparametric approaches under the same streams.
- No oracle or upper-bound comparisons: The experiments lack baselines with explicit task labels, oracle memory routing, separate harnesses per task family, or unlimited storage. Consequently, the extent to which failures arise from the evolver, task identification, retrieval, or shared-state constraints is not isolated.
- Limited method representation: Only one method represents similarity-based memory retrieval, one represents prompt rewriting, and one represents broad workspace modification. The conclusions may depend on the specific implementations and hyperparameters rather than on the paradigms themselves.
- Hyperparameter sensitivity is not fully characterized: The effects of memory capacity, retrieval top-, similarity thresholds, prompt budgets, edit frequency, validation-set size, mutation strategy, and workspace limits are not systematically ablated.
- Retrieval conditioning is narrowly studied: ReasoningBank relies primarily on query similarity and top-1 retrieval. The paper does not test metadata-aware retrieval, workflow-aware embeddings, tool-compatibility filters, reranking, diversity-based retrieval, episodic context, or learned retrieval policies.
- The role of memory quality is unresolved: The study does not disentangle the effects of memory content, reflection quality, redundancy, inaccurate trajectories, and reward sparsity. It remains unclear whether better memory distillation alone could substantially reduce retrieval collision.
- No explicit mechanisms for conflict resolution are evaluated: The paper identifies conflicting workflows but does not test policy versioning, contradictory-memory detection, precedence rules, uncertainty-aware selection, or conditional execution of incompatible instructions.
- Task-specific isolation mechanisms are proposed but not implemented: The conclusion calls for isolating updates in the harness, yet the study does not evaluate modular prompts, namespaces, skill routing, context-dependent controllers, or separate memory partitions.
- The causes of prompt compression and overwriting are not fully identified: The analysis shows changes in character shares and prompt contents, but does not determine whether forgetting is caused by context-window limits, the evolver’s optimization objective, validation data, model instruction-following behavior, or the prompt editor’s compression strategy.
- Workspace growth and resource costs are insufficiently analyzed: Additive updates can preserve knowledge but increase prompt or workspace size. The paper does not quantify latency, token consumption, monetary cost, execution time, storage growth, or the point at which additive memory becomes operationally infeasible.
- Validation-gated editing is not thoroughly examined: Meta-Harness’s apparent stability partly results from rejecting updates, but the study does not analyze validation-set leakage, overfitting, gate reliability, acceptance thresholds, or how validation performance predicts future cross-task performance.
- Evaluation checkpoints may miss transient failures: Harnesses are evaluated only at block boundaries, despite updates occurring after every episode. Short-lived interference, recovery dynamics, and instability within blocks remain unmeasured.
- Randomness and statistical uncertainty are underreported: The paper reports averages from eight rollouts per evaluation instance but does not clearly provide multiple independent seeds, confidence intervals for the main comparisons, significance tests, or variance across evolver runs.
- Binary task success is an incomplete feedback signal: Using only binary rewards may obscure partial progress, tool-use quality, safety violations, efficiency, answer calibration, and degrees of task completion.
- No analysis of transfer quality: Improvements are measured largely relative to seed performance, but the experiments do not distinguish positive transfer, negative transfer, task-specific memorization, and genuine acquisition of reusable strategies.
- Seed-harness dependence is unresolved: The reported gains and failures may depend strongly on the initial prompt and controller. Alternative seed harnesses, stronger handcrafted instructions, or different baseline capabilities are not evaluated.
- Model stochasticity and decoding settings are not systematically varied: It is unclear how temperature, sampling strategy, context-window behavior, and rollout diversity affect both self-evolution and measured stability.
- No safety or robustness evaluation: The study does not examine whether self-modification introduces unsafe tool calls, prompt injection vulnerabilities, data leakage, unauthorized workspace changes, or policy degradation under adversarial tasks.
- No human or operational deployment evaluation: The streams are offline benchmark sequences. The paper leaves open how self-evolving agents perform with real users, nonstationary user preferences, noisy feedback, delayed rewards, ambiguous requirements, and changing external tools.
- Feedback reliability is assumed: The methods receive task-success feedback after every training episode, but real deployments may provide sparse, delayed, biased, or incorrect feedback. The impact of noisy feedback on memory formation and prompt edits is untested.
- Unknown behavior under task recurrence: The study mainly presents forward-moving streams. It does not establish whether agents can recover previously learned workflows when an old task returns after long exposure to conflicting tasks.
- No principled objective for balancing plasticity and stability: The paper documents the trade-off but does not derive or optimize a unified objective that specifies how much forgetting is acceptable relative to learning new tasks.
- Metrics may obscure task-level asymmetries: Averaging across families can hide severe failures on particular groups, especially when baseline accuracies differ substantially. The adequacy of terminal advantage, average advantage, , and for deployment decisions remains unresolved.
- Stability is defined only as drops below seed or within-stream performance: The metrics do not capture recovery speed, cumulative regret, worst-case degradation, calibration, or whether an agent can intentionally trade performance across tasks according to user priorities.
- The relationship between semantic similarity and workflow compatibility is not formalized: The paper demonstrates retrieval collisions empirically but does not provide a model or diagnostic for identifying when two semantically similar tasks require incompatible policies.
- No learned or adaptive task-boundary detection is studied: Because the agent must infer drift without boundary information, future work is needed to determine whether explicit change-point detection or uncertainty signals can improve update timing and state isolation.
- Parametric and nonparametric adaptation are not directly compared: The paper motivates frozen-weight adaptation but does not determine when prompt, memory, code, or weight updates are preferable, or whether hybrid updates can achieve better plasticity–stability trade-offs.
- The generality of the plasticity–stability correlation is uncertain: The reported correlation is based on only 12 method–scenario observations, so its strength and consistency across larger method sets, models, task orders, and stream regimes remain to be established.
- Mechanisms for reliable additive behavior are unknown: GEPA sometimes preserves domain-specific sections additively and sometimes compresses them. The conditions that trigger each behavior are not identified, preventing reliable control of preservation versus consolidation.
- Long-term knowledge consolidation is unexplored: The study does not examine whether redundant experiences can be summarized, verified, and consolidated into durable abstractions without erasing task-specific exceptions.
- Open question: can agents distinguish general rules from local exceptions? The experiments show that workflows may conflict, but they do not test mechanisms that explicitly represent universal strategies, domain-specific rules, and exception conditions separately.
- Open question: can a shared harness remain effective with modular conditional execution? It remains unknown whether dynamically selecting prompts, memories, skills, and controller code based on task context can preserve the benefits of shared knowledge without global interference.
Practical Applications
Immediate Applications
- Safer deployment of adaptive LLM agents in software and enterprise automation
- Use the paper’s findings to avoid allowing an agent to rewrite a single global system prompt or controller after every task.
- A deployable workflow is to keep the base prompt and core controller immutable, while storing adaptations in isolated, versioned memory namespaces—for example, by customer, application, database schema, toolset, or workflow.
- This is relevant to coding agents, database assistants, customer-service automation, API orchestration, and enterprise copilots.
- Assumptions and dependencies: The system must be able to identify or infer task context reliably. Isolation is only useful if the routing metadata is accurate; otherwise, memories may still be applied to incompatible tasks.
- Context-aware memory retrieval with abstention
- Replace unconditional top-1 similarity retrieval with a retrieval policy that can return no memory when similarity is insufficient or when the retrieved experience conflicts with the current task.
- A practical implementation could use a combination of embedding similarity, tool compatibility, domain labels, workflow type, task history, and a confidence threshold.
- For example, a database-query memory should not be injected into a customer-support API task merely because both queries contain similar language.
- Assumptions and dependencies: Thresholds and compatibility classifiers require calibration on representative production traffic. Simple similarity scores are insufficient when tasks have similar wording but opposite procedures.
- Compatibility checks before applying learned instructions
- Add a pre-execution gate that tests whether a proposed memory, skill, or prompt edit is compatible with the incoming task’s tools, environment, required action space, and expected workflow.
- In API agents, this could distinguish “ask for missing arguments” from “execute immediately”; in SQL agents, it could distinguish database-specific schema rules.
- This can be implemented now as a lightweight policy layer that filters memories or selects among multiple prompt modules.
- Assumptions and dependencies: The agent or surrounding infrastructure must expose enough structured information about available tools, argument requirements, and environment state.
- Read-only evaluation and rollback for self-modifying agents
- Treat prompt, memory, skill, and controller updates as candidate versions rather than immediately applying them globally.
- Before deployment, evaluate each candidate on a regression suite covering previously supported task families, unseen tasks, and known conflicting workflows. Reject or roll back edits that improve the active task but reduce other critical capabilities.
- This extends the paper’s validation-gate idea into a production workflow with version control, canary testing, and automated rollback.
- Assumptions and dependencies: A representative, regularly maintained evaluation suite is required. Validation only protects against regressions that the suite measures.
- Continual-learning dashboards for agent operations
- Adopt the paper’s metrics—within-task plasticity, pre-task interference, post-task forgetting, terminal advantage, and average advantage—as operational monitoring measures.
- Organizations can track whether a new adaptation improves the target workflow while degrading unrelated workflows.
- A practical dashboard could report:
- success rate on the active task;
- performance on prior task families;
- foreign-memory retrieval rate;
- regression rate after each harness update;
- frequency of fallback or abstention.
- Assumptions and dependencies: Hold-out tasks and automated success criteria must be available. Binary task success, as used in the paper, may need to be supplemented by cost, latency, safety, and user-satisfaction measures.
- Controlled use of nonparametric adaptation for low-risk workflows
- Experience retrieval and prompt adaptation can be deployed now for relatively bounded applications such as SQL query formatting, internal documentation search, repetitive API calls, test generation, or simulation tasks.
- ReasoningBank-like methods are particularly suitable when task domains are well separated and the cost of an occasional retrieval error is low.
- Assumptions and dependencies: The task distribution should have clear domain separation, and human review or deterministic validation should exist for consequential outputs.
- Academic benchmarking of agent adaptation
- Researchers can use the paper’s drifting-stream setup to evaluate prompt optimizers, memory systems, coding agents, and tool-using models instead of reporting only stationary benchmark performance.
- New systems should be tested under topic drift, workflow drift, domain drift, and compound drift, with checkpoints throughout the stream rather than only at the end.
- This provides an immediately usable evaluation protocol for continual learning in LLM agents.
- Assumptions and dependencies: Benchmarks need disjoint evaluation tasks, reproducible task streams, and sufficient rollout repetitions to account for agent stochasticity.
- Policy and governance requirements for adaptive AI systems
- Regulators and internal governance teams can require that self-modifying agents maintain an audit trail of:
- which interaction caused an update;
- which prompt, memory, or code artifact changed;
- which task groups were evaluated;
- what regressions were detected;
- how the update can be reversed.
- This is especially relevant in healthcare administration, finance, education, and public-sector systems where silent behavioral drift can create compliance or safety problems.
- Assumptions and dependencies: Organizations must define acceptable regression thresholds and retain sufficient logs while respecting privacy and data-governance requirements.
- Daily-life assistants with user- or context-specific memory partitions
- Consumer assistants can maintain separate memories for contexts such as work, family, travel, health routines, or smart-home control rather than applying every learned preference globally.
- For example, a preference for concise work emails should not automatically alter responses in a child-education context, and a smart-home routine should not be retrieved for a travel-planning request.
- Assumptions and dependencies: Users must be given understandable controls for reviewing, editing, and deleting memories. Context classification errors could still cause inappropriate personalization.
Long-Term Applications
- Modular continual-learning architectures with isolated skill policies
- Develop agents that maintain a library of context-conditional modules rather than one shared evolving prompt or undifferentiated memory store.
- Each module could encode a task-specific workflow, tool policy, domain schema, or safety rule, with a router selecting the appropriate module at inference time.
- The resulting product could resemble a “skill operating system” for agents: immutable core reasoning plus versioned, composable skills.
- Assumptions and dependencies: Reliable task identification, module composition, conflict resolution, and limits on memory growth remain open research problems.
- Workflow-aware retrieval models
- Build retrieval systems that represent not only semantic content but also the action policy associated with an experience: required arguments, tool sequence, clarification behavior, success conditions, and environment state.
- Such representations could prevent the collision observed between BFCL-Miss and BFCL-Base, where similar requests required opposite workflows.
- Potential tools include structured trajectory indexes, workflow classifiers, graph-based memory stores, and retrieval models trained on successful and failed action sequences.
- Assumptions and dependencies: Systems need structured annotations or reliable automatic extraction of workflow properties. These models must also generalize to previously unseen tools and environments.
- Continual agents with explicit conflict detection and policy arbitration
- Future agents could detect when two memories prescribe incompatible actions and ask for clarification, defer to a higher-priority policy, or run a controlled comparison.
- In finance, for example, a conservative compliance rule could override a productivity shortcut. In robotics, a safety constraint could override a learned motion optimization.
- This would convert memory retrieval from passive context injection into explicit policy arbitration.
- Assumptions and dependencies: Conflict detection must be reliable enough for safety-critical use, and organizations must define policy priorities and escalation procedures.
- Adaptive agents for healthcare and clinical operations
- A mature version of the proposed architecture could support separate workflows for appointment scheduling, clinical documentation, medication reconciliation, and patient education while preserving strict separation between them.
- The agent could learn local procedures without allowing a scheduling adaptation to alter clinical guidance.
- Assumptions and dependencies: Deployment would require validated clinical workflows, privacy protection, human oversight, regulatory approval, and strong guarantees against cross-context contamination. The paper’s benchmark results alone do not establish clinical safety.
- Robotics and embodied systems that adapt across environments
- In robotics, isolated environment-specific skills could allow an agent to learn navigation, object manipulation, or laboratory procedures without overwriting policies for earlier environments.
- A domain-aware memory system could distinguish visual observations, action spaces, and safety constraints across robots or facilities.
- This is particularly relevant to household robots, warehouse automation, laboratory automation, and scientific simulators.
- Assumptions and dependencies: The approach must handle noisy rewards, partial observability, physical risk, changing hardware, and substantially longer-horizon tasks than those evaluated in the paper.
- Continual software engineering agents
- A future coding agent could maintain separate repositories of project-specific conventions, build systems, APIs, and testing workflows, while sharing general programming skills.
- The agent would need to prevent a convention learned in one repository from contaminating another and preserve older project knowledge after learning a new framework.
- Possible products include repository-scoped memory, branch-aware skill management, regression-test-driven prompt evolution, and automatically generated project playbooks.
- Assumptions and dependencies: Repository context, dependency versions, access permissions, and automated tests must be available. Code changes still require security scanning and human or CI-based approval.
- Energy and infrastructure control systems
- Continual agents could adapt to changing equipment, weather regimes, operating procedures, or grid conditions while keeping safety-critical control policies isolated.
- For example, a data-center cooling agent might learn a new facility’s operating constraints without modifying the policies used for another facility.
- Assumptions and dependencies: Deployment requires high-fidelity simulators, formal safety constraints, fail-safe fallback controllers, and extensive testing under rare or adversarial conditions.
- Finance and compliance systems with auditable policy memory
- Financial agents could maintain jurisdiction-, product-, and institution-specific procedures without merging incompatible rules.
- A workflow-aware system could distinguish “request missing customer information” from “execute an already authorized transaction,” reducing the kind of behavioral reversal identified in the paper.
- Assumptions and dependencies: Rules change frequently and may conflict across jurisdictions. Any deployment would require versioned policies, explainability, auditability, data protection, and human authorization for consequential actions.
- New benchmarks and standardized certification for adaptive agents
- The paper’s plasticity–stability framing could support an industry-wide certification suite for agents that learn after deployment.
- Certification could require performance under:
- unseen-task interference;
- sequential domain changes;
- conflicting workflows;
- memory contamination;
- prompt and workspace overwriting;
- recovery after rollback.
- This would shift evaluation from “does the agent improve on the current task?” to “does it improve without harming the rest of its competence?”
- Assumptions and dependencies: Standardized task families, realistic drift schedules, reproducible environments, and agreed safety thresholds are needed.
- Hybrid parametric–nonparametric continual learning
- The paper studies frozen model weights, but its findings motivate systems that decide whether new knowledge belongs in a memory, a modular prompt, a skill artifact, or the model parameters themselves.
- Stable, broadly reusable knowledge might eventually be consolidated into model weights, while volatile or context-specific procedures remain externally stored and isolated.
- Assumptions and dependencies: Such systems require reliable consolidation criteria, protection against catastrophic forgetting during weight updates, privacy controls, and methods for detecting when external memories have become obsolete.
Glossary
- Action space: The set of actions available to an agent in an environment. “The environment itself changes along with its observation and action spaces”
- Catastrophic forgetting: The loss of previously acquired knowledge when a model learns new tasks. “must acquire each without degrading performance on earlier ones, known as catastrophic forgetting”
- Checkpoint: A saved model or harness state used for later evaluation. “we compute accuracy at checkpoints”
- Continual learning: Learning successive tasks while retaining performance on earlier tasks. “This form of learning resembles the classical continual learning problem”
- Cosine similarity: A measure of similarity between vectors based on the angle between them, commonly used for comparing embeddings. “queries from ScienceWorld, BFCL-Base, and BIRD have much higher within-group than cross-group nearest-match cosine similarity”
- Cross-block forgetting: Performance degradation on a task group after subsequent task blocks have been learned. “post-block forgetting , the drop from the end of family 's block”
- Cross-task interference: The impairment of one task caused by updates learned for another task. “Self-modification causes cross-task interference and overwriting”
- Distribution shift: A change in the statistical properties of data or tasks encountered by a model. “we construct four drifting streams that isolate three types of distribution shift”
- Drifting stream: A task stream whose task distribution changes over time. “We instantiate the task stream as a drifting stream”
- Embedding space: A vector space in which data items are represented numerically according to their semantic or structural properties. “distinct task groups occupy the same region of the embedding space”
- Experience augmentation: Adapting an agent by storing and retrieving prior experiences during later executions. “experience augmentation appends past reflections, successful trajectories, or tool libraries to an external store”
- Experience retrieval: The process of selecting stored prior experiences for use on a new task. “we evaluate three representative methods spanning experience retrieval and self-modification”
- Evolver: A function or mechanism that updates an agent’s harness using accumulated experience. “After formalizing the evolver, the function that updates the harness”
- Fine-tuning: Additional training of a pretrained model on a smaller, task-specific dataset. “fine-tuning on a small number of recent interactions is known to erode general capabilities”
- Gradient update: A parameter update computed from the gradient of a training objective. “at a pace that conventional gradient updates cannot sustain”
- Harness: The collection of prompts, memories, skills, and control code surrounding and operating an underlying model. “an agent can update its harness, everything wrapped around the model”
- Hold-out set: Data reserved for evaluation rather than used during adaptation or training. “on a disjoint hold-out set”
- In-context learning: The ability of a model to adapt its output behavior from examples or instructions provided in the input context without changing its parameters. “The in-context learning ability of LLMs”
- Inference time: The period during which a trained model is used to generate outputs. “Self-evolving agents are therefore natural candidates for continual learning at inference time.”
- Interference: A reduction in performance caused by an update intended for another task or task group. “It combines pre-block interference ”
- Memory bank: An external storage system containing experiences that can be retrieved during subsequent tasks. “Storing experiences in an external memory bank”
- Meta-Harness: A self-modifying method that edits an agent’s prompt, memories, skills, and controller code. “Meta-Harness, which proposes edits to a persistent workspace”
- Nonparametric: Describing adaptation that changes external state or structures rather than the model’s learned parameters. “We study the nonparametric case, in which is frozen and only the harness is updated”
- One-hot: A representation in which exactly one category has a nonzero value. “ is one-hot and piecewise-constant”
- Parametric: Describing adaptation that changes a model’s learned parameters or weights. “Parametric counterparts instead update the weights at test time”
- Plasticity: The capacity to learn or improve on a currently encountered task. “Plasticity measures how much each family learns”
- Policy: A mapping from inputs or observations to a distribution over actions or response trajectories. “A stochastic policy ”
- Post-block forgetting: The loss of performance on a task group after later blocks have been processed. “post-block forgetting , the drop from the end of family 's block”
- Pre-block interference: Performance degradation on a task group before that group’s own training block begins. “pre-block interference , the mean drop below the seed at checkpoints before family 's block begins”
- Query similarity: A measure of the semantic or vector-based resemblance between an incoming query and stored queries. “Retrieving experiences by query similarity”
- ReAct-style loop: An agent-control procedure that alternates between observing the environment and taking actions. “use Gemini 3.6 Flash in a ReAct-style loop”
- Reflective mutation: An iterative modification process in which an agent uses feedback or reflection to alter its instructions or behavior. “rewrites a shared strategy section in the system prompt by reflective mutation over collected rollouts”
- Retrieval collision: A failure in which similar retrieval representations cause incompatible experiences to be selected for a task. “suffers from retrieval collision when similar queries require conflicting workflows”
- Reward function: A function that assigns a numerical evaluation to an agent’s response or trajectory. “ is a reward function”
- Rollout: A complete execution or interaction trajectory generated by an agent in an environment. “reflective mutation over collected rollouts”
- Self-modification: Adaptation in which an agent rewrites its own prompts, code, or other operational artifacts. “Self-modification rewrites the artifacts on which the agent runs”
- Self-evolving agent: An agent that updates its external operating structures from feedback or experience while operating. “Nonparametric self-evolving agents are a natural alternative”
- Semantic similarity: Similarity based on meaning or representation rather than exact lexical overlap. “Semantic similarity between queries captures topical overlap”
- Stability: The ability to preserve performance on other task groups while learning a current task. “Stability measures the degradation a family incurs”
- Stateful API: An application programming interface that maintains information about prior interactions or requests. “multi-turn user requests served by calling stateful APIs”
- System prompt: The high-level instruction provided to an LLM that governs its behavior. “the system prompt, the accumulated memory, the skill libraries”
- Task family: A set of related tasks sharing an input distribution, output space, and reward function. “We define a task family as a tuple”
- Task-free continual learning: Continual learning in which task identities and explicit task boundaries are not provided to the learner. “mirroring task-free continual learning scenarios”
- Task stream: A time-ordered sequence of tasks or inputs encountered by a learner. “A task stream is a sequence of inputs”
- Terminal advantage: The average performance improvement over the initial harness at the end of a stream. “the terminal advantage (), which averages ”
- Topic drift: A change in the subject matter of tasks while other task properties remain relatively constant. “The environment and workflow stay fixed while the underlying topic changes”
- Trajectory: The complete sequence of states, actions, and observations produced during an episode. “the full interaction trajectory generated in the task environment”
- Workflow drift: A change in the procedure required to complete tasks correctly. “the interaction workflow required for a correct response changes”
- Workspace: A persistent collection of editable files or artifacts used by an agent. “Meta-Harness modifies a shared global workspace”