Enabling LLMs to Improve Through Experience

Determine effective approaches for enabling large language models to improve their performance through accumulated experience from prior tasks and interactions.

Background

The paper highlights that while LLMs demonstrate strong reasoning and code-generation capabilities, a key challenge is how to enable these models to improve based on prior experience, rather than relying solely on single-pass prompting or costly parameter updates.

The authors discuss reinforcement learning as one mechanism for agent improvement but note its computational and data intensity. They motivate lightweight, training-free strategies such as Reflexion and their proposed Multi-Agent Reflexion (MAR) as steps toward addressing the broader open problem of experience-driven improvement in LLMs.

References

LLMs has evolved to generate strong reasoning traces and high-quality code, but enabling them to improve through experience remains an open problem.

— MAR:Multi-Agent Reflexion Improves Reasoning Abilities in LLMs  (2512.20845 - Ozer et al., 23 Dec 2025) in Introduction, first paragraph

These gains should be interpreted as in session reuse rather than be seen as evidence of persistent learning. Resumed sessions retain conversation history, measurements and generated scripts, and our experiment does not isolate their individual contributions to the speedup. Nevertheless, the improvement on rearranged scenes is promising: the agent can reuse prior work without repeating the full discovery process. This motivates future experiments on whether that experience can be consolidated into skills that benefit fresh sessions.

— Generalizing Manipulation Skills with a Local Coding Agent  (2609.26499 - Talwar et al., 22 Sep 2026) in Section 4, subsection “Second Pass Results” (immediately preceding Table II)