Live-Evo: Online Evolution of Agentic Memory from Continuous Feedback

Abstract: LLM agents are increasingly equipped with memory, which are stored experience and reusable guidance that can improve task-solving performance. Recent \emph{self-evolving} systems update memory based on interaction outcomes, but most existing evolution pipelines are developed for static train/test splits and only approximate online learning by folding static benchmarks, making them brittle under true distribution shift and continuous feedback. We introduce \textsc{Live-Evo}, an online self-evolving memory system that learns from a stream of incoming data over time. \textsc{Live-Evo} decouples \emph{what happened} from \emph{how to use it} via an Experience Bank and a Meta-Guideline Bank, compiling task-adaptive guidelines from retrieved experiences for each task. To manage memory online, \textsc{Live-Evo} maintains experience weights and updates them from feedback: experiences that consistently help are reinforced and retrieved more often, while misleading or stale experiences are down-weighted and gradually forgotten, analogous to reinforcement and decay in human memory. On the live \textit{Prophet Arena} benchmark over a 10-week horizon, \textsc{Live-Evo} improves Brier score by 20.8\% and increases market returns by 12.9\%, while also transferring to deep-research benchmarks with consistent gains over strong baselines. Our code is available at https://github.com/ag2ai/Live-Evo.

• The paper proposes Live-Evo, an online system for agentic memory evolution that continuously learns from real-world tasks and feedback, achieving a 20.8% reduction in Brier score and a 12.9% increase in market returns on the Prophet Arena benchmark.
• Live-Evo consists of two banks: an Experience Bank storing past interactions, and a Meta-Guideline Bank for transforming them into task-adaptive guidance utilising a retrieval and compilation loop that favours practical over semantic overlaps to enhance agent decision-making.
• Contrastiveeval is a novel component that tracks guided vs unguided performance of tasks to continuously refine how agents retrieve, interpret, and act on previous insights.

Live-Evo addresses a structural gap in self-evolving LLM agent memory: most existing evolution pipelines are built around static train/test splits and only approximate online learning by folding static benchmarks, which makes them brittle under genuine distribution shift. The paper proposes an online memory system that learns from a continuous stream of tasks and from feedback that is revealed over time, evaluated on Prophet Arena, a live future-prediction benchmark where tasks arrive weekly and outcomes resolve against real-world events (2602.02369).

Motivation and problem setting

The authors argue that memory evolution is inherently an online problem. In deployment, experience accrues sequentially, and memory must be continually extended, revised, and pruned rather than rebuilt from a fixed corpus. Live benchmarks such as Prophet Arena (Yang et al., 20 Oct 2025) and FutureX (Zeng et al., 16 Aug 2025) instantiate this setting: agents forecast probability distributions over upcoming events and are scored with calibration metrics (Brier score) and decision-oriented metrics (market returns relative to prediction-market prices). Prior online-memory work, notably Evo-Memory/ReMem (Wei et al., 25 Nov 2025), approximates streaming by splitting static datasets into folds; Live-Evo instead operates on tasks sampled from the real world, where environments and markets drift continuously. In this regime, the authors contend, success depends less on retrieving more information than on judiciously curating past experience — useful experiences provide inductive bias, but stale or misleading ones actively harm performance.

Method

Live-Evo maintains two banks that decouple what happened from how to use it: an Experience Bank $\mathcal{E}$ storing structured past task interactions, and a Meta-Guideline Bank $\mathcal{M}$ storing meta-heuristics for transforming retrieved experiences into task-adaptive guidance. Each incoming task triggers a four-stage loop:

• Retrieve: the agent generates search queries (rather than using the raw question) targeting both question matching and experience-content matching, enabling retrieval of structural analogies rather than pure semantic overlap. Experiences are ranked by $Score = Weight \cdot Sim(exp, query)$, where weights are learned online.
• Compile: retrieved experiences are distilled into a task-specific guideline $g$, conditioned on a selected meta-guideline $\hat{m}$ — extracting cross-experience regularities, grounding them in the current task, and instantiating actionable guidance.
• Act: the agent executes with the guideline, and additionally performs ContrastiveEval — re-solving the same task without the compiled guideline — to measure the causal contribution of memory on that specific task.
• Update: the contrastive gain $(r^{on} - r^{off})$ adjusts the weights of the retrieved experiences upward or downward, implementing reinforcement-and-decay dynamics analogous to human memory. If the guideline underperforms ($r^{on} - r^{off} \le 0$), reflection produces a new meta-guideline appended to $\mathcal{M}$.

Memory growth is controlled through selective acquisition: after each batch, the worst-performing fraction of tasks ($\rho = 0.3$) is summarized into candidate experiences, which are committed to the Experience Bank only if re-evaluation confirms a statistically significant improvement (minimum Brier improvement 0.05). This "verify before update" protocol ensures every new entry is justified by measurable gain.

Experimental results

Evaluation covers 10 weeks of Prophet Arena (500 tasks), with strict time-based filtering on web search to prevent leakage past event close time, using GPT-4.1-mini as the default backbone. Headline results:

Metric	Base agent	Live-Evo	Improvement
Brier score (↓)	0.19	0.14	20.8%
Market return (↑)	1.24	1.46	12.9%

Live-Evo achieves the best average Brier score among all baselines, including open-source deep-research frameworks (MiroFlow at 0.32, Qwen Deep Research at 0.20) and ReMem (0.16). Under a simplified strategy investing $100 per week, Live-Evo yields roughly $150 more cumulative return over 10 weeks, with the gap widening over time — evidence that decision quality improves as experience accumulates rather than degrading under drift. Gains are largest in weeks where the base agent is poorly calibrated (Weeks 5–6).

Across backbones, improvements hold for GPT-4.1-mini (20.8% Brier improvement), GPT-4.1 (3.0%), GPT-5-mini (4.5%), and Qwen3-8B (3.5%), with market-return gains of 12.9%, 4.4%, 1.6%, and 0.5% respectively. The authors attribute the larger gains on weaker models to greater headroom and denser failure signals early in the stream; stronger models leave less room because their predictions are already well calibrated. This is a plausible but untested explanation — no analysis isolates failure-signal density as the causal factor.

On Xbench-DeepResearch, where the benchmark is split into 10 sequential folds, Live-Evo reaches 46% accuracy versus 45% for MiroFlow, 43% for Qwen-DeepResearch, and 40% for ReMem, suggesting that experience management learned under non-stationary live conditions transfers to conventional deep-research tasks without task-specific heuristics.

Ablations confirm each component contributes: removing weight updates degrades Brier score by 17.01% and return by 8.01%; removing active retrieval costs 14.97% and 16.77%; removing guideline compilation causes the largest return drop (−20.40%), underscoring that converting stored experience into actionable guidance — not storage itself — drives the market gains. A case study illustrates the weight dynamics concretely: an experience containing a hallucination (retrieving speech content when the task was predicting its outcome) is progressively down-weighted, while a reusable guideline about analyzing recent match form is reinforced.

Limitations and open questions

The authors concede two constraints directly. First, the system depends on dense environmental feedback (continuous scores such as Brier residuals); applicability to sparse or subjective reward settings is unaddressed. Second, the verify-before-update admission criterion can delay adoption of subtle or emerging heuristics, since candidates must show statistically significant immediate improvement before being committed. Two further caveats bear on interpretation: closed-source deep-research systems were excluded from baselines because their search tools lack time-based filtering, so the comparison against frontier research agents remains incomplete; and the generalization study reports small margins for strong backbones (e.g., 1.6% return gain for GPT-5-mini) over a single 10-week window, leaving open whether these gains persist across longer horizons or different market regimes. Whether the contrastive-evaluation overhead — solving each task twice — remains acceptable at scale is also not analyzed.

Conclusion

Live-Evo reframes agentic memory evolution as a genuinely online problem grounded in continuous real-world feedback. Its central design — separating experience content from learned usage policies, and scoring both via contrastive evaluation against a memory-free baseline — yields a 20.8% Brier-score improvement and 12.9% higher market returns on Prophet Arena, with consistent if smaller gains across four backbone models and transfer to deep-research evaluation. The results indicate that feedback-driven curation of how memory is used, rather than accumulation of memory itself, is the operative mechanism behind the observed gains in non-stationary environments.