Experience-Guided Reasoner (EGuR)

• Experience-Guided Reasoner (EGuR) is a framework that adapts inference strategies at runtime by leveraging a structured memory of past experiences.
• The system generates complete computational procedures—including prompt templates, sampling parameters, tool configurations, and control logic—for each new query.
• EGuR enhances performance in tasks like combinatorial search and retrosynthesis by learning from execution feedback to improve both accuracy and cost-efficiency.

An Experience-Guided Reasoner (EGuR) is a framework for agentic problem-solving systems that adapt and optimize their inference-time strategies based on accumulated experience, enabling dynamic tailoring of reasoning procedures for each new query. Unlike traditional architectures, which apply fixed, static strategies and update only textual memory or require offline retraining, EGuR can generate complete, ready-to-run computational procedures at inference time—modulating prompts, sampling hyperparameters, tool usage, and control flow—based on structured memory of prior episodes. This design closes the loop between strategy generation, execution, feedback, and continual memory refinement, with applications ranging from LLM reasoning to combinatorial search in retrosynthesis and other domains (Stein et al., 14 Nov 2025, Hong et al., 2021).

1. Motivation and Distinguishing Features

Most LLM-based and agentic AI systems historically deploy a manually designed or preselected “strategy” (the combination of prompts, tool usage, and system configuration) that is independent of prior inference outcomes. Such an approach repeatedly incurs avoidable costs on recurring inputs and fails to learn from unsuccessful attempts on difficult problems. EGuR addresses this by promoting the notion of a “reasoning strategy” to a first-class object that can be generated, adapted, and selected at inference time, conditioned on what the agent has learned from past interactions. Notably, EGuR adapts not only textual prompts, but all aspects of the end-to-end reasoning pipeline—sampling parameters, the choice and configuration of external tools, and the internal control logic dictating procedural flow. This capability distinguishes EGuR from methods that update only superficial state or require retraining to adapt their tactics (Stein et al., 14 Nov 2025).

2. System Architecture

EGuR comprises two tightly coupled components: the Guide and the Consolidator.

• Guide (Meta-Strategy Generator): The Guide is an LLM-based process conditioned on the current problem $q$, a structured memory $\Sigma$ of previous problem-strategy-outcome tuples, and an exploration budget $k$. For each input $q$, the Guide outputs $k$ candidate strategies $\{S_1, \ldots, S_k\}$. Each $S_i$ is a formal specification incorporating a set of prompt templates $P$, sampling parameters $\theta$ (such as temperature or top-$k$), tool selection or configuration $\Sigma$0, and control logic $\Sigma$1, collectively $\Sigma$2. Unlike typical agent prompts, the Guide outputs “code” or declarative strategy specifications which can be directly executed (Stein et al., 14 Nov 2025).
• Consolidator (Memory Updater): After candidate strategies are executed—each producing an answer $\Sigma$3, incurring computational cost $\Sigma$4, and receiving correctness or reward feedback $\Sigma$5—the Consolidator updates the memory $\Sigma$6 by abstracting and retaining useful strategy fragments, metalevel notes, and execution statistics. The memory structures are bounded in size and selectively pruned to maintain relevance and computational tractability.

The high-level inference-time pseudocode is:

$\{S_1, \ldots, S_k\}$1 (Stein et al., 14 Nov 2025)

This separation of concern mirrors and extends ideas from experience-guided planning methods in other domains, such as retrosynthetic search (Hong et al., 2021).

3. Formalism: Strategy and Memory Representation

Each strategy $\Sigma$7 is parameterized as:

$\Sigma$8

where:

• $\Sigma$9: an explicit prompt template or set thereof,
• $k$0: sampling parameters (e.g., temperature, top-$k$1),
• $k$2: tool configuration (including which external tools or interpreters to employ),
• $k$3: control logic (branching and looping constructs for orchestrating LLM and tool calls).

Upon receiving an input $k$4, executing $k$5 in context $k$6 yields an answer $k$7 and a possibly updated state $k$8: $k$9

Each strategy is scored by a utility-weighted metric: $q$0 where $q$1 is the reward (e.g., binary correctness), $q$2 is cumulative cost, $q$3 the episode history, and $q$4 a cost-accuracy tradeoff parameter (Stein et al., 14 Nov 2025).

The experience memory $q$5 (or $q$6) has two substructures:

• Strategy Library ($q$7): Tuples $q$8 for successful runs, where $q$9 denotes problem features.
• General Notes ($k$0): Abstracted heuristics, e.g., task/tool/parameter interactions.

Memory is updated via a function $k$1 that conditions inclusion or removal on observed reward, cost, and relevance budget constraints.

4. Mechanisms of Adaptation and Learning

The core EGuR loop adapts strategies via direct integration of episodic feedback. For each new query, multiple candidate strategies are generated and executed; the first’s output is used, but all executions are logged. Performance and resource data (accuracy, cost) plus verifier feedback are aggregated. The Consolidator promotes full strategies or abstracted heuristics with empirically demonstrated utility to the strategy library or notes, and eliminates stale/outdated entries to control memory size.

In its current instantiation, the Guide is implemented via a prompt-based mechanism (“prompt engineering”) with few-shot or zero-shot LLM calls: $k$2 No gradient-based fine-tuning of LLM parameters $k$3 is performed, though this is a plausible extension. Guidance is provided by in-context retrieval from the strategy library, using embedding-based similarity on $k$4 features to seed the meta-strategy with relevant precedent (Stein et al., 14 Nov 2025).

5. Empirical Evaluation and Results

EGuR was benchmarked on five challenging reasoning tasks, including AIME 2025 mathematics (trained on 2022–2024, evaluated on a held-out 2025 set), 3-SAT (combinatorial satisfiability, 5–40 variables), and three BigBench Extra Hard tasks (movie recommendation, word sorting, object counting). Baselines included CodeAct (stateless agent), Dynamic Cheatsheet, and Mem0.

Key evaluation metrics:

• Prequential Accuracy: accuracy measured before memory update
• Cumulative Cost per Sample: measured in USD, capturing the cost of LLM/tool queries

Table: EGuR (k=5) Performance Summary (Stein et al., 14 Nov 2025)

Task	EGuR-5 Accuracy	Best Baseline Accuracy	EGuR-5 Cost	Baseline Cost	Relative Gains
3-SAT	96.0%	77.0%	$k$50.26	+19 pp, 1.7× che.
AIME	58.9%	61.1%	$k$60.48	–2.2 pp, 1.6× che
Obj. Counting	96.0%	87.0%	$k$70.174	+9 pp, 2.3× che.

On 3-SAT, EGuR achieved 19 pp higher accuracy and 1.7× cost reduction. Across all benchmarks, EGuR yielded up to 14 percentage points in accuracy gains and up to 111× cost reductions, with both metrics monotonically improving as experience accrues.

Ablations revealed that increasing the exploration factor $k$8 (i.e., number of candidate strategies generated and evaluated per query) substantially accelerated the discovery of efficient strategies. With $k$9 (no memory updates) performance deteriorated, highlighting the importance of continual adaptation.

6. Application to Retrosynthetic Planning

The EGuR approach is instantiated in Experience-Guided Monte Carlo Tree Search (EG-MCTS) for retrosynthetic analysis (Hong et al., 2021). Here, multi-step retrosynhetic search is framed as an AND-OR tree exploration problem. The Experience Guidance Network (EGN) learns from both successful and failed synthetic routes, providing node-specific Q-value estimates that bias Monte Carlo tree exploration towards promising decompositions, away from dead ends.

• State: Set of molecules remaining to be solved; each molecule is an OR node.
• Actions: Reaction templates (AND nodes).
• Guidance: EGN outputs a Q-value given molecular and template fingerprints; this Q-value directly modulates tree policy via a UCT-style selection criterion.
• Learning: EGN is trained via self-play, aggregating empirical Q-values across attempted decompositions.

This method produced higher solution rates and shorter synthetic routes than prior methods (e.g., Retro*+), and offers a recipe generalizable to combinatorial search problems (theorem proving, program synthesis) sharing the same search structure (Hong et al., 2021).

7. Limitations and Future Directions

Identified limitations include reliance on strong “oracle” verifiers to assess candidate answers (with uncertain generalization when only weak LLM-based critics are available), heuristic nature of the Consolidator’s memory management (rather than formal meta-learning objectives), and potential failures of the zero-shot Guide on out-of-distribution problem types. Additionally, the caching/retrieval mechanism depends on robust feature representations, and prompt-based Guides may be suboptimal compared to approaches involving model fine-tuning or reinforcement learning.

Potential extensions include:

• Incorporating RL or supervised learning to optimize the Guide’s strategy distribution $\{S_1, \ldots, S_k\}$0,
• Designing formal compression and memory retention objectives for the Consolidator,
• Relaxing assumptions on verifier strength,
• Adapting similar meta-strategy mechanisms to other combinatorial and agentic reasoning domains.

EGuR constitutes an emergent framework for continual, self-improving agentic inference, enabling on-the-fly, resource-aware adaptation of reasoning pipelines without any offline retraining (Stein et al., 14 Nov 2025, Hong et al., 2021).