SEER: Stepwise Experience Recall for LLMs

• SEER is an inference-time framework that enhances multi-step function calling in LLMs by retrieving and reusing successful tool-use trajectories.
• It improves performance through fine-grained retrieval based on trajectory similarity, toolchain coverage, and intent alignment, yielding measurable gains on benchmarks.
• SEER enables continual self-guided experience accumulation during deployment, reducing reliance on static demonstrations and manual prompt engineering.

Stepwise Experience Recall (SEER) is an inference-time framework for improving multi-step function calling in LLMs by retrieving and reusing prior successful tool-use trajectories at the level of the current interaction step, while continually growing the memory of those trajectories over time (Cui et al., 21 Aug 2025). It is defined for a tool-augmented LLM agent that interacts with a user and a set of external tools to achieve a goal $g \in \mathcal{G}$. The framework’s central claim is that multi-step tool-use failures arise not only because a model lacks tool descriptions, but because it lacks contextually appropriate prior experiences aligned with the current partial trajectory, the likely user intent, and the tools already being used. SEER therefore treats successful trajectories as reusable experiences and recalls them repeatedly during execution, conditioned on the evolving interaction history rather than only on the initial query (Cui et al., 21 Aug 2025).

1. Conceptual scope and problem regime

In SEER’s formulation, the interaction history up to time $t$ is

$H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$

and the full trajectory is

$\tau = \{o_0, a_0, o_1, a_1, \dots\}.$

At each step, the agent observes user input and/or tool outputs, chooses either a natural-language response or a tool invocation, and then receives the next observation (Cui et al., 21 Aug 2025). This is precisely the regime in which LLMs often struggle with multi-step function calling.

The method is motivated by three recurring failure modes: incorrect tool selection, incorrect parameter generation for tool calls, and poor tool-chain planning across multiple steps. These failures become acute when a task requires several dependent tool invocations, because the model must interpret the user query, decide what to do next based on partial progress, and condition on previous tool outputs. A single static few-shot prompt rarely captures all relevant tool combinations and edge cases, especially under context window limits (Cui et al., 21 Aug 2025).

SEER is explicitly described as a self-guided method. In this setting, “self-guided” means that the system improves itself online without parameter updates or manual data curation: after deployment, it evaluates its own completed trajectories, identifies successful ones, and adds them to memory so they can guide future decisions. The guidance comes from retrieval rather than gradient-based learning. This makes SEER an inference-time retrieval augmentation method rather than a fine-tuning, reinforcement-learning, or supervised-training procedure (Cui et al., 21 Aug 2025).

2. Motivation and distinction from prior retrieval schemes

The method is positioned against two common design patterns in tool-using LLM systems: manually designed demonstrations with task-specific prompt engineering, and retrieval from a curated static library. The paper argues that manually designed demonstrations are expensive and brittle, while curated retrieval libraries do not scale well because they depend on pre-collected, static examples (Cui et al., 21 Aug 2025).

A central point in the paper is that “task similarity” alone is not sufficient for multi-step tool use. Prior retrieval methods often operate at coarse granularity, typically matching either the user query or the whole task, without considering whether the retrieved example aligns with the current partial execution state or with the tool usage pattern already emerging in the current trajectory. SEER emphasizes two additional alignment signals beyond semantic similarity: tool-chain alignment and intent alignment (Cui et al., 21 Aug 2025).

This yields the main conceptual contrast with static demonstration retrieval. Static retrieval libraries store whole demonstrations offline and retrieve them from the current input query. SEER instead maintains an experience pool that expands over time and retrieves trajectories stepwise, using the current trajectory prefix as the retrieval key. The most useful prior experience is therefore not assumed to be the example most similar at the initial-query level, but the one whose current prefix resembles the current history, overlaps in tool usage, and reflects the same inferred user intent (Cui et al., 21 Aug 2025).

Relative to specific baselines, SEER is described as differing from standard prompting baselines such as CoT and ReAct by adding a memory mechanism over prior successful trajectories; from ART by using finer-grained retrieval than task-level similarity; from ExpeL by relying on an online-updated experience pool and stepwise recall from current history; and from RAG-style retrieval by recalling trajectory experiences with a composite relevance score over trajectory state, toolchain overlap, and intent alignment rather than retrieving knowledge documents or static demonstrations solely by semantic similarity (Cui et al., 21 Aug 2025).

3. Architecture and formal representation

The framework has three named components: trajectory experience extraction, stepwise experience recall, and continual experience accumulation (Cui et al., 21 Aug 2025).

Component	Function	Representation
Trajectory experience extraction	Converts a completed interaction into structured memory	$$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$
Stepwise experience recall	Retrieves relevant past trajectories for the current partial history	top-$k$ recalled trajectories from $\mathcal{D}$
Continual experience accumulation	Inserts successful new trajectories into memory	$\mathcal{D} \gets \mathcal{D} \cup \{(\tau, d)\}$

Trajectory experience extraction maps a completed trajectory to

$$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle.$$

Here, $E^{\tau}$ is the embedding of the interaction trajectory $t$0; $t$1 is the embedding of the user’s first query, namely the first observation $t$2; $t$3 is the inferred user intent selected from a predefined discrete intent set $t$4; and $t$5 is the tool invocation sequence represented as a directed path

$t$6

An experience in SEER is therefore not raw text alone, but a tuple containing the original trajectory together with its trajectory embedding, initial-query embedding, inferred intent, and tool-use path (Cui et al., 21 Aug 2025).

The intent variable is not manually annotated. Instead, the LLM itself classifies the intent from the initial query using a prompt-based intent recognizer. The appendix prompt asks the model to classify a question into one of eight intent categories and output the predicted category in the form $t$7. For successful-completion judgment, the evaluator $t$8 is implemented with an LLM-as-a-judge style prompt that compares the correct answer and the model response and returns either $t$9 or $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$0, while tolerating minor formatting differences and slight numerical deviations (Cui et al., 21 Aug 2025).

At inference time, for each current partial history $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$1, the agent constructs an analogous representation $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$2, recalls relevant trajectories from the experience pool $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$3, and inserts those recalled trajectories as in-context exemplars to guide the next decision $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$4. The default foundational model is Qwen2.5-72B-Instruct, with Qwen2.5-7B-Instruct also used on $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$5-bench. For retrieval embeddings, the system uses bge-large-en-v1.5. Temperature is set to $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$6; unless stated otherwise, top-$H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$7 is $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$8, $H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},$9, and the maximum interaction horizon is $\tau = \{o_0, a_0, o_1, a_1, \dots\}.$0 for ToolQA and $\tau = \{o_0, a_0, o_1, a_1, \dots\}.$1 for $\tau = \{o_0, a_0, o_1, a_1, \dots\}.$2-bench (Cui et al., 21 Aug 2025).

4. Retrieval scoring and continual memory growth

SEER’s retrieval score has three components. The first is trajectory similarity,

$\tau = \{o_0, a_0, o_1, a_1, \dots\}.$3

which is the normalized cosine similarity between the embedding of the current interaction history and the embedding of a candidate past trajectory $\tau = \{o_0, a_0, o_1, a_1, \dots\}.$4. The normalization maps cosine similarity into $\tau = \{o_0, a_0, o_1, a_1, \dots\}.$5. The intended effect is to recover trajectories with similar structural progression and decision patterns rather than merely similar initial wording (Cui et al., 21 Aug 2025).

The second component is toolchain coverage,

$\tau = \{o_0, a_0, o_1, a_1, \dots\}.$6

which measures how much of the tool set already used in the current task is covered by the candidate trajectory. When computing this score, the directionality of $\tau = \{o_0, a_0, o_1, a_1, \dots\}.$7 is ignored and the toolchain is treated as an unordered set of tools. The paper’s interpretation is that similar tool usage often reflects similar operational strategies even when the query wording differs (Cui et al., 21 Aug 2025).

The third component is intent match,

$\tau = \{o_0, a_0, o_1, a_1, \dots\}.$8

a binary score indicating whether the inferred user intent of the current history matches that of the candidate trajectory. The final relevance score is

$\tau = \{o_0, a_0, o_1, a_1, \dots\}.$9

with $$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$0 controlling the contribution of the three terms; in the experiments, all three are set equally to $$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$1. Candidate trajectories are sorted by this score, and the top-$$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$2 are returned as $$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$3 (Cui et al., 21 Aug 2025).

The second algorithmic mechanism is continual experience accumulation. After task completion, the system decides whether the trajectory should be added to memory. If $$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$4 returns true, the method extracts the experience tuple

$$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$5

and inserts $$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$6 into the experience pool:

$$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$7

This makes the experience pool dynamic rather than static (Cui et al., 21 Aug 2025).

The memory is initialized with a small number of demonstrations and then expanded online with new successful trajectories. On ToolQA, the demonstration pool for ART, ExpeL, and SEER is initialized with the same 8 examples. On $$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$8-bench, the initial pool contains 2 examples. The maximum experience pool size is set to 1000. No eviction rule, deduplication rule, or compression strategy is described beyond this cap. A plausible implication is that retrieval quality and memory-management policy become increasingly important as the system is deployed for longer periods (Cui et al., 21 Aug 2025).

5. Empirical performance and ablation findings

The main experiments are conducted on ToolQA and $$d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle$$9-bench. ToolQA contains 800 easy questions from 55 templates and 730 hard questions from 62 templates across eight domains: flight, coffee, Yelp, Airbnb, DBLP, SciREX, agenda, and GSM8K-related numerical reasoning. The hard split is important because it requires more complex multi-step tool usage. $k$0-bench evaluates realistic multi-turn tool-agent-user interaction across two domains. The paper’s table reports 115 tasks and 15 tools for one domain and 50 tasks and 13 tools for the other, and presents two domains totaling 165 tasks (Cui et al., 21 Aug 2025).

On ToolQA, SEER achieves the best average accuracy across tasks, with 67.9% on easy and 31.1% on hard. The strongest baseline in the main table is ExpeL at 61.8% easy and 26.4% hard, so SEER improves by 6.1 points on easy and 4.7 points on hard (Cui et al., 21 Aug 2025). Domain-level numbers reported for SEER are: Flight 82.0 easy and 25.0 hard; Coffee 87.0 easy and 41.5 hard; Yelp 90.0 easy and 58.0 hard; Airbnb 94.0 easy and 33.0 hard; DBLP 37.0 easy and 35.0 hard; SciREX 6.0 easy and 23.0 hard; Agenda 68.0 easy and 2.0 hard; GSM8K 79.0 easy. Compared with ExpeL, the method is described as especially strong on Yelp, Airbnb, Coffee-Hard, DBLP-Hard, and Agenda-Easy, with slight drops on Flight-Hard and SciREX-Hard relative to the best baseline, and a noticeable drop on Coffee-Easy attributed to “suboptimal and cumbersome examples” that caused overthinking on simpler tasks (Cui et al., 21 Aug 2025).

On $k$1-bench, the reported gains are larger for open-source models. Qwen2.5-7B-Instruct improves from 9.34% average to 16.78% with SEER, an absolute gain of 7.44 points. Qwen2.5-72B-Instruct improves from 28.46% to 51.84%, a gain of 23.38 points. Domain-wise, the 7B model rises from 8.16% to 20.41% on Airline and from 10.53% to 13.16% on Retail, while the 72B model rises from 30.61% to 38.78% on Airline and from 26.32% to 64.91% on Retail. The paper highlights that SEER with Qwen2.5-72B reaches 51.84%, approaching GPT-4o’s 54.76% (Cui et al., 21 Aug 2025).

The self-improvement analysis is central to the “self-guided” characterization. On ToolQA, all 1,530 questions are shuffled and divided into 10 batches of 153. After each batch, all correctly answered instances are added to the experience pool and become available for the next batch. The batch accuracy is defined as

$k$2

and the reported trend is that SEER starts at 37.7% on the first batch, reaches 52.3% by the fifth batch, and 54.9% on the last batch. This is presented as evidence that the growing experience pool steadily improves performance over time (Cui et al., 21 Aug 2025).

The ablation studies isolate the value of the retrieval design. Replacing full-trajectory similarity with query-only similarity yields 67.5% easy and 30.5% hard, slightly below full SEER’s 67.9% easy and 31.1% hard. Removing toolchain coverage $k$3 yields 64.3% easy and 25.5% hard, the largest hard-set drop among the ablations. Removing intent match $k$4 yields 56.1% easy and 30.1% hard, which hurts easy questions most dramatically. The paper interprets these results as evidence that trajectory-level retrieval, toolchain overlap, and intent alignment each contribute materially to accuracy (Cui et al., 21 Aug 2025).

The number of retrieved demonstrations also matters. For full SEER, top-$k$5 yields 37.63% easy and 19.04% hard; $k$6 yields 58.88% and 27.26%; $k$7 gives the best overall reported balance at 67.88% and 31.51%; $k$8 drops to 65.88% and 27.12%; and $k$9 gives 67.13% and 30.14%. The overall pattern is that a moderate number of demonstrations helps most, whereas too many examples overwhelm the model and degrade performance. This supports the method’s argument for selective, fine-grained retrieval rather than simply increasing context size (Cui et al., 21 Aug 2025).

6. Practical significance, limitations, and acronym disambiguation

Practically, SEER is described as useful in LLM systems that must repeatedly solve related tool-using tasks in deployment and can observe some notion of success afterward. Examples given in the paper include customer support agents, data-querying assistants, booking agents, and enterprise workflows with repeated tool patterns. Its appeal is that it can improve over time without retraining the base model, and it appears particularly beneficial in settings with complex tool chains or weaker base models, as reflected in the large gains on $\mathcal{D}$0-bench and on hard ToolQA subsets (Cui et al., 21 Aug 2025).

The method also carries clear operational trade-offs. It introduces retrieval overhead at every interaction step because the current history must be scored against the experience pool. Its effectiveness depends on having prior successful trajectories; early in deployment, performance may be lower until the pool grows. Memory management and retrieval efficiency can become concerns as the pool expands, yet the paper specifies only a maximum size of 1000 and does not detail pruning or indexing strategies. The method further assumes access either to a reference answer or to a sufficiently reliable evaluator signal for determining success. In ToolQA, the evaluator compares the output to the known answer; in more open-ended real environments, this assumption may be harder to satisfy. The limitations section also notes that the diversity of memory is bounded by the underlying LLM’s capabilities and that fixed retrieval weights may be suboptimal across heterogeneous tasks (Cui et al., 21 Aug 2025).

The acronym “SEER” is overloaded in adjacent literatures and requires disambiguation. In "Self-Guided Function Calling in LLMs via Stepwise Experience Recall," SEER refers specifically to the stepwise retrieval-and-memory framework described above (Cui et al., 21 Aug 2025). By contrast, "SEER: Enhancing Chain-of-Thought Code Generation through Self-Exploring Deep Reasoning" expands SEER as “SElf-Exploring deep Reasoning” and concerns code generation with diverse reasoning path exploration, reasoning quality-aware model training, and adaptive CoT reasoning rather than experience recall over prior tool-use trajectories (Gao et al., 20 Oct 2025). Preference-based reinforcement learning work also uses SEER for a different method built around label smoothing and replay-aligned conservative value estimation (Bai et al., 2024). This acronym overlap can obscure the specific contribution of Stepwise Experience Recall, whose defining features are stepwise trajectory retrieval, intent-and-toolchain-aware relevance scoring, and continual online accumulation of successful tool-use experiences.