---
title: 'SEER: Stepwise Experience Recall for LLMs'
url: https://www.emergentmind.com/topics/stepwise-experience-recall-seer
type: topic
---

# SEER: Stepwise Experience Recall for LLMs

Stepwise Experience Recall (SEER) is an inference-time framework for improving multi-step function calling in large language models by retrieving and reusing prior successful tool-use trajectories at the level of the current interaction step, while continually growing the memory of those trajectories over time [2508.15214]. It is defined for a tool-augmented LLM agent that interacts with a user and a set of external tools to achieve a goal \(g \in \mathcal{G}\). The framework’s central claim is that multi-step tool-use failures arise not only because a model lacks tool descriptions, but because it lacks contextually appropriate prior experiences aligned with the current partial trajectory, the likely user intent, and the tools already being used. SEER therefore treats successful trajectories as reusable experiences and recalls them repeatedly during execution, conditioned on the evolving interaction history rather than only on the initial query [2508.15214].

## 1. Conceptual scope and problem regime

In SEER’s formulation, the interaction history up to time \(t\) is

\[
H_t = \{o_0, a_0, \dots, o_{t-1}, a_{t-1}, o_t\},
\]

and the full trajectory is

\[
\tau = \{o_0, a_0, o_1, a_1, \dots\}.
\]

At each step, the agent observes user input and/or tool outputs, chooses either a natural-language response or a tool invocation, and then receives the next observation [2508.15214]. This is precisely the regime in which large language models often struggle with multi-step function calling.

The method is motivated by three recurring failure modes: incorrect tool selection, incorrect parameter generation for tool calls, and poor tool-chain planning across multiple steps. These failures become acute when a task requires several dependent tool invocations, because the model must interpret the user query, decide what to do next based on partial progress, and condition on previous tool outputs. A single static few-shot prompt rarely captures all relevant tool combinations and edge cases, especially under context window limits [2508.15214].

SEER is explicitly described as a self-guided method. In this setting, “self-guided” means that the system improves itself online without parameter updates or manual data curation: after deployment, it evaluates its own completed trajectories, identifies successful ones, and adds them to memory so they can guide future decisions. The guidance comes from retrieval rather than gradient-based learning. This makes SEER an inference-time retrieval augmentation method rather than a fine-tuning, reinforcement-learning, or supervised-training procedure [2508.15214].

## 2. Motivation and distinction from prior retrieval schemes

The method is positioned against two common design patterns in tool-using LLM systems: manually designed demonstrations with task-specific prompt engineering, and retrieval from a curated static library. The paper argues that manually designed demonstrations are expensive and brittle, while curated retrieval libraries do not scale well because they depend on pre-collected, static examples [2508.15214].

A central point in the paper is that “task similarity” alone is not sufficient for multi-step tool use. Prior retrieval methods often operate at coarse granularity, typically matching either the user query or the whole task, without considering whether the retrieved example aligns with the current partial execution state or with the tool usage pattern already emerging in the current trajectory. SEER emphasizes two additional alignment signals beyond semantic similarity: tool-chain alignment and intent alignment [2508.15214].

This yields the main conceptual contrast with static demonstration retrieval. Static retrieval libraries store whole demonstrations offline and retrieve them from the current input query. SEER instead maintains an experience pool that expands over time and retrieves trajectories stepwise, using the current trajectory prefix as the retrieval key. The most useful prior experience is therefore not assumed to be the example most similar at the initial-query level, but the one whose current prefix resembles the current history, overlaps in tool usage, and reflects the same inferred user intent [2508.15214].

Relative to specific baselines, SEER is described as differing from standard prompting baselines such as CoT and ReAct by adding a memory mechanism over prior successful trajectories; from ART by using finer-grained retrieval than task-level similarity; from ExpeL by relying on an online-updated experience pool and stepwise recall from current history; and from RAG-style retrieval by recalling trajectory experiences with a composite relevance score over trajectory state, toolchain overlap, and intent alignment rather than retrieving knowledge documents or static demonstrations solely by semantic similarity [2508.15214].

## 3. Architecture and formal representation

The framework has three named components: trajectory experience extraction, stepwise experience recall, and continual experience accumulation [2508.15214].

| Component | Function | Representation |
|---|---|---|
| Trajectory experience extraction | Converts a completed interaction into structured memory | \(d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle\) |
| Stepwise experience recall | Retrieves relevant past trajectories for the current partial history | top-\(k\) recalled trajectories from \(\mathcal{D}\) |
| Continual experience accumulation | Inserts successful new trajectories into memory | \(\mathcal{D} \gets \mathcal{D} \cup \{(\tau, d)\}\) |

Trajectory experience extraction maps a completed trajectory to

\[
d^{\tau} = \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle.
\]

Here, \(E^{\tau}\) is the embedding of the interaction trajectory \(\tau\); \(E^q\) is the embedding of the user’s first query, namely the first observation \(o_0\); \(I^{\tau}\) is the inferred user intent selected from a predefined discrete intent set \(\mathcal{I}\); and \(U^{\tau}\) is the tool invocation sequence represented as a directed path

\[
u_1 \rightarrow u_2 \rightarrow \dots \rightarrow u_n.
\]

An experience in SEER is therefore not raw text alone, but a tuple containing the original trajectory together with its trajectory embedding, initial-query embedding, inferred intent, and tool-use path [2508.15214].

The intent variable is not manually annotated. Instead, the LLM itself classifies the intent from the initial query using a prompt-based intent recognizer. The appendix prompt asks the model to classify a question into one of eight intent categories and output the predicted category in the form \([INTENT]\). For successful-completion judgment, the evaluator \(\mathcal{E}\) is implemented with an LLM-as-a-judge style prompt that compares the correct answer and the model response and returns either \([Match]\) or \([No\ Match]\), while tolerating minor formatting differences and slight numerical deviations [2508.15214].

At inference time, for each current partial history \(H_t\), the agent constructs an analogous representation \(d^{H_t}\), recalls relevant trajectories from the experience pool \(\mathcal{D}\), and inserts those recalled trajectories as in-context exemplars to guide the next decision \(a_t\). The default foundational model is Qwen2.5-72B-Instruct, with Qwen2.5-7B-Instruct also used on \(\tau\)-bench. For retrieval embeddings, the system uses bge-large-en-v1.5. Temperature is set to \(0\); unless stated otherwise, top-\(k\) is \(4\), \(\lambda_1=\lambda_2=\lambda_3=1/3\), and the maximum interaction horizon is \(T=6\) for ToolQA and \(T=30\) for \(\tau\)-bench [2508.15214].

## 4. Retrieval scoring and continual memory growth

SEER’s retrieval score has three components. The first is trajectory similarity,

\[
s_1 = (1 + \cos(E^{H_t}, E^{\tau'}))/{2},
\]

which is the normalized cosine similarity between the embedding of the current interaction history and the embedding of a candidate past trajectory \(\tau'\). The normalization maps cosine similarity into \([0,1]\). The intended effect is to recover trajectories with similar structural progression and decision patterns rather than merely similar initial wording [2508.15214].

The second component is toolchain coverage,

\[
s_2 = |U^{H_t} \cap U^{\tau'}| \,/\, |U^{H_t}|,
\]

which measures how much of the tool set already used in the current task is covered by the candidate trajectory. When computing this score, the directionality of \(U\) is ignored and the toolchain is treated as an unordered set of tools. The paper’s interpretation is that similar tool usage often reflects similar operational strategies even when the query wording differs [2508.15214].

The third component is intent match,

\[
s_3 = 1[I^{H_t} = I^{\tau'}],
\]

a binary score indicating whether the inferred user intent of the current history matches that of the candidate trajectory. The final relevance score is

\[
\text{Score}^{\tau} = \sum_{i=1}^{3} \lambda_i s_i,
\]

with \(\lambda_1, \lambda_2, \lambda_3\) controlling the contribution of the three terms; in the experiments, all three are set equally to \(1/3\). Candidate trajectories are sorted by this score, and the top-\(k\) are returned as \(\mathcal{D}_{\rm recall}\) [2508.15214].

The second algorithmic mechanism is continual experience accumulation. After task completion, the system decides whether the trajectory should be added to memory. If \(\mathcal{E}.isSuccessful(H_t)\) returns true, the method extracts the experience tuple

\[
d^{\tau} \gets \langle E^{\tau}, E^q, I^{\tau}, U^{\tau} \rangle
\]

and inserts \((\tau, d^{\tau})\) into the experience pool:

\[
\mathcal{D} \gets \mathcal{D} \cup \{(\tau, d)\}.
\]

This makes the experience pool dynamic rather than static [2508.15214].

The memory is initialized with a small number of demonstrations and then expanded online with new successful trajectories. On ToolQA, the demonstration pool for ART, ExpeL, and SEER is initialized with the same 8 examples. On \(\tau\)-bench, the initial pool contains 2 examples. The maximum experience pool size is set to 1000. No eviction rule, deduplication rule, or compression strategy is described beyond this cap. A plausible implication is that retrieval quality and memory-management policy become increasingly important as the system is deployed for longer periods [2508.15214].

## 5. Empirical performance and ablation findings

The main experiments are conducted on ToolQA and \(\tau\)-bench. ToolQA contains 800 easy questions from 55 templates and 730 hard questions from 62 templates across eight domains: flight, coffee, Yelp, Airbnb, DBLP, SciREX, agenda, and GSM8K-related numerical reasoning. The hard split is important because it requires more complex multi-step tool usage. \(\tau\)-bench evaluates realistic multi-turn tool-agent-user interaction across two domains. The paper’s table reports 115 tasks and 15 tools for one domain and 50 tasks and 13 tools for the other, and presents two domains totaling 165 tasks [2508.15214].

On ToolQA, SEER achieves the best average accuracy across tasks, with 67.9% on easy and 31.1% on hard. The strongest baseline in the main table is ExpeL at 61.8% easy and 26.4% hard, so SEER improves by 6.1 points on easy and 4.7 points on hard [2508.15214]. Domain-level numbers reported for SEER are: Flight 82.0 easy and 25.0 hard; Coffee 87.0 easy and 41.5 hard; Yelp 90.0 easy and 58.0 hard; Airbnb 94.0 easy and 33.0 hard; DBLP 37.0 easy and 35.0 hard; SciREX 6.0 easy and 23.0 hard; Agenda 68.0 easy and 2.0 hard; GSM8K 79.0 easy. Compared with ExpeL, the method is described as especially strong on Yelp, Airbnb, Coffee-Hard, DBLP-Hard, and Agenda-Easy, with slight drops on Flight-Hard and SciREX-Hard relative to the best baseline, and a noticeable drop on Coffee-Easy attributed to “suboptimal and cumbersome examples” that caused overthinking on simpler tasks [2508.15214].

On \(\tau\)-bench, the reported gains are larger for open-source models. Qwen2.5-7B-Instruct improves from 9.34% average to 16.78% with SEER, an absolute gain of 7.44 points. Qwen2.5-72B-Instruct improves from 28.46% to 51.84%, a gain of 23.38 points. Domain-wise, the 7B model rises from 8.16% to 20.41% on Airline and from 10.53% to 13.16% on Retail, while the 72B model rises from 30.61% to 38.78% on Airline and from 26.32% to 64.91% on Retail. The paper highlights that SEER with Qwen2.5-72B reaches 51.84%, approaching GPT-4o’s 54.76% [2508.15214].

The self-improvement analysis is central to the “self-guided” characterization. On ToolQA, all 1,530 questions are shuffled and divided into 10 batches of 153. After each batch, all correctly answered instances are added to the experience pool and become available for the next batch. The batch accuracy is defined as

\[
Acc_j^{\text{batch}} = \frac{1}{|Q_{b_j}|} \sum_{i=1}^{|Q_{b_j}|} \mathbf{1}[y_i = y'_i],
\]

and the reported trend is that SEER starts at 37.7% on the first batch, reaches 52.3% by the fifth batch, and 54.9% on the last batch. This is presented as evidence that the growing experience pool steadily improves performance over time [2508.15214].

The ablation studies isolate the value of the retrieval design. Replacing full-trajectory similarity with query-only similarity yields 67.5% easy and 30.5% hard, slightly below full SEER’s 67.9% easy and 31.1% hard. Removing toolchain coverage \(s_2\) yields 64.3% easy and 25.5% hard, the largest hard-set drop among the ablations. Removing intent match \(s_3\) yields 56.1% easy and 30.1% hard, which hurts easy questions most dramatically. The paper interprets these results as evidence that trajectory-level retrieval, toolchain overlap, and intent alignment each contribute materially to accuracy [2508.15214].

The number of retrieved demonstrations also matters. For full SEER, top-\(k=0\) yields 37.63% easy and 19.04% hard; \(k=2\) yields 58.88% and 27.26%; \(k=4\) gives the best overall reported balance at 67.88% and 31.51%; \(k=6\) drops to 65.88% and 27.12%; and \(k=8\) gives 67.13% and 30.14%. The overall pattern is that a moderate number of demonstrations helps most, whereas too many examples overwhelm the model and degrade performance. This supports the method’s argument for selective, fine-grained retrieval rather than simply increasing context size [2508.15214].

## 6. Practical significance, limitations, and acronym disambiguation

Practically, SEER is described as useful in LLM systems that must repeatedly solve related tool-using tasks in deployment and can observe some notion of success afterward. Examples given in the paper include customer support agents, data-querying assistants, booking agents, and enterprise workflows with repeated tool patterns. Its appeal is that it can improve over time without retraining the base model, and it appears particularly beneficial in settings with complex tool chains or weaker base models, as reflected in the large gains on \(\tau\)-bench and on hard ToolQA subsets [2508.15214].

The method also carries clear operational trade-offs. It introduces retrieval overhead at every interaction step because the current history must be scored against the experience pool. Its effectiveness depends on having prior successful trajectories; early in deployment, performance may be lower until the pool grows. Memory management and retrieval efficiency can become concerns as the pool expands, yet the paper specifies only a maximum size of 1000 and does not detail pruning or indexing strategies. The method further assumes access either to a reference answer or to a sufficiently reliable evaluator signal for determining success. In ToolQA, the evaluator compares the output to the known answer; in more open-ended real environments, this assumption may be harder to satisfy. The limitations section also notes that the diversity of memory is bounded by the underlying LLM’s capabilities and that fixed retrieval weights may be suboptimal across heterogeneous tasks [2508.15214].

The acronym “SEER” is overloaded in adjacent literatures and requires disambiguation. In "Self-Guided Function Calling in Large Language Models via Stepwise Experience Recall," SEER refers specifically to the stepwise retrieval-and-memory framework described above [2508.15214]. By contrast, "SEER: Enhancing Chain-of-Thought Code Generation through Self-Exploring Deep Reasoning" expands SEER as “SElf-Exploring deep Reasoning” and concerns code generation with diverse reasoning path exploration, reasoning quality-aware model training, and adaptive CoT reasoning rather than experience recall over prior tool-use trajectories [2510.17130]. Preference-based reinforcement learning work also uses SEER for a different method built around label smoothing and replay-aligned conservative value estimation [2405.18688]. This acronym overlap can obscure the specific contribution of Stepwise Experience Recall, whose defining features are stepwise trajectory retrieval, intent-and-toolchain-aware relevance scoring, and continual online accumulation of successful tool-use experiences.

Source: https://www.emergentmind.com/topics/stepwise-experience-recall-seer