---
title: Experience Graph Overview
url: https://www.emergentmind.com/topics/experience-graph
type: topic
---

# Experience Graph Overview

Searching arXiv for recent papers on “experience graph” and closely related graph-structured experience memory/planning formulations.
Experience graph denotes a graph-structured representation of accumulated experience in which reusable structure is preserved explicitly rather than left implicit in flat logs, isolated trajectories, or uniformly sampled replay buffers. Across the literature, the term is used for several related constructions: motion-planning graphs that store prior feasible configurations and transitions for later reuse, replay graphs that encode state dependencies for more effective value backup or sample selection, and agent-memory graphs that record executable artifacts, tool outputs, rewards, sibling comparisons, mutable search statistics, and causal lineage as durable state [2606.29823]. The literature therefore suggests that “experience graph” is not a single canonical object, but a family of graph-based experience representations specialized to different workloads.

## 1. Conceptual scope and recurrent structure

A common feature of experience-graph formulations is that experience is treated as structured data with explicit nodes, edges, and update rules. Nodes may denote robot configurations, observed states, replay candidates, queries, tool invocations, reasoning traces, answers, skills, or entire experiences. Edges may denote feasible motion transitions, temporal transitions, semantic similarity, causal lineage, tool-to-tool dependency, or composition contracts. What is preserved is not only what happened, but also how one episode relates to others and which substructures are reusable.

A second recurrent feature is that graph structure is coupled to a task-specific control operation. In motion planning, this operation is shortest-path retrieval with lazy validation or repair. In reinforcement learning, it is reverse breadth-first search or prioritized replay over dependency structure. In recent LLM-agent systems, it is graph retrieval, diffusion, applicability-aware traversal, or verifier-backed promotion of reusable skills. The graph is therefore not merely archival; it is an operational substrate for search, replay, and adaptation.

| Domain | Typical graph contents | Representative papers |
|---|---|---|
| Motion planning | Configurations, feasible transitions, reused path segments | [1410.1950], [2109.10209] |
| RL and continual learning | States, transitions, replay nodes, feature/topology scores | [2203.15845], [2003.09908], [2407.19429] |
| Agent memory and tool use | Artifacts, rewards, tool dependencies, state summaries, skills | [2606.29823], [2605.30712], [2512.07287] |

This breadth matters because claims about experience graphs are often domain-specific. A sparse roadmap spanner for humanoid motion planning and a heterogeneous query-thinking-answer graph for reward prediction solve different problems even though both are graph-structured memories of prior experience.

## 2. Motion-planning lineage

Some of the clearest early experience-graph formulations arose in motion planning. The Thunder framework stores experiences in a single Sparse Roadmap Spanner rather than in individual paths, with the explicit aim of eliminating redundant information, enabling path-segment reuse, and imposing a theoretical limit on graph size [1410.1950]. In that formulation, solution paths are discretized into states and selectively incorporated into SPARS under coverage, connectivity, and quality criteria. Because shared subpaths are stored once, the graph becomes both a memory-saving device and a retrieval structure.

Thunder demonstrated that graph-based storage can change the asymptotics and empirical profile of experience reuse. In environments with variable obstacles and stability constraints, it was reported to be on average an order of magnitude faster than Lightning and planning from scratch, while using 98.8% less memory after 10,000 trials [1410.1950]. The significance of these results is not merely quantitative. They establish a core design principle that reappears in later work: reusable experience is often more naturally represented as a graph of overlapping substructures than as a database of whole episodes.

A later development is the lazy experience graph in LTR*. Here the graph is persistent and incrementally constructed, but edge validity is checked only upon usage, not during initial graph construction [2109.10209]. LTR* grows bidirectional trees for each new task, stores new nodes and edges in a temporary graph, merges them into a persistent graph, and then attempts lazy shortest-path search through the accumulated structure when subsequent tasks arise. Because the workspace changes after each pick-and-place operation, prior paths cannot simply be replayed as fixed assets. The lazy graph instead supports selective revalidation, edge removal when collisions appear, and continued growth in newly free regions. The method is accompanied by proofs of probabilistic completeness and almost-surely asymptotic optimal guarantees [2109.10209].

These motion-planning papers illustrate a durable interpretation of experience graph: a compact reusable connectivity skeleton over a changing search space. The graph stores not only successes, but also enough structure to support repair when previous solutions become invalid.

## 3. Replay, dependency structure, and continual graph learning

In reinforcement learning, experience graphs arise when the dependency structure among updates is made explicit. Topological Experience Replay organizes experience as a directed graph \( \mathcal{G} = (V,E) \) in which vertices are encoded states and each edge denotes an experienced transition \((s,a,r,s')\) [2203.15845]. Instead of sampling transitions uniformly or solely by TD error, TER performs value backups by reverse breadth-first search starting from terminal states. This is motivated by the observation that the Q-value of a state depends on the Q-values of successor states, so backup order matters. On NChain, TER was reported to achieve the optimal policy in 30 value backups, whereas UER, PER, and DisCor took 3–4x more backups and EBU was still 2x slower [2203.15845].

Continual graph learning introduces a related but distinct use of experience graphs. ER-GNN addresses catastrophic forgetting by storing representative nodes from earlier graph tasks and replaying them during learning of later tasks [2003.09908]. Its buffer update is governed by node-selection strategies—mean of feature, coverage maximization, and influence maximization—and its training loss combines current-task data and replayed experiences through a dynamic weight,
\[
\mathcal{L}_{T_i}'=\beta \mathcal{L}_{T_i}(D_i^{\mathrm{tr}})+(1-\beta)\mathcal{L}_{T_i}(B),
\qquad
\beta=\frac{|B|}{|\mathcal{D}_i^{\mathrm{tr}}|+|B|}.
\]
The central idea is that experience replay for graphs must choose representative nodes rather than merely accumulate past batches.

FTF-ER extends this logic by arguing that replay selection should fuse feature informativeness with global topological importance rather than privileging only one of them [2407.19429]. Its node score is
\[
\mathbf{S}^{mix}(v)=\beta \cdot \mathbf{S}^{feature}(v)+(1-\beta)\cdot \mathbf{S}^{topo}(v),
\]
and the topology term is computed using Hodge Potential Score, derived from Hodge decomposition on graphs. HPS is intended to provide a global node ranking and to avoid the memory overhead of neighbor sampling. On OGB-Arxiv in the class-incremental setting, FTF-ER reported a 3.6% improvement in AA and a 7.1% improvement in AF relative to state-of-the-art methods [2407.19429].

Taken together, these papers show two distinct replay roles for experience graphs. One role is to encode transition dependency so that update order becomes structurally informed, as in TER. The other is to inform which graph elements should be retained and replayed under severe memory constraints, as in ER-GNN and FTF-ER.

## 4. External memory for agentic systems

Recent LLM-agent work generalizes the experience-graph concept from transitions and configurations to richer semantic and executable objects. MemReward constructs a heterogeneous graph in which queries, thinking processes, and answers are separate node types connected by similarity and structural edges, and a GNN propagates rewards from labeled to unlabeled rollouts during online optimization [2603.19310]. With only 20% labels, it reported 97.3% of Oracle performance on Qwen2.5-3B and 96.6% on Qwen2.5-1.5B, reaching 99.4% of Oracle at 70% labels [2603.19310]. Here the experience graph acts as a semi-supervised reward-propagation substrate rather than a planner or replay buffer.

Other systems use graph memory to support exploration and multi-turn adaptation. KG-Agent organizes GUI interaction history into a persistent State-Action Knowledge Graph whose nodes are GUI states, whose similarity edges connect functionally analogous but visually distinct states, and whose skill edges represent successful execution of skills between states [2510.15259]. SIT-Graph represents tool-use history as
\[
G=(V,E,D,W,I),
\]
where tool nodes and directed tool-dependency edges are augmented with compact state summaries attached to edges [2512.07287]. The objective in both cases is to reuse partially overlapping experience rather than retrieve whole trajectories as indivisible units.

A more clinically structured variant appears in GSEM, which uses a dual-layer memory graph: an entity layer for internal decision structure and an experience layer for relations among full clinical experiences [2603.22096]. Each experience is represented as
\[
e_i=(c_i,s_i,z_i,Q_i),
\]
with condition, strategy, indication or contraindication, and quality score. Across MedR-Bench and MedAgentsBench, GSEM achieved the highest average accuracy among baselines, reaching 70.90% with DeepSeek-V3.2 and 69.24% with Qwen3.5-35B [2603.22096]. The same design impulse appears in trainable graph memory for LLM agents, where queries, FSM-abstracted transition paths, and meta-cognitions form a multi-layer graph whose weights are optimized by reinforcement learning [2511.07800].

A further step is to convert experience not merely into memories but into reusable audited capabilities. ASG-SI compiles successful trajectory fragments into a directed multigraph of skills, with verifier-backed replay, contract checks, evidence bundles, and append-only audit traces controlling promotion into the graph [2512.23760]. In this formulation, the experience graph is explicitly tied to governance and reproducibility rather than only to retrieval quality.

## 5. Retrieval, evolution, and database semantics

A central technical issue is how an experience graph is queried and updated. ExpGraph provides one explicit answer: trajectories are summarized into “skills” and “failure lessons,” inserted as nodes in a self-evolving graph, connected to semantically similar neighbors, and retrieved through graph diffusion plus utility-aware ranking [2605.30712]. Its retrieval copilot chooses diffusion depth and similarity-versus-utility weighting, while node utilities are updated online from downstream task outcomes. On ExpSuite, ExpGraph improved over the strongest baseline by 12.2% and 4.7% on static tasks with smaller and larger executors, and by 21.4% and 12.7% in agentic environments, while reducing average interaction steps by 12.7% and 21.6% [2605.30712].

GSEM provides a complementary update mechanism in which only the node qualities and edge weights of activated experiences are adjusted after each case, while the content of stored experiences is left unchanged [2603.22096]. This distinction is important: in some systems the graph evolves by adding and pruning memory items, whereas in others it evolves by recalibrating trust and relation strength over a fixed semantic substrate.

The most expansive systems view is given by Trellis, which treats the experience graph as first-class database state rather than as process-local memory [2606.29823]. In that account, frontier selection is a query, cross-session reuse is vector-seeded graph retrieval, training-data extraction is a materialized view, and reconstructing what an agent knew at any past step is a time-travel query. The logical payload includes executable artifacts, tool outputs, objective rewards, sibling comparisons, mutable search statistics, and causal lineage. In KernelEvolve, this design reportedly allowed cross-session reuse to reach a target speedup roughly 10x faster at 52% lower token cost [2606.29823].

These systems collectively show that the experience graph is increasingly treated as a dynamic data plane. Retrieval is hybrid—graph, vector, relational, and temporal. Updating is often online and feedback-driven. In the most developed formulations, persistence, provenance, and audit are not peripheral engineering details but part of the graph’s semantics.

## 6. Misconceptions, limits, and emerging synthesis

A common misconception is that an experience graph is equivalent to any stored history. The recent literature explicitly rejects that equivalence. Trellis contrasts an experience graph with episodic logs by emphasizing queryability, traversal, versioning, mutability of search statistics, and direct extraction of training views [2606.29823]. Likewise, GSEM argues that storing experiences as independent records without explicit relational structure can introduce noisy retrieval, unreliable reuse, and in some cases even hurt performance compared to direct LLM inference [2603.22096].

A second misconception is that graph structure alone guarantees safe reuse. Several formulations retain explicit safeguards. Thunder and LTR* use lazy collision checking and repair because previously useful paths may become invalid in changed environments [1410.1950]; [2109.10209]. ASG-SI requires verifier-backed replay and contract checks before promotion into the skill graph [2512.23760]. KG-Agent falls back on trial-and-error when high-quality skills fail, while continuing to update the SA-KG [2510.15259]. The graph is therefore a memory of possibilities and dependencies, not a substitute for validation.

A third issue concerns granularity. Whole-trajectory retrieval is often too coarse for stateful or long-horizon workloads, but atomizing experience too aggressively can destroy compositional meaning. SIT-Graph addresses this by attaching compact state summaries to tool-transition edges rather than retrieving whole past episodes, while trainable graph memory for LLM agents abstracts trajectories first into FSM paths and then into meta-cognitions [2512.07287]; [2511.07800]. This suggests that successful experience-graph designs operate at an intermediate level of abstraction: structured enough for recombination, but semantically rich enough to remain actionable.

The emerging synthesis is that experience graphs function as reusable substrates for cumulative learning under non-i.i.d., long-horizon, or dynamically changing conditions. Their instantiations differ sharply across motion planning, RL, continual graph learning, clinical reasoning, GUI exploration, and self-improving LLM agents, yet they repeatedly converge on the same architectural claim: experience becomes more useful when its relational structure is retained, updated, and queried as a graph rather than discarded as disposable process state.

Source: https://www.emergentmind.com/topics/experience-graph