Agent Record-and-Replay (AgentRR)

• Agent Record-and-Replay (AgentRR) is a framework that captures agent interactions and decision processes as durable records rather than transient conversations.
• It transforms detailed execution traces into structured, replayable experiences that improve reproducibility, auditability, and regression control.
• AgentRR systems balance low-level precision with high-level abstraction to ensure safe replay, efficient debugging, and robust release engineering.

Agent Record-and-Replay (AgentRR) designates a class of agent architectures in which execution is treated as a durable artifact rather than an ephemeral conversation. In its explicit formulation, AgentRR “introduces the classical record-and-replay mechanism into AI agent frameworks” by recording an agent’s interaction trace with its environment and internal decision process, summarizing that trace into a structured “experience,” and replaying the experience on later tasks to guide behavior (Feng et al., 23 May 2025). Adjacent work broadens the same idea into execution traces, state-machine programs, typed logs, effect logs, and structured reasoning records, but the shared principle is stable: future agent behavior is constrained, audited, or accelerated through recorded prior behavior instead of being regenerated entirely from scratch (Zhang, 8 Jan 2026, Li, 16 Jun 2026, Nakajima, 21 May 2026).

1. Conceptual lineage and scope

AgentRR inherits from classical record-and-replay systems, which were developed for reverse-execution debugging, intermittent failures, and forensic analysis. The user-space Linux system rr records and replays real-world low-parallelism workloads with low overhead while avoiding kernel modification, pervasive instrumentation, and custom compilers or runtimes (O'Callahan et al., 2017). Concurrency-model-agnostic work extends the same discipline to high-level nondeterministic events across threads and locks, communicating event loops, CSP, and STM, using a uniform trace format and per-entity ordering rather than low-level memory tracing (Aumayr et al., 2021). Actor-language record-and-replay similarly demonstrates deterministic replay debugging of production-oriented actor systems with an average run-time overhead of 10% on Savina benchmarks and about 1.4 MB/s of trace data for Acme-Air (Aumayr et al., 2018).

Within agent systems, the abstraction boundary moves upward. Instead of recording only system calls, thread schedules, or actor mailbox events, AgentRR records agent-environment transitions, tool invocations, internal decision artifacts, and, in some systems, reasoning provenance. The core state-transition view in the explicit AgentRR paper is: $S \xrightarrow{A} S'$ with complete trajectories written as

$$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$

Here, states may denote UI layouts, filesystem or OS state, or abstract workflow states such as “Logged_in” or “Form_page_loaded,” while actions may be GUI operations, API calls, or tool invocations (Feng et al., 23 May 2025).

This suggests that AgentRR is less a single framework than a systems pattern. The pattern reappears in release pipelines that treat traces as the central artifact (Zhang, 8 Jan 2026), in state-machine compilation for computer-use agents (Li, 16 Jun 2026), in event-sourced runtimes where the append-only log is the source of truth (Nakajima, 21 May 2026), and in control planes where every intended action is made visible in a shared log before execution (Balakrishnan et al., 9 Apr 2026).

2. Primary artifacts: traces, records, and trajectories

AgentRR systems differ primarily in what they choose to record and later replay. The explicit AgentRR proposal records environment states, meta-operations, and task metadata, then abstracts them into reusable experiences (Feng et al., 23 May 2025). AgentDevel records, per example, a final output, a structured execution trace, scorer output, critic output, a final pass indicator, a symptom label, and a short symptom description, yielding a quality record

$$r_t(x) = (\hat{y}_t(x), \tau_t(x), g_t(x), \tilde{p}_t(x), p_t(x), \ell_t(x), d_t(x))$$

that is implementation-agnostic because it depends on observable behavior rather than blueprint internals (Zhang, 8 Jan 2026). EyeNavGS shows the same design in an embodied setting by recording head pose and eye gaze for 46 participants across twelve real-world 3DGS scenes and replaying sessions by injecting recorded ViewData into the renderer (Ding et al., 3 Jun 2025).

Work	Primary artifact	Replay/control mode
AgentRR (Feng et al., 23 May 2025)	Multi-level “experience” plus check functions	Experience-guided replay
AgentDevel (Zhang, 8 Jan 2026)	Execution traces, quality records, diagnostic scripts	Regression-aware release gating
PreAct (Li, 16 Jun 2026)	State-machine program $P=(S,T,M,V)$	Direct screen-checked replay
AER (Vispute, 23 Mar 2026)	Step-level reasoning provenance and verdict chain	Narrate, mock replay, live replay
ActiveGraph (Nakajima, 21 May 2026)	Append-only event log and deterministic graph projection	Deterministic replay and cheap forking
LogAct (Balakrishnan et al., 9 Apr 2026)	Typed shared log with Intent/Vote/Commit/Result	Logged pre-execution control

The granularity of recording is decisive. In AgentRR, the record phase captures state and meta-operations such as click(object_id), type(field_id, text), scroll(amount), and invoke_api(endpoint, params) (Feng et al., 23 May 2025). In EyeNavGS, each rendered frame stores per-eye field of view, eye position, head orientation, gaze origin, gaze orientation, and timestamps, producing a passive replay of a trajectory rather than an active policy rerun (Ding et al., 3 Jun 2025). In AgentDevel, traces are richer debugging artifacts, recording actions, tool calls, observations, errors, and final output, then passing these through deterministic scorers and an implementation-blind critic (Zhang, 8 Jan 2026).

A common misconception is that raw checkpoints or observability traces are sufficient on their own. The AER paper argues that reasoning provenance cannot in general be faithfully reconstructed from computational state persistence, because intent multiplicity, observation ambiguity, and inference volatility make post hoc extraction non-identifiable as a normalized, schema-conforming representation (Vispute, 23 Mar 2026). In AgentRR terms, replayable behavior and replayable explanation are related but distinct recording problems.

3. Experience compilation and executable replay

The most explicit AgentRR mechanism is the transformation from trace to reusable “experience.” The original AgentRR paper proposes a multi-level abstraction in which low-level experiences are close to raw scripts or tool sequences, while high-level experiences are abstract workflows or plans expressed in natural language and structured state descriptions (Feng et al., 23 May 2025). The purpose is to balance specificity and generality. Low-level experiences maximize reliability and speed when the environment closely matches the recorded one; high-level experiences sacrifice direct executability in exchange for adaptation across changed layouts or related tasks. Check functions serve as the trust anchor by verifying execution flow integrity, state preconditions, parameter constraints, and safety invariants before each replayed action (Feng et al., 23 May 2025).

PreAct operationalizes this idea for computer-using agents by compiling the first successful run into a small state-machine program and replaying it directly instead of invoking the agent, yielding 8.5–13x faster repeated runs with no per-step language-model calls (Li, 16 Jun 2026). The compiled artifact is

$P = (S, T, M, V),$

where $S$ is a set of states with verification predicates, $T$ is a set of transitions $(s_i, s_j, a)$, $M$ is metadata, and $V$ contains optional value-extraction predicates. Replay is not blind: at each state, PreAct checks that the live screen matches the expected predicate before acting, and hands control back to the underlying computer-using agent as soon as something is off (Li, 16 Jun 2026).

The store discipline is equally important. PreAct does not admit a newly compiled program into its corpus immediately. It performs a store-time verification gate: $$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$0 and stores the program only when

$$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$1

This gate is load-bearing: across mobile, desktop, and web benchmarks it separated repeated runs that improved from ones that degraded as faulty programs accumulated, worth 1.75–2.6 tasks per benchmark (Li, 16 Jun 2026). By contrast, the same paper reports what did not matter in aggregate: prompt wording, runtime guardrails, and whether reuse selection was done by a LLM or a plain embedding retriever (Li, 16 Jun 2026).

This suggests a useful distinction inside AgentRR. Some systems replay by executing a compiled artifact; others replay by guiding a live model through a stored experience. The former tends to maximize latency reduction and determinism; the latter preserves flexibility but reintroduces some online reasoning cost.

4. Release engineering, regression control, and non-regression

AgentDevel reframes self-evolving LLM agents as release engineering. The agent is a shippable artifact defined by a blueprint $$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$2; improvement is externalized into an outer loop that runs the current version, diagnoses failures from traces, synthesizes at most one release candidate, and either promotes or discards it (Zhang, 8 Jan 2026). The resulting version history is a single canonical line

$$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$3

not a population of variants.

Its core loop is: Run → Score → Critic → Diagnose(script) → RC → Gate(P→F / F→P) → Promote (Zhang, 8 Jan 2026). Two elements are central for AgentRR. First, the critic is implementation-blind: it sees the rubric, the execution trace, and optional scorer output, but not the blueprint. It returns a pass/fail judgment, a symptom label, and a short symptom description, thereby generating symptom-level signals rather than causal repair proposals (Zhang, 8 Jan 2026). Second, gating is flip-centered. If $$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$4 and $$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$5 denote pre- and post-change pass indicators, then

$$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$6

$$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$7

Pass→fail flips are treated as regressions; fail→pass flips are treated as fixes. This makes per-example change, rather than aggregate score, the primary release signal (Zhang, 8 Jan 2026).

The empirical results make the release-engineering view concrete. AgentDevel improved SWE-bench Lite from 11% to 22%, SWE-bench Verified from 15% to 30%, WebArena success from 17% to 35.5%, and StableToolBench SoWR from 54% to 73.5% (Zhang, 8 Jan 2026). On WebArena, the full system reported final test 34.2, P→F rate 3.1%, and 0 bad releases; removing the flip gate slightly increased final test score to 35.0 but raised P→F to 14.8% and produced 4 bad releases (Zhang, 8 Jan 2026). In other words, higher average reward without flip accounting can correspond to a less reliable release history.

This is a broader AgentRR lesson. Replay is not only a debugging mechanism; it is also a release-control mechanism. Recorded traces, per-example judgments, and replayable diagnosis scripts define a regression suite against which every new agent version can be audited.

5. Event-sourced provenance and auditable control planes

A stronger interpretation of AgentRR treats the log itself as the agent’s substrate. ActiveGraph states this literally: the append-only event log is the only source of truth, the working graph is a deterministic projection of that log, and behaviors react to graph changes by emitting new events (Nakajima, 21 May 2026). This yields deterministic replay of any run from its log, cheap forking at any event without re-executing the shared prefix, and end-to-end lineage from a high-level goal to the individual model call that produced each artifact (Nakajima, 21 May 2026). In this design, replay is not a secondary debugging mode; it is the normal way state is reconstructed.

LogAct reaches a similar destination through typed shared logs. Each logical agent is decomposed into a Driver, Voters, Decider, and Executor that all play an AgentBus containing typed payloads such as InfIn, InfOut, Intent, Vote, Commit, Abort, Result, Mail, and Policy (Balakrishnan et al., 9 Apr 2026). Agentic actions are visible in the shared log before they are executed and can be stopped by pluggable, decoupled voters. The evaluation reports that this architecture can stop all unwanted actions for a target model on a representative benchmark with just a 3% drop in benign utility (Balakrishnan et al., 9 Apr 2026). This suggests that AgentRR can function as a runtime control plane, not merely as retrospective observability.

The AER model adds structured reasoning provenance to the same log-centric picture. At step $$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$8, reasoning provenance is formalized as

$$S_0 \xrightarrow{A_1} S_1 \xrightarrow{A_2} S_2 \cdots \xrightarrow{A_n} S_n.$$9

where $$r_t(x) = (\hat{y}_t(x), \tau_t(x), g_t(x), \tilde{p}_t(x), p_t(x), \ell_t(x), d_t(x))$$0 is intent, $$r_t(x) = (\hat{y}_t(x), \tau_t(x), g_t(x), \tilde{p}_t(x), p_t(x), \ell_t(x), d_t(x))$$1 is observation, $$r_t(x) = (\hat{y}_t(x), \tau_t(x), g_t(x), \tilde{p}_t(x), p_t(x), \ell_t(x), d_t(x))$$2 is inference, and $$r_t(x) = (\hat{y}_t(x), \tau_t(x), g_t(x), \tilde{p}_t(x), p_t(x), \ell_t(x), d_t(x))$$3 is the motivating plan version. This is contrasted with computational state

$$r_t(x) = (\hat{y}_t(x), \tau_t(x), g_t(x), \tilde{p}_t(x), p_t(x), \ell_t(x), d_t(x))$$4

covering message history, framework-specific channel values, and tool calls (Vispute, 23 Mar 2026). AERs support narrate replay, mock replay, and live replay, and the paper’s preliminary storage comparison estimates about 560 KB per investigation for cumulative checkpoints versus about 25–130 KB for AERs, making AER 4–22× more compact while also being directly queryable for population-level analytics (Vispute, 23 Mar 2026).

Taken together, these systems imply a widening of AgentRR from trajectory playback into event-sourced provenance. The replay target is no longer only “what action happened next,” but also “why this action happened, under what authority, with which evidence, and under which plan revision.”

6. Safety, failure modes, and practical limits

AgentRR raises a distinctive safety problem once checkpoint-restore is combined with irreversible tool use. ACRFence identifies semantic rollback attacks: Action Replay and Authority Resurrection (Zheng et al., 21 Mar 2026). In Action Replay, a restored agent re-synthesizes a logically identical but concretely different request, such as a bank transfer with a new UUID, so downstream duplicate detection fails and the side effect is executed again. The proof-of-concept experiment reports that all 10 checkpoint-restore trials produced duplicate commits, while a no-checkpoint baseline produced none (Zheng et al., 21 Mar 2026). In Authority Resurrection, a single-use approval token reappears in rolled-back agent state and can be reused on a different target; the paper reports that stateless validation allowed all 2/2 token-reuse attempts, whereas stateful validation rejected all of them (Zheng et al., 21 Mar 2026). The proposed mitigation records irreversible tool effects and enforces replay-or-fork semantics at restore time (Zheng et al., 21 Mar 2026).

Practical replay is also fragile at the interface level. The Android record-and-replay study found that 17% of user scenarios, 38% of non-crashing failures, and 44% of crashing bugs could not be reliably recorded and replayed, mainly because of action interval resolution, API incompatibility, and Android tooling limitations (Song et al., 28 Apr 2025). The same study found that replay flakiness is common enough to justify five replay attempts per trace, and that even automated input generation tools run with the same seeds rarely reproduce the same crashes with the same order and timing (Song et al., 28 Apr 2025). These findings generalize directly to AgentRR: if the environment interface is asynchronous, visually dynamic, or only partially observable, traces that appear faithful at one level of abstraction may still be unstable under replay.

A further limitation is partial coverage of nondeterminism. Lightweight record-and-replay for Servo deliberately focuses on message-passing nondeterminism via channels and select, not on network I/O, file I/O, timers, randomness, or explicit lock-based shared memory concurrency (Leija et al., 2019). It can greatly reduce intermittent failures for some tests, but not others, and its authors describe performance overhead as a work in progress even though log sizes are small (Leija et al., 2019). This is a useful caution for AgentRR: replay guarantees are only as strong as the set of nondeterministic channels that are actually captured.

Accordingly, the mature AgentRR designs converge on a small number of operational principles. They record first-class execution artifacts; they distinguish replay from fork; they gate reusable traces or programs before storing them; they keep evaluation blind to current implementation when possible; and they isolate or log side effects that cannot safely be regenerated. What varies is the abstraction level—state trajectories, executable programs, release candidates, reasoning records, or event logs—but the underlying objective remains the same: to make agent behavior reproducible enough to debug, auditable enough to govern, and structured enough to improve without uncontrolled regression.