MemoryArena: Multi-Session Memory Benchmark

• MemoryArena is a benchmark for persistent memory that integrates memorization and action across interdependent multi-session tasks.
• It evaluates agent performance using metrics such as task progress, success rate, and latency in environments like bundled shopping and group travel planning.
• The benchmark highlights limitations in current memory systems and guides advances in execution state management for long-horizon tasks.

MemoryArena is a benchmark for persistent, functionally useful memory in multi-session agents. It was introduced as a unified evaluation gym for benchmarking agent memory in interdependent multi-session agentic tasks, with the explicit aim of coupling memorization and action inside a Memory-Agent-Environment loop rather than evaluating them in isolation. The benchmark’s core premise is that agents acquire information while interacting with an environment, distill useful information into memory, and subsequently retrieve that memory to guide later subtasks; later sessions are therefore underspecified unless earlier experience has been stored and reused (He et al., 18 Feb 2026).

1. Conceptual scope and benchmark identity

MemoryArena is defined around a gap in prior evaluation practice. One class of benchmarks tests memorization by asking models to recall past conversations or text, but does not capture how memory guides future decisions. Another class tests acting in dynamic environments, but typically within a single session, so persistent memory is not causally necessary. MemoryArena is designed to unify these settings by making earlier actions and feedback materially relevant to later subtasks, across session boundaries (He et al., 18 Feb 2026).

This design makes MemoryArena a benchmark rather than merely a dataset in the passive-corpus sense. The benchmark is built from human-crafted multi-session tasks with explicitly interdependent subtasks, and it is organized so that memory is part of the control loop rather than a post hoc retrieval aid. Later work characterizes it specifically as a cross-session-dependency benchmark, where answering a question can require combining information from two or more earlier sessions, thereby directly testing whether a memory system can bridge temporal gaps rather than only retrieve isolated facts (Bering, 26 Apr 2026).

The benchmark spans four task families: bundled web shopping, preference-constrained group travel planning, progressive information search, and sequential formal reasoning over math and physics. In all four, the intended failure mode is not simple forgetting of a fact in a long prompt, but failure to preserve and reactivate prior state that constrains later action (He et al., 18 Feb 2026).

2. Formal interaction model

MemoryArena formalizes both single-session and multi-session interaction. In the single-session setting, for a task $s_i$ with $T_i$ steps, the agent chooses action $a_{i,t}$ conditioned on the task instruction and session history: $$a_{i,t} \sim \mathcal{T}_A(\cdot \mid S_i, O_{i,1:t-1}, a_{i,1:t-1}), \quad O_{i,t} \in \mathcal{O}.$$ In this regime, the in-context trace can still function as working memory (He et al., 18 Feb 2026).

The benchmark’s distinctive setting is multi-session interaction. A task is split into an ordered sequence of subtasks,

$$S = \{s_i\}_{i=1}^{m}, \quad s_1 \rightarrow s_2 \rightarrow \cdots \rightarrow s_m,$$

and earlier traces are no longer directly available in later sessions unless they have been explicitly written into a persistent memory system $M$. At step $t$ of subtask $s_i$, the agent first retrieves task-relevant information,

$$m_{i,t} = \text{RETRIEVE}(M, S_i, a_{i,1:t-1}, O_{i,1:t-1}),$$

then conditions action selection on both current-session history and retrieved memory,

$$a_{i,t} \sim \mathcal{T}_A(\cdot \mid S_i, O_{i,1:t-1}, a_{i,1:t-1}, m_{i,t}),$$

and finally updates memory at session end using the full interaction trace,

$T_i$0

This retrieve-act-write structure is the benchmark’s formal Memory-Agent-Environment loop (He et al., 18 Feb 2026).

Subsequent work restates the same setting in MDP terms, writing the environment dynamics as

$T_i$1

with interaction history

$T_i$2

and emphasizing that long-horizon success depends on maintaining cumulative execution state rather than retrieving semantically similar snippets (Chen et al., 4 Jun 2026). This suggests that MemoryArena is compatible with both memory-as-retrieval and memory-as-state-management formulations.

3. Task families and construction logic

The benchmark’s four environments are constructed so that interdependence is explicit and evaluable rather than incidental.

Task family	Dependency structure	Reported scale
Bundled web shopping	Later purchases must be compatible with earlier ones and satisfy a preference criterion	150 bundled shopping tasks
Preference-constrained group travel planning	New travelers join one by one and impose join or relation constraints	270 group travel instances
Progressive information search	A complex question is decomposed into causally ordered subqueries	256 tasks
Sequential formal reasoning	Intermediate statements, lemmas, propositions, and final conclusions are distributed across sessions	40 math problems and 20 physics problems

The bundled web shopping environment extends WebShop and focuses on five large product domains: electronics, home decor, baking, beauty/personal care, and grocery. Its construction pipeline groups products using the category hierarchy, extracts key product attributes, builds compatibility maps, generates chains of compatible products across sessions, adds incompatible distractors, and has human annotators verify compatibility chains. Each session asks the agent to choose a product that is both compatible with prior purchases and optimal under a preference criterion such as highest rating, highest price, or lowest price (He et al., 18 Feb 2026).

The preference-constrained group travel planning environment is based on TravelPlanner. One traveler’s plan is already finalized, then additional travelers join one by one. Each new traveler can either join some activity already chosen by a prior traveler or specify a relation constraint such as “my hotel rating should be two levels higher than Rebecca’s.” The benchmark constructs 45 base single-traveler instances and augments each with 5 to 8 additional travelers, yielding 270 group travel instances; dependency chains can reach depth four (He et al., 18 Feb 2026).

The progressive information search environment is built from BrowseComp-Plus. The authors filter out items solvable in a single interaction, then manually decompose the remaining instances into causally ordered subqueries, rejecting decompositions in which a later subquery would be required to solve an earlier one. The result is 256 tasks designed to test progressive evidence accumulation across sessions (He et al., 18 Feb 2026).

The sequential formal reasoning environment uses expert-curated derivation chains from real research papers. Senior PhD-level experts select papers with long, structured arguments and decompose central claims into sequences of intermediate statements, lemmas, propositions, and final conclusions. The final set contains 40 math problems and 20 physics problems (He et al., 18 Feb 2026).

4. Evaluation protocol and metrics

MemoryArena evaluates whether agents can acquire task-relevant knowledge during one session, distill it into memory, retrieve it later, and use it to solve later subtasks correctly. The benchmark therefore measures functional reuse of remembered state, not merely recall fidelity (He et al., 18 Feb 2026).

The principal aggregate metric is Task Progress Score. For task $T_i$3 with ordered subtasks, if $T_i$4 subtasks are solved, then

$T_i$5

and the benchmark average is

$T_i$6

This gives partial credit when only a prefix of the interdependent chain is completed (He et al., 18 Feb 2026).

Task Success Rate measures the fraction of tasks fully solved. For bundled web shopping and group travel planning, success means that the final bundle or plan satisfies all constraints. For progressive web search and formal reasoning, success is determined by the correctness of the final subtask or final answer. Because group travel planning is particularly difficult, the benchmark also reports a soft Progress Score, $T_i$7, that gives partial credit according to how many constraints a subtask satisfies. In addition, $T_i$8 measures success at subtask depth $T_i$9, making depth-wise degradation explicit (He et al., 18 Feb 2026).

The benchmark compares several system families. The original study groups them into long-context agents, external memory agents such as MemGPT, Mem0, Mem0-g, and ReasoningBank, and RAG-based systems such as BM25, text-embedding retrieval, MemoRAG, and GraphRAG. It also characterizes memory systems by design complexity: 0D for raw history or flat retrieval without abstraction, 1D for flat but distilled or consolidated memory, and 2D for structured memory with graph- or tree-style organization (He et al., 18 Feb 2026).

5. Empirical findings in the original benchmark

The central empirical result is that MemoryArena exposes a substantial gap between strong performance on existing long-context memory benchmarks and actual competence in multi-session agentic settings. Systems with near-saturated performance on benchmarks such as LoCoMo still perform poorly in MemoryArena, indicating that static recall does not transfer directly to interactive, interdependent task execution (He et al., 18 Feb 2026).

The original study reports that all evaluated methods perform poorly overall. Group Travel Planning is the hardest environment and yields near-zero $a_{i,t}$0 and $a_{i,t}$1 for all systems. This environment requires the agent to remember exact prior choices while simultaneously propagating multiple linked constraints across itinerary slots and travelers, and the result is used as evidence that current memory systems do not yet handle long-horizon constraint tracking reliably (He et al., 18 Feb 2026).

A second finding is that external memory and RAG are not universally better than long context. The benchmark attributes this to two mismatches: a representation mismatch, because retrieved memory may be compressed, segmented, or reordered relative to the raw sequential traces over which the agent reasons, and a training mismatch, because the memory module and the acting policy are not jointly optimized, so the agent may not know how to query or use memory effectively. Consequently, there is no reliable “1 + 1 > 2” effect from simply attaching a memory system to a strong long-context model (He et al., 18 Feb 2026).

Where external memory does help is in the most long-horizon settings, especially progressive web search and formal reasoning. In these domains, long contexts become noisy and attention-saturated, while retrieval or memory can surface the relevant prior facts or intermediate derivations. The $a_{i,t}$2 curves show that performance degrades with subtask depth for all methods, with especially strong decay in progressive web search, where traces can exceed 122k tokens. Retrieval-based systems tend to decay more slowly when exact prior information must be reactivated (He et al., 18 Feb 2026).

The benchmark also reports end-to-end subtask completion latency. The general pattern is that long-context agents are fastest, retrieval systems are intermediate, and external memory agents are slowest. At the same time, the study notes that memory architecture complexity does not map monotonically to latency; implementation details and the quality of the agent-memory interaction matter materially (He et al., 18 Feb 2026).

6. Subsequent uses, derived variants, and broader significance

MemoryArena was rapidly adopted as a testbed for diverse theories of agent memory, and later papers use it to probe retrieval quality, execution-state management, decision-aware compression, systems tradeoffs, and preference compliance.

Paper	How MemoryArena is used	Representative reported finding
"ZenBrain: A Neuroscience-Inspired 7-Layer Memory Architecture for Autonomous AI Systems" (Bering, 26 Apr 2026)	Retrieval benchmark for cross-session dependencies	ZenBrain improves over a Flat Store baseline by +19.5% F1 with $a_{i,t}$3
"Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents" (Chen et al., 4 Jun 2026)	Main benchmark for execution-state memory	MAGE improves average task success rate by 7.8–20.4 percentage points and reduces token consumption by 55.1%
"Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory" (Zou et al., 11 May 2026)	Benchmark for decision-centric bounded-memory compression	DeMem reaches Avg SR $a_{i,t}$4, versus RAG $a_{i,t}$5, Long Context $a_{i,t}$6, and Mem0 $a_{i,t}$7
"Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads" (Omri et al., 4 Jun 2026)	Systems benchmark for freshness–latency tradeoffs	Uses the physics split: 20 multi-session tasks
"Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents" (Zhou et al., 11 Jun 2026)	MemoryArena-derived tasks with user/project constraints	In-distribution, TRACE attains 17.3% task pass and 60.5% violation rate

These follow-on studies are notable because they do not treat MemoryArena as a single-purpose benchmark. ZenBrain uses it as a cross-session retrieval benchmark and reports F1, BLEU-1, ROUGE-L, and cosine similarity under a unified retrieval framework. In that table, BM25-only actually has the highest aggregate F1, while ZenBrain’s claim is narrower: routed multi-layer memory outperforms a flat dense store on cross-session composition rather than universally dominating lexical retrieval (Bering, 26 Apr 2026). This nuance is important because MemoryArena can expose differences between retrieval quality, routing quality, and downstream task competence.

MAGE uses MemoryArena to argue that semantic-RAG memory is insufficient for stateful long-horizon tasks. It treats the benchmark as a continuous Memory-Agent-Environment loop for up to hundreds of steps and attributes gains to preserving the active root-to-current execution path, validating summaries, and revising from explicit boundaries rather than retrieving semantically similar fragments (Chen et al., 4 Jun 2026). DeMem, by contrast, uses MemoryArena to support a rate-distortion account of memory, arguing that the relevant compression criterion is whether histories that would induce different good actions remain distinguishable under a fixed budget (Zou et al., 11 May 2026).

System-level work further broadens the benchmark’s significance. A phase-aware characterization study uses MemoryArena’s physics split of 20 multi-session tasks to surface a systems tradeoff that static recall benchmarks do not reveal: if session arrival is faster than memory construction, a system must choose between blocking for freshness and serving stale memory (Omri et al., 4 Jun 2026). TRACE then introduces MemoryArena-derived tasks by adding a user-in-the-loop wrapper with project-level constraints and correction opportunities, shifting the benchmark toward persistent preference compliance; in this setting, the paper’s key claim is that memory alone is insufficient, and runtime enforcement changes the outcome (Zhou et al., 11 Jun 2026).

A recurring misconception is that MemoryArena is simply another long-context recall benchmark. The benchmark was designed precisely to avoid that reduction: actions generate information that must be stored, and later actions require that stored information. A second misconception is that adding external memory should always outperform long context. MemoryArena’s original results, together with later retrieval and systems studies, do not support universal superiority; they instead show a structured trade space involving representation, routing, query volume, construction cost, freshness, and action relevance (He et al., 18 Feb 2026).

Taken together, these results position MemoryArena as a benchmark for persistent, action-relevant state rather than for passive recollection alone. This suggests a broader interpretation of agent memory research: the decisive question is not only whether a system can remember past content, but whether it can preserve exactly the parts of prior interaction that constrain future decisions, across sessions, under bounded context and bounded runtime resources (Zou et al., 11 May 2026).