LEGOMem: Multi-Agent LLM Memory Framework

• LEGOMem is a modular procedural memory framework that converts successful multi-agent execution traces into reusable, role-specific memories for orchestration and execution.
• It decomposes workflows into full-task memories for high-level planning and subtask memories for tool-grounded execution, enhancing task accuracy and efficiency.
• Experimental results on OfficeBench show success rate improvements up to 13 points with reductions in steps and failed actions across complex, multi-application tasks.

LEGOMem is a modular procedural memory framework for multi-agent LLM systems for workflow automation. It is designed for settings in which a central orchestrator plans and delegates, specialized task agents execute subtasks with tools, and an external environment contains applications and APIs. Its central premise is that contemporary multi-agent LLM systems are usually stateless, so they repeatedly solve similar workflows from scratch; LEGOMem addresses this by turning successful past task trajectories into reusable memory units that are allocated according to agent role, with full-task memories supporting orchestration and subtask memories supporting execution (Han et al., 6 Oct 2025).

1. Problem setting and conceptual scope

LEGOMem studies procedural memory in a common multi-agent architecture for workflow automation: a central orchestrator plans and delegates, multiple specialized task agents execute subtasks using tools, and the environment contains the applications and APIs needed to complete office workflows. The paper focuses on long-horizon office workflows, multi-application tasks, and weaker or smaller models, arguing that these regimes are especially sensitive to the absence of persistent procedural memory (Han et al., 6 Oct 2025).

The framework is motivated by the observation that each task is typically solved independently even when previous tasks exhibited similar decomposition patterns, delegation choices, and tool-use routines. Existing memory methods such as Synapse and AWM are described as mainly targeting single-agent settings, which leaves open three multi-agent design questions: where procedural memory should live, how it should be retrieved, and which agents benefit most from it (Han et al., 6 Oct 2025).

LEGOMem therefore functions both as a practical memory layer and as a design lens. The paper’s central question is not only how to store past experience, but how to decompose that experience into role-specific memory units that can improve both high-level planning and low-level execution. This role-aware decomposition is the main conceptual contribution.

2. Role-aware memory decomposition and system model

LEGOMem divides successful trajectories into two kinds of procedural memory. Full-task memory captures the global workflow: task description, high-level plan, subtask sequence, final answer, and brief reflection. This memory is intended primarily for the orchestrator, because it supports task decomposition, next-step planning, delegation, and replanning. Subtask memory captures localized execution behavior: the responsible task agent, subtask description, tool-use actions, concise observations, and lightweight reasoning in a think-action format. This memory is intended primarily for task agents, because it supports accurate tool usage and correct action ordering on similar subtasks (Han et al., 6 Oct 2025).

The formal system comprises an orchestrator $A_{\text{orch}}$, task agents $A=\{A_1,\ldots,A_k\}$, and an environment $\mathcal{E}$. A task $T$ has natural-language description $d$. The orchestrator first produces an initial plan

$\pi_0 = \{s_1, \ldots, s_m\}.$

At orchestration step $t$, the next subtask is

$s_t = \pi_{\text{orch}(\sigma_t)},$

and after a task agent returns summary $r_t$, the orchestrator updates its state by

$\sigma_{t+1} = f(\sigma_t, r_t).$

If execution stalls, the orchestrator may produce a revised plan $A=\{A_1,\ldots,A_k\}$0 (Han et al., 6 Oct 2025).

In the OfficeBench instantiation, the task-agent set includes Word, Excel, Calendar, Email, System, and OCR-PDF agents. This arrangement makes the distinction between global and local procedural memory operational rather than merely conceptual: full-task memories inform orchestration, while subtask memories are routed to the specialized executor most likely to reuse them (Han et al., 6 Oct 2025).

3. Memory construction, storage, and online use

LEGOMem operates in two phases. In the offline phase, successful execution logs are converted into structured JSON memories by an LLM. The output contains "high_level_plan", "subtasks" with agent, description, steps, and observations, "final_answer", and "reflections". The prompt instructs the model to group related actions into subtasks, keep only successful actions, omit malformed or error actions, truncate repetitive actions if too many, and exclude orchestrator coordination steps from subtask memories. From each successful trajectory, LEGOMem derives 1 full-task memory and multiple subtask memories (Han et al., 6 Oct 2025).

These memories are stored in a global memory bank $A=\{A_1,\ldots,A_k\}$1 for full tasks and, optionally, in per-agent subtask banks $A=\{A_1,\ldots,A_k\}$2. Retrieval is implemented with embeddings from OpenAI text-embedding-3-large and a FAISS index. For full-task retrieval, the system embeds the task description $A=\{A_1,\ldots,A_k\}$3; for subtask retrieval, it embeds either the runtime subtask $A=\{A_1,\ldots,A_k\}$4 or a rewritten draft subtask. At inference time, top-5 memories are used for the orchestrator and top-3 memories for each task agent (Han et al., 6 Oct 2025).

Online inference begins by embedding the new task description $A=\{A_1,\ldots,A_k\}$5 and retrieving similar full-task memories from $A=\{A_1,\ldots,A_k\}$6. Full-task memories are passed to the orchestrator, while associated or separately retrieved subtask memories are allocated to relevant task agents. The orchestrator uses the retrieved full-task memories to form an initial plan $A=\{A_1,\ldots,A_k\}$7, chooses the next agent $A=\{A_1,\ldots,A_k\}$8, generates subtask $A=\{A_1,\ldots,A_k\}$9, receives execution summary $\mathcal{E}$0, updates its state, and replans if the workflow stalls. The framework therefore implements a retrieval-augmented procedural memory layer for multi-agent LLM workflows rather than a monolithic archive of past dialogues (Han et al., 6 Oct 2025).

4. Variants and design-space exploration

The basic LEGOMem variant uses a single global bank $\mathcal{E}$1 indexed by task-description embeddings. It retrieves full-task memories once, extracts subtask memories from those retrieved tasks, and statically assigns them to agents. The paper characterizes this version as lightweight, simple, and surprisingly competitive overall, while noting that globally similar tasks can still contain mismatched subtasks, which introduces local noise (Han et al., 6 Oct 2025).

LEGOMem-Dynamic adds per-agent subtask banks $\mathcal{E}$2 and performs just-in-time retrieval after the orchestrator generates subtask $\mathcal{E}$3. The retrieval key is $\mathcal{E}$4, and the intended benefit is less irrelevant subtask memory and better localized execution support. Its explicit cost is repeated embedding and retrieval during execution (Han et al., 6 Oct 2025).

LEGOMem-QueryRewrite shifts fine-grained retrieval earlier. A query rewriting model $\mathcal{E}$5 generates a draft plan

$\mathcal{E}$6

and each rewritten subtask $\mathcal{E}$7 is used to retrieve from the relevant agent memory bank $\mathcal{E}$8. The appendix prompt asks the LLM to break the new task into 3–5 high-level steps and output them in a numbered list inside <start> and <end>. The intended trade-off is to preserve fine-grained subtask retrieval while avoiding repeated runtime retrieval (Han et al., 6 Oct 2025).

These variants instantiate the paper’s broader design-space study. Their differences are not merely engineering alternatives; they operationalize the core questions of memory placement, retrieval granularity, and agent-specific benefit.

5. Experimental setting and empirical results

Experiments are conducted on OfficeBench, a benchmark for office workflow automation with 300 total tasks: 148 for training and memory curation, and 152 for test and evaluation. Tasks are grouped into Level 1 single-application, Level 2 two-application, and Level 3 multi-application workflows. Agents interact with simulated Word, Excel, Calendar, Email, System, and OCR/PDF applications in Docker via APIs. Three multi-agent team configurations are evaluated: an LLM team with GPT-4o orchestrator and GPT-4o task agents, a Hybrid team with GPT-4o orchestrator and GPT-4o-mini task agents, and an SLM team with GPT-4o-mini orchestrator and GPT-4o-mini task agents; OCR uses Phi-3.5-mini as the vision-LLM (Han et al., 6 Oct 2025).

Using the 148 training tasks, the memory curation procedure keeps successful trajectories and yields 93 full-task memories and 250 subtask memories. The primary metric is success rate, evaluated programmatically from the final environment state. The paper compares LEGOMem against No memory, Synapse, and AWM, with the latter two adapted from single-agent methods to the multi-agent setting (Han et al., 6 Oct 2025).

Team	No memory	LEGOMem	LEGOMem-Dynamic	LEGOMem-QueryRewrite
LLM	45.83	58.44	57.12	55.26
Hybrid	35.31	48.03	47.59	50.22
SLM	24.78	38.16	37.72	36.40

Across these settings, the reported absolute improvements over no-memory teams are +12.61 points for the LLM team, +12.72 points for the Hybrid team, and +13.38 points for the SLM team. The paper further notes that Hybrid + LEGOMem-QueryRewrite = 50.22% exceeds memory-less LLM team = 45.83%, and SLM + vanilla LEGOMem = 38.16% exceeds memory-less Hybrid team = 35.31%, indicating that weaker teams benefit disproportionately from procedural memory (Han et al., 6 Oct 2025).

The memory-placement ablation is one of the most consequential results. For the LLM team, No memory = 45.83, Task Agent memory only = 49.78, Orchestrator memory only = 53.29, and Orchestrator + Agent memory = 58.44. For the Hybrid team, No memory = 35.31, Task Agent memory only = 35.31, Orchestrator memory only = 47.59, and Orchestrator + Agent memory = 48.03. The paper concludes that orchestrator memory is essential for high-level planning, task decomposition, subtask orchestration, and replanning (Han et al., 6 Oct 2025).

Process-level metrics show related effects. For Level 3 tasks in the LLM team, average steps drop from 26.5 to 22.2, a 16.2% reduction, and the failed-step rate drops from 0.275 to 0.225. The paper also reports that adding lightweight reasoning traces to memory is not consistently necessary: for the LLM team, LEGOMem without reasoning = 56.36 and LEGOMem with reasoning = 58.44; for the Hybrid team, without reasoning = 49.78 and with reasoning = 48.03. Across variants, the differences are mostly within 2 points, suggesting that the modular memory structure itself carries most of the value (Han et al., 6 Oct 2025).

6. Significance, limitations, and relation to adjacent memory research

LEGOMem’s principal significance lies in its role-aware account of procedural memory. The experiments support a hierarchy of importance: orchestrator memory matters most because decomposition, delegation, and coordination determine whether the right subtasks are attempted in the right order; task-agent memory still matters because it improves execution fidelity; and fine-grained subtask retrieval helps most when task agents are weaker. The ablations also show that task-agent-only memory can underperform, vanilla subtask allocation can miss relevant local traces, additional reasoning in memory is not reliably beneficial, dynamic retrieval has runtime overhead, and query rewriting depends on draft-plan quality (Han et al., 6 Oct 2025).

The framework’s limitations are explicit. The memory bank is built from successful runs only, which constrains coverage and may bias memory toward easier or already-solved workflows. The experiments are only on OfficeBench. The architecture studied is also specific: one orchestrator and specialized task agents. Dynamic retrieval and query rewriting add complexity, while vanilla LEGOMem is often competitive. The paper identifies future directions including continual learning from failed trajectories, scaling to open-ended environments, and scaling to broader tool ecosystems (Han et al., 6 Oct 2025).

Adjacent work clarifies where LEGOMem fits in the broader memory-systems landscape. Neuromem argues that external memory should be evaluated under an interleaved insertion-and-retrieval protocol and decomposes the memory lifecycle into five dimensions, including memory data structure, normalization strategy, consolidation policy, query formulation strategy, and context integration mechanism (Zhang et al., 15 Feb 2026). This suggests that LEGOMem’s gains could be further dissected at operator level, especially if future versions introduce continual insertion, maintenance, or consolidation of procedural memories. MemReader reframes memory extraction as active memory management with an action space of add_memory, buffer_memory, search_memory, and ignore_memory, and reports strong gains on knowledge updating and temporal reasoning (Kang et al., 9 Apr 2026). A plausible implication is that LEGOMem-like systems may benefit from a more selective write policy, particularly if they begin to ingest failed or partial trajectories rather than only successful ones. MemGym, finally, proposes memory-isolated scores that decouple memory performance from reasoning, retrieval, and tool-use ability across tool-use dialogue, deep research, coding, and computer use (Xu et al., 20 May 2026). This suggests a natural next step for LEGOMem evaluation: measuring procedural memory quality independently of the raw capability of the orchestrator and task agents.

Within the literature surveyed here, LEGOMem is most precisely characterized as a retrieval-augmented procedural memory layer for multi-agent workflow automation, centered on modular, role-aware reuse of successful execution traces. Its distinguishing claim is not merely that memory helps, but that procedural memory should be placed where the workflow actually needs it: globally at the orchestrator for decomposition and coordination, and locally at task agents for tool-grounded execution.