Scaling Teams or Scaling Time? Memory Enabled Lifelong Learning in LLM Multi-Agent Systems

Abstract: LLM multi-agent systems can scale along two distinct dimensions: by increasing the number of agents and by improving through accumulated experience over time. Although prior work has studied these dimensions separately, their interaction under realistic cost constraints remains unclear. In this paper, we introduce a conceptual scaling view of multi-agent systems that jointly considers team size and lifelong learning ability, and we study how memory design shares this landscape. To this end, we propose \textbf{LLMA-Mem}, a lifelong memory framework for LLM multi-agent systems under flexible memory topologies. We evaluate LLMA-Mem on \textsc{MultiAgentBench} across coding, research, and database environments. Empirically, LLMA-Mem consistently improves long-horizon performance over baselines while reducing cost. Our analysis further reveals a non-monotonic scaling landscape: larger teams do not always produce better long-term performance, and smaller teams can outperform larger ones when memory better supports the reuse of experience. These findings position memory design as a practical path for scaling multi-agent systems more effectively and more efficiently over time.

• The paper introduces LLMA-Mem, which combines episodic, procedural, and transactive memory across local, shared, and hybrid topologies to convert task experience into reusable knowledge.
• The framework improves average performance across nearly all tested model–environment pairs, reaches cumulative gains of 167.0 in coding and 321.8 in research, and reduces token use by 9.4%–71.7%.
• The study finds that scaling team size and learning over time interact non-monotonically: well-supported smaller teams can outperform larger teams, while local memory and consolidation every five episodes perform best in the reported ablation.

Motivation and problem framing

LLM-based multi-agent systems (MAS) can be scaled along two largely independent axes: horizontally, by increasing the number of collaborating agents, and temporally, by allowing agents to accumulate and reuse experience across a sequence of tasks. Prior work has examined each axis separately—team-size scaling studies report logistic saturation patterns (Qian et al., 2024) and cases where coordination overhead makes larger teams net-negative under fixed budgets (Kim et al., 9 Dec 2025), while lifelong-learning research focuses on memory architectures for individual agents (Zheng et al., 13 Jan 2025). The paper's central contribution is to treat these axes jointly as an interacting scaling space and to argue that memory design is the mechanism that determines the shape of that space. Two research questions structure the work: how memory design can improve lifelong learning in MAS (RQ1), and how lifelong learning interacts with team size (RQ2).

A preliminary study motivates the design. Applying MARBLE (the MultiAgentBench memory framework) and A-Mem (Xu et al., 17 Feb 2025) to sequential coding tasks yields cumulative performance gain (CMA) curves that plateau or drift rather than grow steadily, indicating that neither general-purpose storage nor dynamic memory organization reliably converts raw experience into reusable cross-task knowledge. The authors also note that both baselines fix the memory topology, leaving open how topology affects lifelong learning complexity.

The LLMA-Mem framework

LLMA-Mem decomposes agent memory into three components. Episodic memory stores full task trajectories with context, actions, outcomes, extracted lessons, and links to related procedures. Procedural memory stores generalized strategies abstracted from episodes, each annotated with success/failure counts yielding an empirical reliability estimate $\rho_j = s_j/(s_j+f_j)$. Transactive memory models "who knows what" via per-agent profiles (specializations, task-type proficiency, collaboration history, success rate) and team patterns mapping compositions to suited task types.

The framework supports three topologies: local (private per-agent stores), shared (a single centralized store), and hybrid (local episodic plus shared procedural/transactive memory). The lifecycle proceeds through retrieval—a hierarchical strategy preferring procedures over episodes, scored by standardized semantic relevance plus importance—update of episode records, procedure statistics, and transactive statistics after each task, and periodic consolidation every $N$ episodes, in which semantically clustered successful lessons are abstracted into candidate procedures with redundancy pruning.

Empirical results on MultiAgentBench

Experiments use three collaborative MultiAgentBench environments (coding, research, database; 100 tasks each), four LLM backbones (Claude-Sonnet-4.5, DeepSeek-V3.2, Qwen3-next-80B, Qwen3-32B-Instruct), and three baselines: no-memory, MARBLE, and A-Mem. Performance is summarized by Task Score (TS), Communication Score (CS), their average $S_t$, running-mean AS, its average AAS, and CMA relative to the no-memory baseline.

The headline findings are:

Result	Evidence
Consistent AAS improvement	LLMA-Mem beats no-memory in nearly all model–environment pairs; largest gains: DeepSeek-V3.2/Research (+5.92 AAS), Qwen3-32B/Database (+3.19)
Stable long-horizon accumulation	On DeepSeek-V3.2, final CMA reaches 167.0 (Coding) and 321.8 (Research) while MARBLE and A-Mem drift negative
Token efficiency	9.4%–71.7% reduction in average token usage versus memory baselines

The cost reduction stems primarily from input-token savings: because episodic traces are consolidated into compact procedural memories, retrieval injects compressed strategies rather than replaying long trajectories, avoiding the context inflation that afflicts MARBLE and A-Mem as experience accumulates. This supports the paper's claim that memory consolidation scales time more effectively than naively scaling context.

One notable exception qualifies these results: for Qwen3-next-80B on Research, LLMA-Mem raises TS but lowers CS enough to reduce AAS below baseline, showing that better memory alone does not guarantee improvement when coordination quality degrades. CS improvements are also more mixed than TS improvements overall, suggesting coordination quality is more sensitive to model family and environment than task-solving ability.

Non-monotonic interaction between team size and lifelong learning

Using 16 research tasks supporting up to seven agents, with team sizes {1, 3, 5, 7}, the paper finds that both temporal accumulation (performance rises with task index at all sizes) and spatial scaling generally help—but the joint landscape is non-monotonic. With Qwen3-32B-Instruct, a 3-agent team outperforms a 5-agent team; with Claude-Sonnet-4.5, the 5-agent team surpasses the 7-agent team after roughly the fifth task. Meanwhile token cost grows monotonically and steeply with team size (e.g., Claude-Sonnet-4.5: ~53K tokens/task at size 1 versus ~344K at size 7).

Two implications follow directly. First, smaller teams can outperform larger ones when memory supports stronger accumulation and reuse of experience, so team size should be treated as a variable interacting with learning ability rather than a fixed hyperparameter. Second, when model scale enters the cost accounting, Qwen3-32B-Instruct with a 3-agent team achieves a TS advantage despite far fewer parameters—a local optimum in the cost–performance trade-off among all evaluated configurations. The authors also observe that TS and CS peaks occur at different team sizes for some models, indicating communication overhead becomes limiting as teams grow.

Ablations: topology and consolidation interval

Under a fixed consolidation interval ($N{=}5$) on the coding environment with Qwen3-32B-Instruct, the local topology dominates, achieving the best TS (49.68), CS (46.69), and AAS (49.43) versus shared and hybrid. The authors attribute this to role-specific memory formation: agents in distinct roles (decomposition, implementation, debugging, verification) build procedures tightly coupled to their responsibilities, whereas shared stores permit cross-role retrieval of incompatible knowledge. This is a somewhat counterintuitive result given that sharing is usually presumed to aid collective reuse, and it may not generalize to settings where roles overlap heavily—an assumption the paper does not test explicitly.

Varying the consolidation interval reveals a non-monotonic effect: $N{=}5$ outperforms both aggressive ($N{=}2$) and sparse ($N{=}10, 20$) schedules. Too-frequent consolidation produces unstable, overly specific procedures from insufficient evidence; too-sparse consolidation delays reuse. The strongest configuration overall is local topology with moderate consolidation frequency.

Limitations and open questions

The paper concedes three limitations. Team sizes are capped at seven agents, so behavior at larger collectives—where communication bottlenecks, role specialization, and memory interference intensify—remains uncharacterized. Evaluation covers only three collaborative MultiAgentBench environments, excluding web search, embodied control, and personalization scenarios where memory dynamics may differ substantially. And the study measures downstream task and communication performance without directly assessing memory quality itself—redundancy, staleness, or robustness to retrieval errors—which leaves open whether the observed gains are attributable to memory content or partly to favorable retrieval noise characteristics. Additionally, the topology ablation is confined to one model and one environment, so the superiority of the local topology is established only narrowly.

Conclusion

This paper reframes multi-agent scaling as a two-dimensional space spanned by team size and lifelong learning ability, and positions memory architecture—notably the separation of episodic substrate, distilled procedural knowledge, and transactive team modeling—as the lever governing that space. LLMA-Mem delivers consistent long-horizon gains over existing memory baselines while reducing token consumption by 9.4%–71.7%, and its analysis demonstrates that the team-size/lifelong-learning interaction is non-monotonic, with well-supported small teams capable of beating larger ones at lower cost. The practical takeaway is that investing in memory consolidation can substitute for horizontal scaling, though the generality of this trade-off beyond seven-agent teams and three benchmark domains remains to be established.