DecentMem: Decentralized Memory for LLM MAS

• DecentMem is a decentralized memory framework that equips each agent with dual exploitation and exploration pools to overcome inefficiencies in centralized LLM multi-agent systems.
• Its adaptive online learning mechanism dynamically balances exploitation and exploration using stage-wise external feedback to enhance convergence and performance.
• Empirical evaluations reveal up to 23.8% accuracy gain and 49% token reduction over centralized methods across diverse MAS benchmarks.

DecentMem is a decentralized memory framework for LLM multi-agent systems, designed to address inefficiencies, privacy limitations, and loss of specialization associated with traditional centralized memory repositories. By equipping each agent with a private, dual-pool memory comprising exploitation and exploration trajectories, and dynamically balancing usage via stage-wise external feedback, DecentMem achieves provably order-optimal cumulative regret and empirical performance gains across diverse reasoning and coordination tasks (Hao et al., 21 May 2026).

1. Motivation and Problem Context

Conventional multi-agent systems (MAS) built atop LLMs have relied on a monolithic, globally shared memory pool for all agents. This approach exhibits several structural drawbacks:

• Communication and Coordination Overhead: Every agent must access and update shared artifacts at each interaction, incurring linear or worse token I/O costs and increased latency.
• Privacy and Access-Control Risks: Universally accessible memory exposes potential leakage of sensitive or role-specific information.
• Loss of Specialization and Diversity: Agents that continually share the same experience pool tend to converge toward homogenized behavior, undermining the complementary specialization that multi-agent paradigms are designed to leverage.

These limitations motivate a shift to decentralized agent-specific memory architectures for collaborative and evolving MAS.

2. DecentMem Architecture: Dual-Pool, Agent-Private Memory

DecentMem endows each agent $a_m$ with two private memory pools:

• Exploitation Pool ($\mathcal{M}_{m,\mathrm{E}}$): A cache of consolidated, high-utility memory elements, each being a tuple $z=(\xi,\,r^\star)$ where $\xi$ represents a context prototype, and $r^\star$ encodes a past trajectory and self-commentary. Retrieval uses embedding-based similarity search with a configurable threshold $\tau$.
• Exploration Pool ($\mathcal{M}_{m,\mathrm{X}}$): A buffer of LLM-generated, as-yet-unconsolidated candidate strategies created for novel or poorly matched contexts.

At each decision stage $t$, the agent probabilistically selects between exploiting prior experience or exploring newly generated solutions, populating its action context with items drawn from the appropriate pool. After each task, the exploration pool is merged into the exploitation pool, recursively enriching experience and accelerating adaptation.

3. Online Pool-Reweighting and Learning Mechanism

The DecentMem framework implements an online learning rule to adaptively modulate each agent's exploitation–exploration trade-off, orchestrated via feedback from an external “LLM-as-judge.” At each stage:

• Let $w_{m,E}(t)$ and $w_{m,X}(t)\equiv 1.0$ denote nonnegative weights for the E- and X-pools.
• The exploitation rate is $\mathcal{M}_{m,\mathrm{E}}$0.
• The pool choice at each turn is sampled from $\mathcal{M}_{m,\mathrm{E}}$1.
• Upon receiving the judge's feedback (improvement indicator $\mathcal{M}_{m,\mathrm{E}}$2), weights are updated:

$\mathcal{M}_{m,\mathrm{E}}$3

where $\mathcal{M}_{m,\mathrm{E}}$4 by default.

This policy rewards both successful exploitation and successful exploration, converging toward the optimal exploitation mixture in expectation.

4. Theoretical Properties and Guarantees

The DecentMem routing protocol possesses two key theoretical properties:

• Global Reachability: Modeling each agent's solution space as a graph $\mathcal{M}_{m,\mathrm{E}}$5, the DecentMem-induced Markov chain remains irreducible and aperiodic, guaranteeing with nonzero probability that any solution state is eventually reachable, thus avoiding permanent entrapment in local minima.
• Order-Optimal Regret: The pool-switching process is interpretable as a two-armed bandit problem with strictly concave expected reward. The cumulative regret satisfies:

$\mathcal{M}_{m,\mathrm{E}}$6

matching the lower bound for stochastic bandit processes up to constants.

5. Algorithmic Realization

The DecentMem workflow is efficiently realizable and incorporates the following steps per agent:

1. At each stage, select E-pool (w.p. $\mathcal{M}_{m,\mathrm{E}}$7) or X-pool (otherwise).
2. If E-pool is selected, retrieve top-$\mathcal{M}_{m,\mathrm{E}}$8 similar elements as action context; if none suffice, fall back to exploration.
3. If X-pool, generate a candidate via LLM, store for subsequent consolidation.
4. After each stage, update the pool weights based on external feedback.
5. After the task, merge X-pool candidates into the E-pool.

The per-stage computational cost is dominated by embedding similarity ($\mathcal{M}_{m,\mathrm{E}}$9), with constant per-step LLM and weight update overhead.

6. Empirical Results and Comparative Analysis

Evaluation on five MAS benchmarks (math: AIME 2024+25, code: MBPP-Plus, QA: BBH, embodied: ALFWorld), across three MAS frameworks (AutoGen, DyLAN, AgentNet) and five LLM backbones (Qwen3-4B/8B/14B, Gemma4-E2B/E4B), demonstrates:

Metric	DecentMem vs. Centralized	DecentMem vs. No memory
Avg. accuracy gain	Up to 23.8%	Up to 52.5%
Token usage reduction	Up to 49%	—

Performance margins are widest under high-coordination-stochasticity frameworks (e.g., AgentNet), indicating that decentralized memory is essential for preserving specialization and role diversity. Fixed-exploitation, pure-exploration, and static $z=(\xi,\,r^\star)$0 schemes underperform DecentMem's online router by 3–7% in accuracy. Experience curve analyses on MBPP-Plus reveal 2–2.5× faster convergence compared to centralized baselines. Cost–performance trade-off analyses show DecentMem occupying the best (high-accuracy, low-token) regime.

7. Limitations and Future Directions

DecentMem's current instantiations have primarily been validated on academic datasets; application to high-stakes domains (e.g., legal, medical) remains untested. Uniform weight-updates across agents have not been meta-tuned to specific domains—dynamic hyperparameter learning could yield further benefits. The absence of selective cross-agent sharing is a potential area for hybrid centralization–decentralization protocols. Prospective extensions include adaptive pool thresholds, hierarchical memory pooling across roles, safe inter-agent graph memory exchange, and refined theoretical regret bounds under agent coordination dependencies.

DecentMem thus provides a rigorously-grounded, empirically-validated framework for decentralized, dual-pool, agent-private memory management in MAS, supporting both theoretical guarantees and substantial practical gains in multi-agent LLM environments (Hao et al., 21 May 2026).