MUTE: Return-Preserving Communication Unlearning for Efficient Multi-Agent Coordination

Abstract: Inter-agent communication is critical for coordinating Multi-Agent Reinforcement Learning (MARL) agents under partial observability to perform effectively in cooperative games; however, real-world bandwidth constraints demand sparse interactions. Prior approaches primarily address this trade-off by optimizing information-theoretic surrogates. We argue that these statistical proxies are fundamentally misaligned with the true objective: a message can be highly informative yet irrelevant to the joint return of the task. In this work, we propose Message Unlearning for Targeted Efficiency (MUTE), a framework that views communication reduction as a value-guided machine unlearning problem. MUTE rigorously quantifies the Counterfactual Message Value using an attention-based estimator, and systematically unlearns the transmission of low-value messages from a policy trained without any communication constraints. This is achieved through a dual-objective mechanism that enforces communication sparsity while preserving the return of the original joint policy. We derive a theoretical upper bound on the performance gap induced by this sparsification, guaranteeing controlled return degradation. We also empirically evaluate MUTE on various complex multi-agent environments, achieving 80% to 90% bandwidth reduction while maintaining performance comparable to state-of-the-art baselines.

• The paper presents the MUTE framework, which unlearns redundant messages to preserve joint returns in MARL through value-guided, counterfactual estimations.
• It employs an attention-based Message Value Estimator that efficiently identifies low-value messages, enabling drastic bandwidth reduction without performance loss.
• Experimental results across benchmarks confirm that MUTE maintains near-expert performance while achieving significant communication sparsity in complex environments.

MUTE: Return-Preserving Communication Unlearning for Efficient Multi-Agent Coordination

Motivation and Problem Context

Inter-agent communication under partial observability is essential for effective coordination in multi-agent reinforcement learning (MARL). However, the deployment of such systems under real-world bandwidth constraints necessitates minimizing communications without degrading performance. Prevailing approaches primarily adopt information-theoretic surrogates (e.g., mutual information or entropy regularization) to promote communication sparsity. Such statistical proxies are decoupled from ultimate task return and can prioritize signals that alter action distributions without contributing to return, which may even result in negative return impact if the information is misleading. Hence, MARL methods require task-value–aware communication minimization that targets joint return preservation.

The "MUTE" framework proposes a sharp departure from conventional approaches by casting bandwidth reduction as a value-guided unlearning problem. Instead of regularizing communication during policy acquisition, MUTE first trains a fully communicative expert, then progressively removes redundant messages identified to be return-irrelevant via precise counterfactual estimations. This is implemented through a dual-objective regime—enforcing communication sparsity while anchoring the decentralized policy to the full-communication expert—and is equipped with theoretical guarantees on return degradation.

Method: Counterfactual Value-Guided Communication Unlearning

The primary contribution of the paper is the Message Unlearning for Targeted Efficiency (MUTE) framework, delineated in several key phases:

Phase 1: MARL Policy Training with Unconstrained Communication

An initial policy is trained under the Centralized Training with Decentralized Execution (CTDE) paradigm with unrestricted, broadcast message passing. This agent ensemble serves as the "expert" for subsequent communication pruning.

Phase 2: Counterfactual Message Value Estimation (CMV/MVE)

A central challenge is directly quantifying the impact of each message on the joint return. MUTE formalizes the Counterfactual Message Value (CMV), which measures the difference in the global Q-value when a single agent’s message is masked versus transmitted (keeping the sender’s action fixed and resampling the others under the new message set). Due to the computational intractability of exact Shapley-value estimation, MUTE instead uses an attention-based Message Value Estimator (MVE) to efficiently predict these marginal values for all agents in a single forward pass, leveraging the permutation-invariant and context-aware properties of self-attention.

Phase 3: Value-Guided Unlearning with Return Regularization

Having identified messages with low CMV (below a dynamic threshold), MUTE introduces a sparsity-inducing $L_1$ penalty on the corresponding generator outputs, attenuating their influence. Crucially, to guarantee retention of behavior, a behavioral anchoring loss is included that directly regularizes the new decentralized Q-values to remain close to the expert's Q-values. The combined objective is:

$$\mathcal{L} = \mathcal{L}_{\text{sparse}} + \beta \mathcal{L}_{\text{anchor}}$$

where $\mathcal{L}_{\text{sparse}}$ penalizes low-value messages and $\mathcal{L}_{\text{anchor}}$ ensures local policies retain expert skills.

Theoretical Guarantees

The authors provide a non-asymptotic performance bound on the joint return loss as a function of estimation errors, sparsity threshold, and the smoothness of $Q^*_{\text{tot}}$, rigorously limiting return degradation induced by message unlearning. Empirical results verify that practical estimation errors indeed satisfy these constraints.

Experimental Validation

MUTE is comprehensively evaluated on Hallway, Traffic Junction, SMAC, SMAC-Communication, and SMACv2. Benchmarks span grid worlds, traffic control, and high-dimensional micromanagement (including stochastic and zero-shot scenarios).

Communication Efficiency and Performance Preservation

MUTE persistently achieves the lowest communication bandwidth among all communication-enabled protocols while maintaining win rates nearly identical to the expert or best baselines. In Hallway and Hallway-Group, it achieves $99\%$ and $90\%$ bandwidth reduction with full win rate. In SMAC (MMM2), MUTE matches MASIA's $94\%$ win rate while reducing communication by $>95\%$. In high-variance settings (SMAC corridor), MUTE sustains $73\%$ win rate at $$\mathcal{L} = \mathcal{L}_{\text{sparse}} + \beta \mathcal{L}_{\text{anchor}}$$01\% bandwidth, where other methods degrade dramatically.

Figure 1: Comparative communication efficiency and task performance of MUTE and baselines across 10 benchmarks, highlighting minimal communication with near-optimal returns.

Robustness to Message Dropout

The selectivity of MUTE is quantified by measuring win rate under increasing message dropout rates. MUTE consistently outperforms baselines in the area under the win rate-dropout curve (AUC), indicating that reserved messages are highly return-critical—dropping redundant information does not compromise coordination even at high dropout rates.

Figure 2: Test win rate versus message dropout rate, showing MUTE's graceful degradation and robustness relative to alternative protocols.

Architectural and Training Mechanism Ablations

Ablations demonstrate the criticality of MUTE's core mechanisms. Removal of behavioral anchoring or value-guided MVE (by random selection or using non-return-aware penalties) leads to catastrophic performance collapse, underlining the necessity of decoupling policy learning from communication pruning and using explicit value estimation for redundancy detection.

Computational Analysis and Scalability

MUTE’s computational cost, both in wall-clock time and FLOPs, is competitive with state-of-the-art. The MVE's attention architecture ensures linear (not exponential) scaling with agent count. The authors directly investigate the impact of agent number and message dimensions, showing that MUTE remains tractable for dense populations ($$\mathcal{L} = \mathcal{L}_{\text{sparse}} + \beta \mathcal{L}_{\text{anchor}}$$1) and moderate message dimensionality.

Implications

Pragmatically, MUTE provides a path to deploy MARL agents in bandwidth-constrained multi-robot, vehicular, and tactical networks with minimal engineering and no sacrifice of task efficiency. Theoretically, MUTE frames communication reduction as a structured machine unlearning problem with semantically meaningful, return-aware regularization, avoiding the shortcomings of information-theoretic or gating-based paradigms.

Notably, MUTE sometimes marginally exceeds the full-communication baseline—an artifact of overfitting elimination and redundant signal suppression, which may enhance generalization, especially in high-variance domains.

Limitations and Future Directions

MUTE's reliance on CMV as a local (single message at a time) estimator does not address higher-order message interactions. While behavioral anchoring mitigates policy collapse in such cases, direct subset-aware value estimation could yield even tighter return bounds. Extending MUTE to multi-level or hierarchical communication, multi-modal message spaces, or direct minimax settings (non-cooperative MARL) is an important avenue for future work.

Conclusion

MUTE establishes a new state of the art in value-preserving minimal communication for MARL through a principled value-guided unlearning protocol, attention-driven message value estimation, and rigorous return gap guarantees. Empirical and theoretical analyses confirm its efficacy, robustness, and computational viability for scalable multi-agent deployment.

Figure 3: Visualization of the Traffic Junction domain, exemplifying partial observability and critical need for communication.

Figure 4: Training curves comparing win rate and communication rate between MUTE and ablation using state plus messages, supporting message-only MVE choice.

Figure 5: Unlearning phase sensitivity to training budget allocation in SMAC, highlighting non-arbitrary default schedule for communication-performance tradeoff.