CoDiMAD: Diffusion-Based Privileged Distillation for Communication-Free Multi-Robot Coordination

Abstract: Decentralized multi-robot coordination under partial observability remains challenging, especially in communication-free settings where agents must act solely from local sensor observations. Privileged policy distillation provides a promising approach by transferring knowledge from a globally informed oracle to sensor-constrained students. However, in multi-agent systems, the same local observation may correspond to multiple global configurations requiring qualitatively different cooperative actions, making the conditional action distribution inherently multi-modal. Standard deterministic distillation collapses these modes to their mean, often yielding invalid or hesitant actions. To address this issue, we propose CoDiMAD, a three-stage framework that trains a privileged oracle with MAPPO, constructs an offline dataset of local-observation-oracle-action pairs, and distills the oracle into decentralized students parameterized as conditional denoising diffusion probabilistic models. By approximating the conditional oracle-action distribution through the diffusion reverse process, CoDiMAD samples decisive actions from coherent coordination modes rather than averaging across them. Theoretical analysis characterizes the mode-averaging failure of deterministic distillation and the distributional recovery property of diffusion-based distillation. Experiments on three cooperative tasks show that CoDiMAD consistently outperforms direct local MARL and deterministic distillation baselines. The source code will be made publicly available upon acceptance.

• The paper demonstrates a diffusion-based distillation framework that transfers a globally informed oracle policy to decentralized student robots, mitigating mode-averaging issues.
• The methodology employs a conditional denoising diffusion model with CNN-GRU encoders and DDIM for fast inference in multi-robot scenarios.
• Experimental results across Coverage, Pursuit-Evasion, and Box Pushing tasks show near-oracle performance with substantial collision reduction and trajectory diversity.

Diffusion-Based Privileged Distillation for Decentralized Multi-Robot Coordination

Motivation and Problem Definition

Communication-free decentralized multi-robot coordination under partial observability faces substantial challenges. Each agent operates solely based on restricted sensor inputs, and the same local percept can be ambiguous with respect to the true global configuration, necessitating qualitatively distinct cooperative actions. Conventional CTDE (Centralized Training Decentralized Execution) approaches struggle to discover optimal strategies in this regime due to slow convergence and local minima. Privileged policy distillation, in which a globally informed oracle’s behavior is transferred to sensor-bound students, has shown promise in single-agent domains. However, in multi-agent settings, this paradigm encounters a fundamental obstacle: the same local observation can correspond to multiple valid oracle actions, inducing an inherently multi-modal conditional action distribution. Standard regression-based behavioral cloning collapses these modes, producing indecisive, invalid or collision-prone actions.

CoDiMAD Framework Overview

CoDiMAD (Cooperative Diffusion-Based Multi-Agent Distillation) addresses the mode-averaging defect in deterministic privilege distillation by training decentralized diffusion policies that sample from the conditional oracle-action distribution. The framework proceeds in three stages:

1. Oracle Training: MAPPO is used to train a privileged oracle policy with access to local and global state encodings. Dual-stream CNN encoders and GRU integration aggregate local and global spatial-temporal information, with a centralized critic for value estimation.
2. Offline Dataset Collection: The converged oracle actor generates high-quality, collision-free trajectories, creating a dataset of local-observation/oracle-action pairs. Only successful, collision-free episodes are retained, ensuring the validity of the distilled distribution.
3. Diffusion-Based Distillation: A decentralized student policy, parameterized as a conditional denoising diffusion probabilistic model (DDPM), is learned. The student encodes local observations via a CNN-GRU and generates actions through iterative denoising, conditioned on observation embeddings. To achieve fast, deployable inference, Denoising Diffusion Implicit Models (DDIM) are adopted, reducing rollout latency by an order of magnitude.

   Figure 1: CoDiMAD’s three-stage distillation pipeline: (i) oracle policy training with privileged global state, (ii) dataset generation, (iii) diffusion-based student distillation conditioned on local observations.

Theoretical Foundations

CoDiMAD formalizes the failure of standard regression-based distillation under partial observability. The key proposition is that, for a fixed local observation, the posterior over global states decomposes into separated classes (configuration modes), each inducing distinct deterministic oracle actions. The resulting conditional oracle-action distribution is a mixture—multi-modal in nature.

Regression-based behavioral cloning (e.g., MSE minimization) yields a deterministic student policy that predicts the conditional mean, which lies exponentially far from any high-density valid action as the modes become well-separated. Thus, agents exhibit hesitant, mode-averaged behavior, significantly increasing collision risk and diminishing coordination quality.

Diffusion models, by contrast, parameterize the full conditional action distribution. The denoising objective trains the model to estimate the conditional score; sampling through the reverse diffusion process retrieves coherent action modes rather than averages. This mode recovery property is essential for robust decentralized multi-agent coordination.

Evaluation Environments

Experiments are conducted across three cooperative multi-robot tasks, instantiated in a continuous $200\times200$ arena with randomized obstacles and $N=3$ agents:

• Coverage: Agents explore the arena to maximize grid coverage, with collision penalties and reward bonuses for completion.
• Pursuit-Evasion: Pursuers must coordinate to capture a faster evader (APF controller), with rewards for capture and distance shaping.
• Box Pushing: Agents jointly push a heavy box to a goal region; rewards encourage coordinated contact and penalize collisions.

  Figure 2: Task environments—Coverage, Pursuit-Evasion, and Box Pushing—each testing distinct cooperative modalities and collision dynamics.

Experimental Results

Quantitative assessment includes coverage rate, capture rate, success rate, and collision metrics. Across all tasks, CoDiMAD approaches privileged oracle-level performance using only local observations and no communication. On the Coverage task, CoDiMAD achieves $95.7\%$ coverage (oracle: $97.6\%$), with $0.55$ collisions per episode (oracle: $0.42$). Pursuit-Evasion shows a $90.6\%$ capture rate (oracle: $99.1\%$) and $0.80$ collisions. For Box Pushing, CoDiMAD obtains $72.2\%$ success rate, notably higher than direct local MARL and deterministic distillation baselines.

Regression-based distillation (BC-RNN) exhibits drastic performance losses, especially in collision metrics ($N=3$0 collisions vs $N=3$1 in Coverage), directly validating the theoretical analysis. Mode-averaged actions, such as those traversing between valid paths, lead to high collision risk and ineffective coordination.

Temporal history encoding via GRUs provides additional benefits, with ablations (CoDiMAD w/o RNN) showing measurable drops in performance.

Analysis of Multi-Modal Behavior

Direct empirical analysis corroborates the theoretical multimodality induced by partial observability. For fixed local observations, action-space visualizations reveal distinct clusters of oracle actions—each corresponding to a plausible coordination mode. CoDiMAD samples cover both modes, while BC-RNN produces a single, invalid mean action in-between, consistent with mode averaging.

Figure 3: Conditional action distributions under partial observability reveal multi-modal oracle action clusters; diffusion-based student samples cover these modes, deterministic regression collapses to the mean.

Trajectory analysis further confirms CoDiMAD’s stochasticity: from identical initial conditions, CoDiMAD rollouts produce diverse, coherent trajectories reflecting different coordination strategies, while BC-RNN rollouts are deterministic and lack diversity.

Figure 4: Trajectory diversity from identical initial states—CoDiMAD generates multiple plausible coordination behaviors, BC-RNN collapses to a single deterministic trajectory.

Implications and Future Directions

CoDiMAD demonstrates that diffusion-based privileged distillation unlocks robust, communication-free decentralized multi-robot coordination. By modeling the full conditional action distribution, agents are capable of selecting valid, decisive actions consistent with coordination modes implied by ambiguous local observations. This mitigates the mode-averaging defect of deterministic regression, substantially reducing collisions and improving task performance.

Practically, CoDiMAD is deployable in real-time scenarios through DDIM inference, requiring only local sensor data and minimal communication. Theoretically, this framework offers a pathway to scalable decentralized coordination in environments where full communication is infeasible.

Future developments include diffusion-compatible online fine-tuning for improved robustness to distribution shifts, scaling to larger teams, and enriching the model architecture (e.g., attention-based denoisers) for more complex coordination spaces. Physical deployment in marine robotics stands as a critical direction for validating real-world applicability.

Conclusion

CoDiMAD advances the state of decentralized multi-robot coordination under partial observability by privileging the distillation of a globally trained oracle into diffusion-based student policies. These students recover multi-modal action distributions, enabling diversity and coherence in coordination without direct communication. The framework consistently outperforms conventional MARL and deterministic distillation baselines, demonstrating theoretical and empirical superiority in both collision mitigation and task efficacy across varied cooperative domains (2607.09587).