Planner-Conditioned Diffusion for Coordinated Multi-Agent Exploration

Abstract: Coordinated multi-agent exploration requires not only efficient individual coverage but also non-redundant coverage across agents over extended planning horizons. Conventional approaches rely on hand-crafted coordination rules, while end-to-end multi-agent learning methods are difficult to scale and train. Diffusion-based planners such as DARE offer a promising alternative by generating long-horizon trajectories instead of single-step actions, but existing methods are trained on a narrow planner distribution, limiting behavioral diversity and inference-time controllability. We propose a Planner-Conditioned Diffusion Policy (PCDP) for graph-based multi-agent exploration. PCDP is trained on demonstrations from multiple planner styles with planner identity as an explicit conditioning input, enabling a single shared model to learn a multimodal trajectory distribution and generate diverse, controllable trajectory candidates from the same observation. Rather than learning coordination end-to-end, we reuse this multimodal single-agent policy across all agents and introduce coordination through local reranking, in which nearby agents jointly select the trajectory combination with minimal predicted overlap. We evaluate PCDP against classical and diffusion-based baselines on 100 held-out maps in a four-agent simulation setting. PCDP matches the perfect success rate of the diffusion-based baselines while improving mean max-agent travel, total team travel, and agent imbalance. Crucially, reranking alone over a single-planner baseline yields only marginal gains, indicating that planner-conditioned multimodality is the main contributor to improved coordination. Qualitative simulation results and real-robot experiments with two agents further validate that diverse long-horizon trajectory generation produces emergent spatial separation between agents without any explicit repulsion mechanism.

• The paper introduces Planner-Conditioned Diffusion Policy (PCDP), which trains one diffusion model on eight planner behaviors and uses planner-conditioned trajectory candidates for coordinated exploration without end-to-end multi-agent training.
• PCDP with local reranking achieves 100% success, explores 1.0000 of each map, and reduces maximum travel to 649.18, total travel to 2502.85, and imbalance to 48.46 in 100 four-agent simulations.
• The results show that behavioral diversity drives coordination gains: compared with DARE plus reranking, PCDP reduces maximum travel by 1.94%, total travel by 1.58%, and imbalance by 5.66%, though data volume and planner diversity remain confounded.

Overview

This paper introduces the Planner-Conditioned Diffusion Policy (PCDP), a framework for coordinated multi-agent exploration that avoids end-to-end multi-agent training. The core idea is to train a single-agent diffusion policy on demonstrations from multiple exploration planners, conditioning explicitly on planner identity so that one shared model represents a multimodal distribution over long-horizon trajectories. At inference time, this shared policy is reused across all agents, and coordination is achieved through lightweight local reranking: nearby agents jointly select the combination of candidate trajectories that minimizes predicted coverage overlap and travel cost. The work builds directly on DARE [(2608.16229)'s cited baseline, cao2025dare], which applies diffusion-based trajectory generation to single-agent exploration but trains on demonstrations from a single ground-truth planner.

The central empirical claim is that structured behavioral diversity in the candidate set—not the reranking mechanism itself—is what drives coordination gains. The ablation supports this: applying local reranking over a single-planner (DARE) candidate set yields only 0.35% improvement in max-agent travel and 0.64% in total team travel, while the same reranking rule applied to PCDP's planner-conditioned candidates reduces max-agent travel by 1.94%, total team travel by 1.58%, and agent imbalance by 5.66% relative to DARE + local reranking. This is a notable and somewhat counterintuitive finding: a fixed selection mechanism becomes substantially more effective when the candidate distribution is multimodal.

Method

PCDP consists of three components: planner-diverse dataset construction, planner-conditioned diffusion training, and inference-time local reranking.

Planner-diverse demonstrations. The training set comprises $M = 8$ planner classes chosen for complementary behaviors: Greedy Nearest (short-range frontier travel), Information Gain (high-utility regions), four directional-bias planners (spatially distinct motion), TARE cao2021tare, and a privileged Ground-Truth coverage planner. Each sample is $(O_t, \tau_t, c_t)$, where $O_t$ is the observation context over graph observations, $\tau_t$ is the demonstrated trajectory window, and $c_t$ is the planner label. The authors emphasize that planner diversity serves as structured behavioral variation rather than generic data augmentation—though, as discussed below, the experiments do not fully disentangle these effects.

Planner-conditioned diffusion policy. The policy extends DARE's architecture: an attention-based graph encoder maps each node observation (relative position, frontier utility count, binary guidepost signal) into latents stacked into $Z_t$, and the planner identity is embedded as a learnable vector $e_t$ fused with $Z_t$. The denoising network $\epsilon_\theta(x_s, s \mid Z_t, e_t)$ is trained with the standard noise-prediction objective. Planner dropout—randomly replacing the true label with a null condition for a fraction of samples—mirrors classifier-free guidance regularization [ho2021classifierfree] and encourages both planner-aware specialization and planner-agnostic behavior. The policy operates with prediction horizon $T_p = 8$, observation horizon $(O_t, \tau_t, c_t)$0, and action horizon $(O_t, \tau_t, c_t)$1, executing only the first action before replanning (receding-horizon execution).

Local reranking. Each agent samples $(O_t, \tau_t, c_t)$2 candidates per step—one under each planner condition, ensuring equal candidate-set sizes across diffusion baselines. Agents are partitioned into coordination groups via a proximity graph ($(O_t, \tau_t, c_t)$3 m); each group enumerates all joint combinations and selects the one minimizing $(O_t, \tau_t, c_t)$4, with $(O_t, \tau_t, c_t)$5 and $(O_t, \tau_t, c_t)$6. Enumeration costs $(O_t, \tau_t, c_t)$7 evaluations—at most $(O_t, \tau_t, c_t)$8 in the four-agent case. Isolated robots fall back to selecting their best individual candidate. Notably, the reranking objective reduces predicted trajectory overlap but does not enforce inter-robot collision avoidance; collision handling is presumably delegated to the underlying motion execution layer.

Experimental results

Evaluation covers 100 held-out episodes on $(O_t, \tau_t, c_t)$9 grid maps (140 m × 140 m) with four agents, comparing against two classical baselines (Nearest, NBVP) and two diffusion baselines (DARE + independent, DARE + local reranking). Training uses random dungeon environments at $O_t$0 resolution; DARE is trained on 4000 expert trajectories from a single ground-truth planner, while PCDP uses 8000 trajectories (1000 per planner class).

Method	Success	Explored	$O_t$1	$O_t$2	$O_t$3
Nearest	88%	0.9618	982.04 ± 388.60	3637.21 ± 1221.60	158.43 ± 327.27
NBVP	51%	0.9554	1109.87 ± 210.33	3188.08 ± 747.41	685.94 ± 401.68
DARE + independent	100%	1.0000	664.33 ± 81.97	2559.38 ± 318.53	49.78 ± 20.78
DARE + local reranking	100%	1.0000	662.02 ± 88.55	2542.91 ± 346.08	51.37 ± 22.55
PCDP + local reranking	100%	1.0000	649.18 ± 91.57	2502.85 ± 355.91	48.46 ± 20.78

Against the strongest classical baseline (Nearest), PCDP improves success from 88% to 100% and reduces max-agent travel by 33.9%, total team travel by 31.2%, and imbalance by 69.4%. Against diffusion baselines, the margins are smaller but consistent: lower mean travel for all four agents individually, lower max-agent travel in 63/100 episodes versus DARE + local reranking, and best max-agent travel ranking in 45 episodes versus worst in only 23. The authors interpret this as a reduction in poor-coordination outcomes rather than merely an average shift, though the effect sizes against DARE are modest relative to the reported standard deviations.

Qualitatively, different planner conditions applied to the same graph observation produce trajectories diverging in direction and extent, supporting the multimodality claim. Real-robot experiments with two mecanum-wheeled robots (OptiTrack localization, P3GASUS-based centralized planning with decentralized execution) show emergent spatial separation when agents are within coordination range and competent independent exploration when isolated, without explicit repulsion or role assignment.

Limitations and open questions

The paper concedes several limitations that bear directly on the interpretation of its results:

• Confounded comparison with DARE. PCDP uses both a larger demonstration set (8000 vs. 4000 trajectories) and greater planner diversity than DARE. The present experiments therefore do not isolate whether the gains stem from planner conditioning per se or simply from doubled training data volume. A data-matched control would resolve this.
• Exponential reranking cost. Group enumeration scales as $O_t$4, which is tractable for four agents but limits scalability to larger teams or larger candidate sets without approximate selection.
• No collision avoidance in the objective. The reranking score penalizes predicted overlap but does not enforce inter-robot safety, leaving collision handling outside the coordination mechanism.
• Modest effect sizes. The improvements over DARE + local reranking (1.58–5.66%) are small relative to episode-level variance, and the paper does not report statistical significance tests.

Open questions include whether learned or adaptive reranking objectives outperform the fixed weighted-overlap score, and how planner conditioning scales to richer or continuous conditioning spaces beyond a finite set of eight planner identities.

Conclusion

PCDP demonstrates that enriching the diversity of long-horizon trajectory candidates—via planner identity as an explicit conditioning signal—is a practical alternative to end-to-end multi-agent coordination learning. The key evidence is the asymmetry between reranking over single-planner candidates (marginal gains) and reranking over planner-conditioned candidates (consistent reductions in travel cost and imbalance at perfect success rate), validated in simulation and on a physical two-agent platform. The main caveat is that the planner-conditioning benefit remains confounded with training-set size, and the coordination mechanism's exponential scaling leaves applicability to larger teams unresolved.