MIMIC-D: Diffusion CTDE for Multi-Agent Imitation

• MIMIC-D is a CTDE framework employing diffusion policies for decentralized multi-agent imitation learning with multi-modal behaviors.
• It uses joint centralized training and receding-horizon replanning to produce coordinated agent trajectories based solely on local observations.
• Empirical results demonstrate significant reductions in collisions and improved coordination compared to BC and MA-GAIL baselines.

MIMIC-D is a Centralized Training, Decentralized Execution (CTDE) method for multi-modal multi-agent imitation learning based on diffusion policies. Its full name is Multi-modal Imitation for MultI-agent Coordination with Decentralized Diffusion Policies. The method targets cooperative settings in which several agents must coordinate under multiple valid joint strategies, but must do so at execution time using only local information, without a centralized planner or explicit communication. In this formulation, diffusion models are used to represent multi-modal action-trajectory distributions, while CTDE is used to induce coordination structure during training and preserve decentralized operation at test time (Dong et al., 17 Sep 2025).

1. Definition and problem setting

MIMIC-D addresses the case in which cooperative multi-agent behavior is intrinsically multi-modal. For a fixed task situation, several distinct coordinated solutions may all be valid, but successful execution requires the participating agents to commit to a compatible shared mode. The paper motivates this with examples such as two agents swapping positions around an obstacle and two robot arms transporting an object around an obstacle, where left-side and right-side solutions may both be valid, but inconsistent commitments create collisions, deadlock, or task failure (Dong et al., 17 Sep 2025).

The method is posed in a decentralized partially observed Markov decision process (dec-POMDP),

$$\langle \mathcal S,\mathcal A,\mathcal O, P, Z, N\rangle,$$

with $N$ agents. The joint state space is

$$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$

the joint action space is

$$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$

and the joint observation space is

$$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$

At time $t$, each agent $i$ receives a local observation $o_t^i \in \mathcal O^i$ and acts via a decentralized policy

$\pi^i(a_t^i \mid o_t^i).$

The joint action is $a_t=(a_t^1,\dots,a_t^N)$, and the next state satisfies

$N$0

The supervision is imitation-based rather than reward-based. The dataset $N$1 contains $N$2 expert demonstrations over horizon $N$3, each demonstration represented as a set of tuples

$N$4

where $N$5 is the observation for agent $N$6 and

$N$7

is that agent’s action trajectory. This formulation makes the central difficulty explicit: the learned system must reproduce coordinated expert behavior under decentralized execution, while preserving the multiple modes present in the demonstrations (Dong et al., 17 Sep 2025).

2. Motivation: multimodality and decentralized coordination

The principal motivation for MIMIC-D is that standard imitation-learning methods handle this regime poorly. The paper states that behavior cloning tends toward mode averaging: if demonstrations contain both “go left” and “go right,” a deterministic regressor may predict an average action corresponding to “go straight,” which is often invalid. It also states that adversarial imitation methods such as GAIL can suffer mode collapse, reproducing only one expert strategy while ignoring others. In a multi-agent setting, these failures are amplified, because each agent must not only select a plausible mode but must select one that is consistent with the other agents’ choices (Dong et al., 17 Sep 2025).

The method is also motivated by the limits of centralized execution. Prior diffusion-based multi-agent approaches commonly assume a centralized planner or explicit communication among agents. MIMIC-D is designed for cases in which that assumption is unavailable or unrealistic, including communication loss, independent robotic operation, or interaction with humans who cannot be expected to participate in an explicit coordination protocol. The method therefore targets implicit coordination without explicit communication, with centralized information used only during training (Dong et al., 17 Sep 2025).

This suggests a specific division of labor between modeling components. Diffusion is used because it can represent a full conditional trajectory distribution rather than a single regression output, and CTDE is used because the coordination structure of demonstrations can be internalized during joint optimization even when the deployed policies remain decentralized. A plausible implication is that MIMIC-D is best understood not as a centralized planner made partially local, but as a family of agent-specific generative trajectory policies whose compatibility is learned statistically from coordinated demonstrations.

3. Diffusion-policy formulation

Each agent in MIMIC-D is modeled by a conditional diffusion model over action trajectories. Agent $N$8 has parameters $N$9, and the full parameter set is

$$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$0

At execution time, the deployed system is the collection of decentralized policies

$$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$1

where each policy depends only on the agent’s own local observation (Dong et al., 17 Sep 2025).

For agent $$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$2, expert trajectories conditioned on observation $$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$3 are assumed drawn from an unknown data distribution $$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$4. Noise is added as

$$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$5

A denoiser $$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$6 is trained with the per-agent objective

$$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$7

Here, $$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$8 is a log-normal noise-level distribution following Karras et al. Sampling begins from Gaussian noise,

$$\mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N,$$9

and iteratively denoises through

$$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$0

yielding $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$1 as the sampled trajectory (Dong et al., 17 Sep 2025).

Architecturally, MIMIC-D uses a transformer-based denoiser in the style of DiT (Diffusion Transformer) rather than a U-Net or CNN. For each agent, the denoiser consumes three inputs: a noisy action trajectory $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$2, the current diffusion noise level $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$3, and the observation $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$4. The policy is therefore a trajectory generator rather than a one-step action regressor. The paper emphasizes that MIMIC-D uses distinct policies for each agent, each with its own parameters $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$5, which permits heterogeneity among agents; this is reflected directly in the hardware experiment involving one xArm7 and one Kinova3 (Dong et al., 17 Sep 2025).

4. Centralized training, decentralized execution

The CTDE mechanism in MIMIC-D is introduced through joint training with a shared loss, while execution remains decentralized. The total objective is

$$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$6

During training, the paper states that the agents’ denoising policies share a single loss and have access to all local observations. During execution, each policy uses only its own local observation. There is no centralized policy at test time and no explicit joint-action decoder (Dong et al., 17 Sep 2025).

The training loop described in the paper is compact. Policy parameters $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$7 and a joint optimizer are initialized. At each training step, $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$8 is set to zero. For each agent $$\mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N,$$9, a batch $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$0 is sampled, noise levels $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$1 are drawn, noisy trajectories are created, denoised trajectories $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$2 are predicted by $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$3, the corresponding $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$4 is computed, and the result is accumulated into $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$5. All parameters are then updated jointly with one AdamW step using $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$6 (Dong et al., 17 Sep 2025).

At inference time, MIMIC-D uses decentralized online execution with receding horizon replanning. Each agent independently acquires its own observation $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$7, samples a trajectory $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$8 of horizon $$\mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N.$$9, and executes only the first $t$0 actions, where $t$1, before replanning. The denoising process uses the probability flow ODE solver from Algorithm 1 of Karras et al. This receding-horizon mechanism is central to how MIMIC-D handles decentralized consistency: even if initial sampled intentions differ across agents, later replanning steps can redirect them toward a common mode (Dong et al., 17 Sep 2025).

This suggests that coordination in MIMIC-D is intentionally implicit rather than explicitly synchronized. The paper does not introduce a shared latent variable, consensus code, or communication protocol. Instead, coordination is expected to emerge from three elements taken together: joint demonstrations, shared centralized optimization, and online replanning.

5. Experimental tasks and empirical performance

The paper evaluates MIMIC-D in simulated navigation, simulated bimanual manipulation, and real hardware bimanual manipulation. The baselines are BC, defined as CTDE behavior cloning with separate policies for each agent and shared loss; MA-GAIL, defined as an adapted multi-agent GAIL with one discriminator and individual generators; and Vanilla CTDE Diffusion, a diffusion baseline in which each policy imitates its agent without accounting for other-agent information during training. The evaluation metrics are collision counts, success rates, and distributional similarity to expert trajectories measured by Wasserstein distance / Earth Mover’s Distance (EMD) using Fréchet distance as the ground metric (Dong et al., 17 Sep 2025).

The main tasks and reported outcomes are summarized below.

Task	Setting	Reported MIMIC-D result
Two-Agent Swap	2D holonomic agents, six valid modes	12 agent-agent collisions, 3 obstacle collisions, 15 total
Three-Agent Road Crossing	3 holonomic agents, timing-based coordination	collision counts: 95, 14, 1, 0 across thresholds 0.74, 0.675, 0.5625, 0.375
Two-Arm Lift (simulation)	Two Kinova3 arms lift and transport a pot	18 successful lifts, 15 overall successes out of 20
Two-arm hardware task	xArm7 + Kinova3 carry a basket	19 successes out of 20 trials

In the Two-Agent Swap task, two holonomic point robots with single-integrator dynamics on a 2D plane must swap positions while avoiding a central obstacle and each other. The paper states that the task has six valid modes: two “trivial” modes where the agents pass on opposite sides of the obstacle, and four “nontrivial” modes where they pass on the same side and one yields. Over 100 sampled trajectories with randomized starts, the reported collision counts are: BC with 18 agent-agent collisions, 98 obstacle collisions, and 98 total; MAGAIL with 97 agent-agent, 99 obstacle, and 99 total; Vanilla CTDE Diffusion with 52 agent-agent, 1 obstacle, and 52 total; and MIMIC-D with 12 agent-agent, 3 obstacle, and 15 total. For EMD to expert trajectories, the reported values are: BC 1.93 and 1.72; MAGAIL 4.67 and 7.99; Vanilla 1.20 and 1.74; MIMIC-D 1.50 and 1.24 (Dong et al., 17 Sep 2025).

In the Three-Agent Road Crossing task, three 2D holonomic agents move toward their goals while avoiding one another. The paper reports collision counts over 100 trials at distance thresholds $t$2, $t$3, $t$4, and $t$5. BC yields 99, 94, 81, and 55; MAGAIL yields 95, 92, 68, and 32; Vanilla CTDE yields 96, 93, 85, and 61; and MIMIC-D yields 95, 14, 1, and 0. For EMD, the reported values are: BC 0.4497, 1.1541, 0.3615; MAGAIL 1.2749, 1.5166, 0.5402; Vanilla 0.4241, 0.5046, 0.3301; and MIMIC-D 0.3259, 0.4394, 0.2845 (Dong et al., 17 Sep 2025).

In the Two-Arm Lift simulation task in Robosuite, two Kinova3 arms must grasp a pot, lift it together, move it around an obstacle, and set it down. Two demonstration modes are collected, passing the obstacle on the left or on the right. Each arm’s observation includes the ego end-effector pose, the other agent’s end-effector pose, and the pot’s initial handle positions, all expressed in the robot’s own local base frame. The action trajectory length is 25, and only the first 10 planned steps are executed before replanning. The paper states that 50 demonstrations per mode are used. Over 20 runs, BC achieves 2 successful lifts and 0 overall successes; MAGAIL achieves 0 lifts and 0 successes; Vanilla achieves 5 lifts and 0 successes; and MIMIC-D achieves 18 lifts and 15 overall successes (Dong et al., 17 Sep 2025).

The hardware experiment uses one xArm7 and one Kinova3 carrying a basket around an obstacle. The basket pose is fixed and removed from the observation vector, and only eight demonstrations per mode are collected, for 16 total. The paper reports 95% success, namely 19 successes out of 20 trials. It also notes that the arms may initially move in opposite directions but, through replanning, eventually reach consensus on a mode. This is one of the clearest empirical demonstrations of MIMIC-D’s decentralized coordination mechanism on heterogeneous real systems (Dong et al., 17 Sep 2025).

6. Interpretation, significance, and limitations

The empirical comparisons function as de facto ablations of the method’s main ingredients. The comparison between diffusion-based methods and BC / MA-GAIL supports the claim that diffusion is needed to model multi-modal demonstrations without averaging or collapse. The comparison between MIMIC-D and Vanilla CTDE Diffusion isolates the role of centralized coordination training: the paper’s interpretation is that diffusion alone can capture obstacle-avoiding or multi-modal single-agent behavior, but joint CTDE training is needed to make these local samples mutually compatible across agents (Dong et al., 17 Sep 2025).

The paper also highlights a qualitative relation to the freezing robot problem, described as a local-minimum or indecision phenomenon in interactive navigation. It suggests that diffusion’s stochastic sampling may help avoid stagnation, because planning begins from random noise and therefore introduces enough variation to break symmetry. This suggests that the generative nature of the policy is not only a device for representing multimodality but may also act as a mechanism for escaping deterministic deadlock in symmetric coordination settings (Dong et al., 17 Sep 2025).

Several assumptions and limitations are explicit. MIMIC-D depends on expert demonstrations that already exhibit coherent coordinated modes. The coordination mechanism remains implicit: there is no explicit shared latent variable, synchronized random seed, or formal probabilistic coupling among agents at inference time. The method assumes sufficiently rich local observations; in the conclusion, the authors explicitly identify perfect state observation as a current assumption they wish to relax in future work by conditioning on camera inputs. They also identify out-of-distribution performance as a future direction. In addition, because MIMIC-D is diffusion-based, inference requires iterative denoising, which is inherently more computationally expensive than a single forward pass, although the paper does not report runtime or latency measurements (Dong et al., 17 Sep 2025).

Within the broader literature, the string “MIMIC-D” can be confused with other uses of “MIMIC,” including dynamic/error-correction MIMIC models in structural equation modeling, where the term refers to Multiple Indicators Multiple Causes rather than multi-agent imitation. In that literature, the closest related term is EMIMIC or an error-correction extension of MIMIC, not the robotic diffusion-policy method discussed here (Srakar et al., 2020). In current robotics usage, however, MIMIC-D denotes the diffusion-based CTDE framework for decentralized multi-agent coordination (Dong et al., 17 Sep 2025).

Taken together, MIMIC-D is significant because it defines a concrete policy class for a specific but important regime: multi-agent coordination with multiple valid joint strategies and no centralized execution channel. Its central contribution is not merely the use of diffusion, nor merely CTDE, but the combination of trajectory-level conditional diffusion policies, joint decentralized-policy optimization, and receding-horizon execution to recover coordinated multi-modal behavior under local-information constraints (Dong et al., 17 Sep 2025).