--- title: 'MIMIC-D: Diffusion CTDE for Multi-Agent Imitation' url: https://www.emergentmind.com/topics/mimic-d type: topic --- # MIMIC-D: Diffusion CTDE for Multi-Agent Imitation MIMIC-D is a **Centralized Training, Decentralized Execution (CTDE)** method for **multi-modal multi-agent imitation learning** based on **diffusion policies**. Its full name is **Multi-modal Imitation for MultI-agent Coordination with Decentralized Diffusion Policies**. The method targets cooperative settings in which several agents must coordinate under **multiple valid joint strategies**, but must do so at execution time using only **local information**, without a centralized planner or explicit communication. In this formulation, diffusion models are used to represent multi-modal action-trajectory distributions, while CTDE is used to induce coordination structure during training and preserve decentralized operation at test time [2509.14159]. ## 1. Definition and problem setting MIMIC-D addresses the case in which cooperative multi-agent behavior is intrinsically **multi-modal**. For a fixed task situation, several distinct coordinated solutions may all be valid, but successful execution requires the participating agents to commit to a **compatible shared mode**. The paper motivates this with examples such as two agents swapping positions around an obstacle and two robot arms transporting an object around an obstacle, where left-side and right-side solutions may both be valid, but inconsistent commitments create collisions, deadlock, or task failure [2509.14159]. The method is posed in a **decentralized partially observed Markov decision process (dec-POMDP)**, \[ \langle \mathcal S,\mathcal A,\mathcal O, P, Z, N\rangle, \] with \(N\) agents. The joint state space is \[ \mathcal S = \mathcal{S}^1 \times \ldots \times \mathcal{S}^N, \] the joint action space is \[ \mathcal A = \mathcal{A}^1 \times \ldots \times \mathcal{A}^N, \] and the joint observation space is \[ \mathcal O = \mathcal{O}^1 \times \ldots \times \mathcal{O}^N. \] At time \(t\), each agent \(i\) receives a local observation \(o_t^i \in \mathcal O^i\) and acts via a decentralized policy \[ \pi^i(a_t^i \mid o_t^i). \] The joint action is \(a_t=(a_t^1,\dots,a_t^N)\), and the next state satisfies \[ s_{t+1} \sim P(s_t,a_t). \] The supervision is imitation-based rather than reward-based. The dataset \(\mathcal D\) contains \(M\) expert demonstrations over horizon \(T\), each demonstration represented as a set of tuples \[ \{(\xi^i,o^i)\}_{i=1}^N, \] where \(o^i\) is the observation for agent \(i\) and \[ \xi^i = \{a^i_0,\ldots,a^i_{T-1}\} \] is that agent’s action trajectory. This formulation makes the central difficulty explicit: the learned system must reproduce **coordinated expert behavior** under **decentralized execution**, while preserving the **multiple modes** present in the demonstrations [2509.14159]. ## 2. Motivation: multimodality and decentralized coordination The principal motivation for MIMIC-D is that standard imitation-learning methods handle this regime poorly. The paper states that **behavior cloning** tends toward **mode averaging**: if demonstrations contain both “go left” and “go right,” a deterministic regressor may predict an average action corresponding to “go straight,” which is often invalid. It also states that adversarial imitation methods such as **GAIL** can suffer **mode collapse**, reproducing only one expert strategy while ignoring others. In a multi-agent setting, these failures are amplified, because each agent must not only select a plausible mode but must select one that is **consistent with the other agents’ choices** [2509.14159]. The method is also motivated by the limits of centralized execution. Prior diffusion-based multi-agent approaches commonly assume a **centralized planner** or **explicit communication among agents**. MIMIC-D is designed for cases in which that assumption is unavailable or unrealistic, including communication loss, independent robotic operation, or interaction with humans who cannot be expected to participate in an explicit coordination protocol. The method therefore targets **implicit coordination without explicit communication**, with centralized information used only during training [2509.14159]. This suggests a specific division of labor between modeling components. Diffusion is used because it can represent a **full conditional trajectory distribution** rather than a single regression output, and CTDE is used because the coordination structure of demonstrations can be internalized during joint optimization even when the deployed policies remain decentralized. A plausible implication is that MIMIC-D is best understood not as a centralized planner made partially local, but as a family of **agent-specific generative trajectory policies** whose compatibility is learned statistically from coordinated demonstrations. ## 3. Diffusion-policy formulation Each agent in MIMIC-D is modeled by a **conditional diffusion model** over action trajectories. Agent \(i\) has parameters \(\theta^i\), and the full parameter set is \[ \theta = \{\theta^i\}_{i=1}^N. \] At execution time, the deployed system is the collection of decentralized policies \[ \{\pi_{\theta^1}^1, \ldots, \pi_{\theta^N}^N\}, \] where each policy depends only on the agent’s own local observation [2509.14159]. For agent \(i\), expert trajectories conditioned on observation \(o^i\) are assumed drawn from an unknown data distribution \(p_{\text{data}}(\xi^i; o^i)\). Noise is added as \[ \xi^i_{\text{noisy}} = \xi^i + \epsilon, \qquad \epsilon \sim \mathcal N(0,\sigma^2 \mathbf I). \] A denoiser \(D_{\theta^i}(\xi^i;\sigma,o^i)\) is trained with the per-agent objective \[ \mathcal L^i_{\text{diff}}(\theta^i) = \mathbb E_{\substack{\xi^i \sim p_{\text{data}},\;\sigma \sim p_\text{train}(\sigma),\;\epsilon \sim \mathcal N(0,\sigma^2\mathbf I)}} \Big[\big\| D_{\theta^i}(\xi^i+\epsilon;\sigma,o^i) - \xi^i \big\|_2^2 \Big]. \] Here, \(p_\text{train}(\sigma)\) is a log-normal noise-level distribution following Karras et al. Sampling begins from Gaussian noise, \[ \xi^i_0 \sim \mathcal N(0,\sigma_{\text{max}}^2 \mathbf I), \] and iteratively denoises through \[ \sigma_0 = \sigma_{\text{max}} > \sigma_1 > \cdots > \sigma_K = 0, \] yielding \(\xi^i_K\) as the sampled trajectory [2509.14159]. Architecturally, MIMIC-D uses a **transformer-based denoiser** in the style of **DiT (Diffusion Transformer)** rather than a U-Net or CNN. For each agent, the denoiser consumes three inputs: a noisy action trajectory \(\xi^i_k\), the current diffusion noise level \(\sigma_k\), and the observation \(o^i\). The policy is therefore a **trajectory generator** rather than a one-step action regressor. The paper emphasizes that MIMIC-D uses **distinct policies for each agent**, each with its own parameters \(\theta^i\), which permits heterogeneity among agents; this is reflected directly in the hardware experiment involving one xArm7 and one Kinova3 [2509.14159]. ## 4. Centralized training, decentralized execution The CTDE mechanism in MIMIC-D is introduced through **joint training with a shared loss**, while execution remains decentralized. The total objective is \[ \mathcal{L}_{\text{total}}(\theta) = \sum_{i=1}^N \mathcal{L}^i_{\text{diff}}(\theta^i). \] During training, the paper states that the agents’ denoising policies **share a single loss** and have access to **all local observations**. During execution, each policy uses only its own local observation. There is no centralized policy at test time and no explicit joint-action decoder [2509.14159]. The training loop described in the paper is compact. Policy parameters \(\theta=\{\theta^1,\ldots,\theta^N\}\) and a joint optimizer are initialized. At each training step, \(\mathcal L_{\text{total}}\) is set to zero. For each agent \(i\), a batch \((\xi^i,o^i)\sim\mathcal D\) is sampled, noise levels \(\sigma\sim p_{\text{train}}(\sigma)\) are drawn, noisy trajectories are created, denoised trajectories \(\hat{\xi}^i\) are predicted by \(D_{\theta^i}\), the corresponding \(\mathcal L^i_{\text{diff}}(\theta^i)\) is computed, and the result is accumulated into \(\mathcal L_{\text{total}}\). All parameters are then updated jointly with one **AdamW** step using \(\nabla_\theta \mathcal L_{\text{total}}\) [2509.14159]. At inference time, MIMIC-D uses **decentralized online execution with receding horizon replanning**. Each agent independently acquires its own observation \(o^i\), samples a trajectory \(\xi^i\) of horizon \(T\), and executes only the first \(h\) actions, where \(h