TDW-MAT Multi-Agent Transport Benchmark

• TDW-MAT is a simulation platform featuring physics-based, visually guided multi-agent transport tasks in complex 3D environments.
• It leverages advanced agent embodiments with articulated arms and decentralized control, using multi-modal feedback to ensure realistic interactions.
• The platform integrates LLM-driven planning and optimal transport theories to enhance coordination, dynamic task allocation, and planning accuracy.

ThreeDWorld Multi-Agent Transport (TDW-MAT) is a simulation and benchmarking platform for studying physically realistic, visually guided multi-agent cooperative transport tasks in complex 3D environments. TDW-MAT extends the foundational ThreeDWorld (TDW) platform, leveraging fully embodied agents—such as robots with articulated arms—operating under physics-based constraints and decentralized partial observation. The environment, agent APIs, and evaluation metrics are specifically crafted to benchmark the efficiency, coordination, and emergent behaviors of advanced multi-agent systems, especially those enhanced by deep learning, reinforcement learning, optimal transport, and LLMs.

1. Foundational Architecture and Simulation Principles

TDW-MAT is architected atop the TDW platform (Gan et al., 2020), inheriting real-time physics simulation, multi-modal sensory data generation (RGB-D, audio, force/acceleration feedback), and agent customization. Agents interact through physics-driven commands—such as “grasp”, “reach_for”, and “apply_impulse”—with motion and collisions governed by the PhysX engine for rigid bodies and NVIDIA Flex for deformables.

Agent embodiment, as exemplified by the Magnebot (Gan et al., 2021), includes high-DOF articulated arms and mobile bases. Actions must respect kinematics, collision constraints, and force/torque limits. Agents explore multi-room layouts, locate objects, and coordinate transport using containers (e.g., baskets), with perceptual input restricted to egocentric RGB-D views and depth-based semantic segmentation. The API supports multi-agent simultaneous action and feedback in each simulation step.

2. Task Specification and Challenge Protocols

TDW-MAT defines benchmark transport tasks requiring 2–4 agents to collect and deliver objects to goal locations under physical constraints (collision, falling objects, occlusions). Tasks span “Food” and “Stuff” transport, each with unique containers and item types. Agents must:

• Plan multi-step navigation and manipulation.
• Negotiate efficient division of labor.
• Implement collision-avoiding, physics-compliant grasps, lifting, stacking, and container usage.

Reward structures capture both sub-goal satisfaction and operational costs (including communication overhead):

$$R(s, a, s') = -c(a) + \sum_{i} \mathbb{1}(s' \text{ satisfies } g_i) - \mathbb{1}(s \text{ satisfies } g_i)$$

where $c(a)$ quantifies action cost, including communication latency, and $g_i$ are defined sub-goals (e.g., “put 2 plates on the dinner table”).

Transport Rate (TR) and Efficiency Improvement (EI) are principal metrics: $TR$ is the fraction of transported objects within episode budget; $EI$ normalizes improvement over baselines.

3. Multi-Agent Planning and Coordination Algorithms

Hierarchical Planning and RL Baselines

Hierarchical agents (Gan et al., 2021) follow a split approach: high-level planners select sub-goals and manipulation plans; low-level planners (e.g., A* search, inverse kinematics) execute atomistic actions. Pure RL agents, despite end-to-end perception-action mapping, underperform due to sparse rewards, high-dimensional state/action spaces, and insufficient long-horizon credit assignment.

Cooperative Planning via LLMs and Modular Frameworks

Recent advances integrate LLMs for high-level cognition, communication, and emergent coordination. The CoELA agent (Zhang et al., 2023) exemplifies a cognitively modular pipeline:

• Perception (Mask-RCNN).
• Memory (episodic, semantic, procedural).
• Planning (LLM-driven, chain-of-thought prompting over an Action List).
• Communication (natural language, state-sharing with cost per message).
• Execution (navigation/manipulation planners).

GPT-4-driven CoELA agents outperform traditional planners, achieving TR $\approx$ 0.71–0.85, efficient division of labor, and context-aware help-request behaviors. Open-source LLMs (e.g., LLAMA-2, CoLLAMA) reach near-baseline performance after LoRA-based fine-tuning.

Cooperative Plan Optimization (CaPo)

CaPo (Liu et al., 7 Nov 2024) introduces a two-phase protocol for LLM-based agents:

1. Meta-Plan Generation: Agents collectively, via multi-turn LLM-mediated discussion, produce a meta-plan with detailed subtasks, allocations, and contingency strategies. Each agent’s partial observation is synthesized to produce robust joint plans.
2. Progress-Adaptive Execution: As agents report progress (object found, subtask completed), additional dialogue rounds refine the meta-plan, reassess workload allocation, and minimize redundant actions. Communication budgets limit dialogue, balancing coordination with network cost.

Experimental results show CaPo exceeding prior methods (CoELA, ProAgent, RoCo) by TR improvements up to $16.7\%$, attributable to reduced redundant actions and dynamic reallocation during task evolution.

Compositional World Models (COMBO)

COMBO (Zhang et al., 16 Apr 2024) introduces generative planning via factorized world models. Each agent’s action is modelled as a conditional text/image prompt within diffusion-based video models:

$$P_\theta(x|a) \propto P_\theta(x) \prod_{i=1}^n \frac{P_\theta(x|a_i)}{P_\theta(x)}$$

This enables multi-agent planning via compositional score synthesis. Planning employs tree search integrating:

• Vision-LLMs for action proposal, intent prediction, and outcome evaluation.
• Generative global state reconstruction from partial egocentric RGB-D inputs.

COMBO achieves $100\%$ completion rates on TDW-MAT-style tasks, with lower plan step counts than recurrent or MARL baselines.

4. Optimal Transport, Assignment, and Shape Control

Optimal transport theory underpins assignment and distribution strategies in TDW-MAT:

• Multi-marginal matching (Pass, 2012): Surplus functions $$b(x_1, ..., x_m) = \sup_{z \in Z} \sum_i f_i(x_i, z)$$ admit unique Monge solutions under twist and regularity conditions, providing deterministic control mappings over agent teams and contracts.
• Dynamic assignment (Kachar et al., 2019): Formulating transport cost as the effort to move agents to targets under full dynamics constraints yields linear program-based one-shot assignments, avoiding repetitive computation and reducing cost by up to $50\%$ compared to static spatial metrics.
• Shape control (Lin et al., 2022): MASCOT regulates formation, density, and proportional splits via Earth Mover’s Distance (EMD) terms in the cost function:

$$J[\{u^{(i)}\}] = g(\{x^{(i)}(T)\}) + \int_{t}^{T} L(\{x^{(i)}(s)\}, \{u^{(i)}(s)\}) ds$$

where $g$ and $L$ include EMD-based penalties for position-reference deviations, enabling dynamic formation control during transport.

Distributed online optimization (Krishnan et al., 2018) employs primal-dual approaches, local communication, and proximal descent schemes to steer collectives toward target distributions, maintaining scalability and theoretical convergence guarantees.

5. Continuous Transport, Partitioning, and Load-Balancing

TDW-MAT tasks connect to continuous transport formulations (Wang et al., 2015) where replenished object pools require ongoing multi-agent collection and delivery. Algorithms employ hybrid centralized-distributed partitioning (e.g., K-means clustering for initial workload assignment), followed by online partition adaption based on Poisson-modeled replenishment rates. Greedy rate maximization and dynamic repartitioning ensure balanced agent workloads with minimal communication overhead; empirical results show significant performance gains over static or random policies.

6. Communication Topologies and Event-Triggered Control

Efficient communication is essential in TDW-MAT for coordination without bandwidth saturation.

• Event-triggered mechanisms (Shibata et al., 2021): Agents update communication only when state changes merit (binary triggers via continuous observation-driven functions), optimized via Multi-Agent Deep Deterministic Policy Gradient (MADDPG). Markov reward functions penalize both control imprecision and message frequency:

$$r_i = -\|x^* - x\|_2 - \lambda (\|w_i\|_1 + \|z_i\|_1)$$

Empirical studies show near-optimal transport performance with communication costs reduced to negligible levels, robust to agent failures.

7. Validation Protocols and Future Directions

TDW-MAT frameworks employ multi-scale validation:

• Microscopic: “Fundamental diagrams” relating edgewise traffic density and flow (Tranouez et al., 2012).
• Macroscopic: Empirical TR, completion rates, and efficiency over multiple episodes and randomized scene initializations.

Planned advancements include:

• Cognitive module enrichment for nuanced BDI-based agent reasoning.
• Enhanced scenario modeling with multi-site emergencies, partial closures, and spatio-temporal vulnerability propagation analyses.
• Further LLM fine-tuning and world model generalizability for multi-agent cooperation under non-stationary, partially observed, dynamically evolving environments.

Conclusion

TDW-MAT synthesizes advanced multi-agent transport research and physically realistic simulation technologies, setting benchmarks for embodied agent cooperation, optimal task planning, and adaptive communication. With rigorous metrics, modular agent architectures, and algorithms incorporating optimal transport, shape control, and language-driven cognition, TDW-MAT provides a robust platform for empirical paper and algorithmic development in multi-agent embodied AI. The empirical and theoretical findings across cited works enable both practical algorithm deployment and foundational research into coordination, planning, and efficiency in complex environments.