DR. WELL: Dynamic Reasoning and Learning with Symbolic World Model for Embodied LLM-Based Multi-Agent Collaboration

Abstract: Cooperative multi-agent planning requires agents to make joint decisions with partial information and limited communication. Coordination at the trajectory level often fails, as small deviations in timing or movement cascade into conflicts. Symbolic planning mitigates this challenge by raising the level of abstraction and providing a minimal vocabulary of actions that enable synchronization and collective progress. We present DR. WELL, a decentralized neurosymbolic framework for cooperative multi-agent planning. Cooperation unfolds through a two-phase negotiation protocol: agents first propose candidate roles with reasoning and then commit to a joint allocation under consensus and environment constraints. After commitment, each agent independently generates and executes a symbolic plan for its role without revealing detailed trajectories. Plans are grounded in execution outcomes via a shared world model that encodes the current state and is updated as agents act. By reasoning over symbolic plans rather than raw trajectories, DR. WELL avoids brittle step-level alignment and enables higher-level operations that are reusable, synchronizable, and interpretable. Experiments on cooperative block-push tasks show that agents adapt across episodes, with the dynamic world model capturing reusable patterns and improving task completion rates and efficiency. Experiments on cooperative block-push tasks show that our dynamic world model improves task completion and efficiency through negotiation and self-refinement, trading a time overhead for evolving, more efficient collaboration strategies.

• The paper presents DR. WELL, a decentralized neurosymbolic framework that uses LLM negotiations and symbolic planning to enable robust multi-agent collaboration in embodied environments.
• It employs a two-phase negotiation protocol and dynamic symbolic world model to achieve near-100% success in cooperative tasks with efficient role specialization.
• Empirical results demonstrate improved performance and scalability over baseline agents, highlighting adaptive planning, decentralized role assignment, and action reuse.

Dynamic Reasoning and Learning with Symbolic World Model for Embodied LLM-Based Multi-Agent Collaboration

Introduction

This paper introduces DR. WELL, a decentralized neurosymbolic framework designed for multi-agent cooperation in embodied environments, where agents negotiate, specialize roles, and coordinate using LLMs and a dynamic symbolic world model. The framework addresses brittleness inherent in trajectory-level MARL coordination by leveraging symbolic abstractions and structured communication. Agents iteratively sync, negotiate roles, commit to tasks, and then plan and execute discretized action sequences independently, relying on shared symbolic state for interpretability and collective progress. The method is evaluated in a cooperative block-pushing domain with heterogeneous task difficulty requiring varied agent collaboration.

Decentralized Multi-Agent Cooperation via Two-Phase Negotiation

DR. WELL formalizes agent cooperation as a two-phase negotiation protocol. Initially, idle agents synchronously enter the "communication room" to propose candidate tasks, justifying their choices with free-form, LLM-generated rationales grounded in shared environment context. These proposals are coordinated using a round-robin order and validated for resource constraints and coordination requirements.

Upon receiving all proposals, agent commitments are finalized by consensus under quorum constraints. This ensures tasks needing teamwork only proceed if sufficient agents commit. Agents exit negotiation with unique role assignments—no plans or trajectories are shared, maintaining decentralization and privacy.

Figure 1: The two-stage negotiation protocol, showing proposal and commit rounds with exchange of feasibility reasoning and convergence on joint task assignments.

Symbolic Planning and Execution Pipeline

Following commitment, each agent independently generates a symbolic plan grounded in its allocated role. Symbolic planning leverages a compact vocabulary tuned to the environment dynamics, e.g., MoveToBlock, Push, Rendezvous, WaitAgents, YieldFace, each parametrized for task, location, and timing. Agents instantiate LLM-generated plans, then iteratively refine using statistics and prototypes from the shared world model.

The symbolic controller checks preconditions locally and translates symbolic actions into primitive motor commands. Post-conditions and environmental effects are verified precisely by the environment. Unmet quorum or unsatisfied preconditions lead to agent suspension or plan advancement, supporting asynchronous execution and automatic deadlock avoidance. Resynchronization occurs when new agents become idle or existing plans terminate.

Figure 2: Detailed depiction of the post-commitment distributed planning, showing asynchronous negotiation, symbolic plan expansion, execution timelines, and periodic resynchronization.

Dynamic Symbolic World Model (WM)

Central to DR. WELL is its dynamic symbolic world model (WM), which acts as both structured episodic memory and coordination scaffold. WM aggregates multi-layer information: task allocation history, plan prototypes (abstract sequences of symbolic actions), and concrete plan instances (parameterized action chains with outcome and timing metadata). WM thus enables empirical reasoning, action reusability, and performance statistics without explicit trajectory sharing.

Agents query WM during negotiation for past completion rates and optimal team sizes, informing proposal quality and consensus feasibility. During planning, WM provides ranked plan prototypes and instances, guiding agents to empirically robust or efficient strategies. The WM is updated episodically; each agent execution results in additive graph expansion of nodes (episodes, tasks, prototypes, instances) and edges (hierarchical relations).

Figure 3: Layered architecture of the world model, illustrating aggregation of communication/train history, plan prototypes, and concrete instance statistics for decentralized agents.

Experimental Evaluation

Cooperative Push Block Environment

DR. WELL is evaluated in a grid-based Cooperative Push Block environment, where agents must collectively push blocks of varying weights into a goal area. Blocks with weight $w$ require $w$ agents to push simultaneously; agent actions are mapped from the symbolic vocabulary into sequences of primitive moves. Observations are multi-modal: tensor-based spatial features and symbolic status descriptions.

Baseline Comparison

The baseline consists of zero-shot agents using the symbolic state and a fixed policy ("work on nearest block") without negotiation, plan revision, or shared episodic memory. Agents fail to adapt; block completion rates are inflexible and insensitive to episode or environmental change, with heavy blocks often left unfinished and inefficiency due to redundant agent assignment.

Figure 4: Block completion outcomes for baseline agents, showing persistent failure on more difficult blocks and lack of adaptation over episodes.

DR. WELL Performance

DR. WELL agents, leveraging structured negotiation and WM-guided plan refinement, show clear adaptive improvement. After several episodes, agents consistently achieve full block completions, including collaborative blocks, with improved division of labor and less overlap. Completion times (measured in wall-clock seconds and environment steps) display a downward trend, indicating increased execution efficiency and deliberate coordination. Negotiation incurs modest time overhead, but is offset by reduced environment steps.

Agents converge to stable task allocation patterns after a few episodes, as the WM accumulates reusable strategies and empirical statistics. Plan instance and prototype diversity correlate with observed performance gains.

Figure 5: Completion outcome trajectory for DR. WELL agents, showing consistent block delivery and improved adaptation across episodes.

Figure 6: Full-resolution WM graph at Episode 1, showing sparse initial structure and high instance/task incompletion.

Figure 7: Full WM graph at Episode 5, revealing structural growth, richer plan prototypes, and improved task linkage.

Figure 8: WM graph at Episode 10; dense node/edge connectivity demonstrates accumulated episodic memory and robust plan adaptation.

Discussion

The DR. WELL framework demonstrates strong empirical claims: agents consistently achieve nearly 100% block completion rates after initial episodes, outperforming baselines on both single-agent and collaborative instances. Efficiency, as measured by environment steps, improves with WM-guided adaptation, although wall-clock time exhibits a minor increase due to negotiation overhead—an explicit trade-off for coordination quality.

Notably, plan sharing remains decentralized and private; coordination emerges solely from symbolic negotiation and WM-referenced revision, a major departure from joint policy or direct trajectory sharing in classical MARL. The symbolic abstraction enables robust intra-team synchronization (via WaitAgents, Rendezvous primitives) and supports flexible action composition.

Implications and Future Directions

DR. WELL advances neurosymbolic agent design by coupling high-level LLM reasoning with empirical, interpretable symbolic memory. This shifts the locus of planning from fragile, prompt-dependent policy learning to reusable, empirically validated action libraries. Structured negotiation reduces deadlocks and enables scalable division of labor. The framework is environment-agnostic so long as symbolic abstraction is available.

Potential future work includes deployment under partial observability, extension to in-group communication at sub-task levels, support for dynamic interruption, incorporation of probabilistic symbolic outcomes, and adaptation to asynchronous or hierarchical environments. Scaling DR. WELL to larger agent populations or more complex embodied domains may require richer world model representations and optimized negotiation protocols.

Conclusion

DR. WELL provides a decentralized, interpretable, and empirically adaptive multi-agent planning architecture by integrating LLM negotiation, symbolic planning, and a dynamic world model. Agents reliably negotiate, specialize, and execute coordinated plans, achieving high efficiency and success rates without centralized policies or step-level synchronization. The evolving WM structures support continuous improvement and strategy reuse, marking a robust approach for generalizable embodied multi-agent cooperation with LLMs.