A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives (2503.13415v1)

Abstract: With the rapid development of artificial intelligence, intelligent decision-making techniques have gradually surpassed human levels in various human-machine competitions, especially in complex multi-agent cooperative task scenarios. Multi-agent cooperative decision-making involves multiple agents working together to complete established tasks and achieve specific objectives. These techniques are widely applicable in real-world scenarios such as autonomous driving, drone navigation, disaster rescue, and simulated military confrontations. This paper begins with a comprehensive survey of the leading simulation environments and platforms used for multi-agent cooperative decision-making. Specifically, we provide an in-depth analysis for these simulation environments from various perspectives, including task formats, reward allocation, and the underlying technologies employed. Subsequently, we provide a comprehensive overview of the mainstream intelligent decision-making approaches, algorithms and models for multi-agent systems (MAS). Theseapproaches can be broadly categorized into five types: rule-based (primarily fuzzy logic), game theory-based, evolutionary algorithms-based, deep multi-agent reinforcement learning (MARL)-based, and LLMs(LLMs)reasoning-based. Given the significant advantages of MARL andLLMs-baseddecision-making methods over the traditional rule, game theory, and evolutionary algorithms, this paper focuses on these multi-agent methods utilizing MARL and LLMs-based techniques. We provide an in-depth discussion of these approaches, highlighting their methodology taxonomies, advantages, and drawbacks. Further, several prominent research directions in the future and potential challenges of multi-agent cooperative decision-making are also detailed.

• The paper presents a comprehensive review of simulation environments and methodologies for multi-agent cooperative decision-making, outlining both traditional and emerging approaches.
• It systematically evaluates techniques such as game theory, evolutionary algorithms, MARL, and LLM-based frameworks, emphasizing their scalability and coordination efficiencies.
• The survey identifies key challenges including scalability, non-stationarity, and communication constraints while proposing hybrid models to enhance future MACD research.

A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives

Introduction

The research on intelligent decision-making technologies has undergone significant advancements, leading to superior performance over human levels in diverse competitive scenarios, particularly in complex multi-agent cooperative systems. Multi-agent cooperative decision-making (MACD) involves coordination among multiple agents to achieve shared goals, with real-world applications spanning autonomous vehicles, drone navigation, disaster recovery, and military simulations. The paper offers a thorough survey of prominent simulation environments and state-of-the-art algorithms in MACD, highlighting five main approaches: rule-based, game theory-based, evolutionary algorithms, deep multi-agent reinforcement learning (MARL), and reasoning frameworks utilizing LLMs.

Figure 1: An overview of the evolution of scenarios and methods in decision-making from single-agent to multi-agent systems.

Simulation Environments

The survey provides an in-depth review of simulation platforms crucial for developing MACD strategies. Platforms like the Multi-Agent Particle Environment (MPE) and StarCraft Multi-Agent Challenge (SMAC) facilitate testing MARL algorithms, fostering advancements by providing challenging environments that mimic real-world complexity. These platforms allow researchers to evaluate the coordination and communication among agents. The environments are vital for testing the scalability of MACD algorithms, enabling agents to develop and refine strategies that cope with partial observability and dynamic adversaries.

Figure 2: Illustration of our systematic review of multi-agent intelligent decision-making research. Compared to previous reviews, we have incorporated comprehensive introduction and analysis, with each segment corresponding to a specific chapter in the survey.

Novel Approaches in Multi-Agent Systems

The paper categorizes MACD methodologies into several paradigms:

1. Rule-Based Systems: Utilize heuristic-driven decision-making, typically using fuzzy logic to manage uncertainty in agent behavior.
2. Game Theory-Based Methods: These strategies leverage mathematical frameworks to model competitive and cooperative interactions among agents.
3. Evolutionary Algorithms: These methods simulate natural selection processes to evolve agent strategies over generations.
4. MARL: This approach stands out by using centralized training with decentralized execution (CTDE), optimizing policies through shared rewards and experiences across diverse agents.
5. LLMs-Based Frameworks: Emerging methods that apply LLM reasoning to improve coordination by understanding and processing natural language instructions, introducing a novel dimension to decision-making.

   Figure 3: The paradigms visualization of CTDE (left), DTDE (centre), and CTCE (right), consisting of three crucial elements: agent (i.e., algorithm or model), environment, central controller (Optional).

Challenges and Future Directions

The paper identifies several challenges affecting MACD systems, such as:

• Scalability: Handling a growing number of agents increases computational complexity and requires robust managing communication channels.
• Non-Stationarity: Constantly changing environments disrupt learning processes, necessitating algorithms that adapt in real-time.
• Coordination and Communication Efficiency: Establishing efficient protocols that balance the cost and benefits of communication among agents.

The survey suggests embracing hybrid models that integrate LLM reasoning within MARL frameworks to enhance adaptability and decision-making. Future research is encouraged to focus on improving sample efficiency, developing robust simulation platforms, and incorporating human-like reasoning and ethical considerations into MACD.

Figure 4: A schematic representation of three distinct communication methods among agents, with arrows indicating the direction of message transmission. (a) Broadcasting communication: The activated agent transmits messages to all other agents within the communication network. (b) Targeted communication: Agents selectively communicate with specific target agents based on a supervisory mechanism that regulates the timing, content, and recipients of the messages. (c) Networked communication: Agents engage in localized interactions with their neighboring agents within the network.

Conclusion

This comprehensive survey of multi-agent cooperative decision-making offers valuable insights into the complexities and methodologies shaping this research domain. As technology evolves, MACD systems promise transformative applications across varied sectors, revolutionizing how autonomous systems interact and make decisions in increasingly dynamic and uncertain environments. The integration of LLMs and MARL techniques represents a promising avenue for research, potentially redefining the capabilities of intelligent systems to achieve human-level performance and beyond.