On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents (2408.00989v4)

Abstract: LLM-based multi-agent systems have shown great abilities across various tasks due to the collaboration of expert agents, each focusing on a specific domain. However, the impact of clumsy or even malicious agents--those who frequently make errors in their tasks--on the overall performance of the system remains underexplored. This paper investigates: (1) What is the resilience of various system structures (e.g., A$\rightarrow$B$\rightarrow$C, A$\leftrightarrow$B$\leftrightarrow$C) under faulty agents, on different downstream tasks? (2) How can we increase system resilience to defend against these agents? To simulate faulty agents, we propose two approaches--AutoTransform and AutoInject--which introduce mistakes into the agents' responses. Experiments on four downstream tasks using six systems show that the "hierarchical" structure, i.e., A$\rightarrow$(B$\leftrightarrow$C), exhibits superior resilience with the lowest performance drop of 5.5%, compared to 10.5% and 23.7% of other two structures. To further improve resilience, we introduce (1) Challenger, that introduces a mechanism for each agent to challenge others' outputs, and (2) Inspector, an additional agent to review and correct messages, recovering up to 96.4% errors made by faulty agents. Our code and data are available at https://github.com/CUHK-ARISE/MAS-Resilience.

• The paper demonstrates that hierarchical MAS architectures are most robust, showing only a 23.6% performance drop under malicious influence.
• The study introduces AutoTransform and AutoInject to simulate subtle and direct errors, evaluating their impact across varied tasks.
• Experimental findings reveal that objective tasks are more prone to errors, while defense strategies like Inspector and Challenger boost resilience.

An Analysis of Resilience in Multi-Agent Systems with Malicious Agents

The paper "On the Resilience of Multi-Agent Systems with Malicious Agents" explores the capability of multi-agent systems (MAS) to maintain operational integrity in the presence of agents that are intentionally causing errors. This paper is crucial given the widespread deployment of MAS across diverse domains such as code generation and text translation. The research aims to understand the resilience of various MAS architectures against malicious interventions and suggest methods to enhance this resilience.

The paper addresses two primary research questions:

1. What is the resilience of different MAS structures (Linear, Flat, Hierarchical) under malicious influence across multiple tasks?
2. How can system resilience be improved to defend against malicious agents?

Simulation of Malicious Agents

Two innovative methods, AutoTransform and AutoInject, are designed to simulate the behavior of malicious agents. AutoTransform modifies an agent's profile to introduce subtle, stealthy errors while maintaining functional integrity. AutoInject, on the other hand, directly injects errors into the messages traded among agents. These methods enable a granular analysis of the impact of malicious agents on various MAS architectures.

Experimental Setup

The experiments span four significant tasks:

1. Code Generation: Using HumanEval to assess the synthesis of correct Python code.
2. Math Problem Solving: Employing CIAR for multi-step reasoning in arithmetic.
3. Translation: Utilizing CommonMT for commonsense translation tasks.
4. Text Evaluation: Leveraging FairEval for the comparison of LLM responses.

Six MAS frameworks representing three architectural paradigms were evaluated:

• Linear: MetaGPT and Self-collab.
• Flat: Camel and SPP.
• Hierarchical: MAD and AgentVerse.

Key Findings

1. System Architecture and Resilience

Hierarchical structures demonstrated the highest resilience with the least performance drop (~23.6%). This is attributed to their inherent ability to incorporate multiple viewpoints and higher-level agents for error correction. Flat structures showed moderate resilience, while Linear systems were the most vulnerable, suffering from a lack of multi-agent communication and oversight.

2. Nature of Downstream Tasks

Objective tasks like code generation and math problem solving are more adversely affected by malicious agents compared to subjective tasks like translation and text evaluation. This indicates that the rigor and formalization required in objective tasks make them more susceptible to subtle errors.

3. Error Rates and Types

The paper found that increasing the proportion of erroneous messages (P_m) had a more significant impact on system performance than increasing the number of errors within a single message (P_e). Moreover, semantic errors caused a greater degradation in performance compared to syntactic errors, as the latter are easier to detect and correct.

Case Studies and Observations

The experiments revealed scenarios where injected errors paradoxically led to performance improvements. For instance, obvious errors prompted agents to engage in corrective measures which also addressed pre-existing issues. This speaks to the potential of deliberate error introduction as a mechanism to bolster system robustness.

Defense Strategies

Two defense strategies were proposed:

1. Inspector: An additional agent reviews all inter-agent messages to identify and correct errors.
2. Challenger: Augmented agent profiles that empower agents to challenge and rectify each other's outputs.

Both strategies showed efficacy in improving resilience, though neither could fully mitigate the impact of malicious agents.

Implications and Future Directions

This research contributes significant insights into the design of resilient MAS. The hierarchical structure, with its multi-agent supervision and error correction capabilities, aligns well with effective human organizational modes. Future research could explore the interplay between agent diversity, role specificity, and system robustness. Moreover, extending these methodologies to other LLMs will help generalize the findings, ensuring better preparedness against malicious interventions in various MAS deployment scenarios.

In conclusion, the paper provides a comprehensive framework for evaluating and enhancing the resilience of MAS against malicious agents, underscoring the importance of robust architectural design and proactive defense mechanisms.