RoundTable: Investigating Group Decision-Making Mechanism in Multi-Agent Collaboration (2411.07161v2)

Abstract: Effective group decision-making is critical in Multi-Agent Systems (MAS). Yet, how different mechanisms for reaching consensus impact collaboration quality and efficiency remains understudied. We conduct a systematic study on group decision-making mechanisms in a decentralized setting. Through controlled experiments, we analyze how different voting rules affect decision quality and efficiency in a multi-round collaboration. Results reveal that majority voting often cause inefficient collaboration due to its strict acceptance criteria. At the extreme, unanimous voting gives 87% lower initial performance than the best-performing method. Our qualitative analysis of cross-agent communication shows that messages become longer and more repetitive over time: while message length increases by 84%, similarity to the previous round increases to 90%. Based on these insights, language-based early stopping methods make the performance 13% closer to oracle while reducing rounds by 50%. Our findings highlight the crucial role of group decision-making in optimizing MAS collaboration.

• The paper introduces RoundTable, a platform for testing decentralized group decision-making in multi-agent systems using varied social choice methods.
• The paper demonstrates that score-based voting mechanisms achieve higher group utility and fairness compared to unanimous methods in both simulated and recommendation environments.
• The paper highlights the role of dialogue analysis and adaptive early stopping in enhancing collaboration efficiency within multi-agent systems.

RoundTable: Investigating Group Decision-Making Mechanism in Multi-Agent Collaboration

This paper explores the capabilities of Multi-Agent Systems (MAS) in situations requiring decentralized group decision-making. It introduces RoundTable, a collaboration platform engineered to examine the dynamics of agent communication and decision processes when employing different social choice methods. The paper explores the interaction dynamics between agents within a MAS when decentralized decision-making frameworks like voting rules are employed to reach a consensus across various social choice models.

Multi-Agent Collaboration and Decision-Making

RoundTable operates a round-based collaboration model where agents concurrently communicate, propose solutions, and vote. Each agent's participation influences the decision selected via a social choice mechanism, showcasing how collaborative efforts in decentralized settings enhance group efficacy (Figure 1).

Figure 1: Overview of our multi-agent collaboration platform: RoundTable. It uses a round-based collaboration where agents simultaneously send messages, propose solutions, and vote. Based on a social choice mechanism, RoundTable selects the most preferred proposal for group decisions.

Social Choice Mechanisms

To evaluate the impact of social choice methods, the paper investigates six distinct mechanisms: Unanimous, Majority, Plurality, Rated, Ranked, and Cumulative voting. Each offers varying levels of decision flexibility, profoundly affecting collaboration efficiency and fairness among agents.

Experiments and Results

Exchange Economy Environment

This simulated environment showcases the strengths and limitations of different voting mechanisms in achieving group utility. In particular, score-based mechanisms demonstrated superior performance in early rounds, illustrating their capability in aligning diverse agent preferences (Figure 2).

Figure 2: Exchange economy environment results in group total utility. Line plots show the utility achieved over rounds, while bar plots indicate the ratio of cases where participants did not reach an agreement.

The performance metrics include overall group utility, fairness measured by a Min/Max ratio, and rigidity, which denotes how frequently the agents maintained old allocations. Findings highlighted that unanimous voting struggled due to its inflexibility, while score-based mechanisms adjusted more dynamically to agent inputs, achieving more equitable group outcomes.

Recommendation System Environment

In more complex real-world applications, such as recommendation systems, decentralized MAS can simulate collective decision-making effectively. The MAS approach demonstrated parity with state-of-the-art models in prediction tasks, showcasing how agent collaboration enriches problem-solving through diverse perspective integration (Figure 3).

Figure 3: Comparison between social choices in the recommendation system environment. Line plots show the comparison of MAE/RMSE achieved over rounds, with bar plots indicating the agreement ratios.

Cross-Agent Conversation Analysis

The paper utilized linguistic features of agent interactions, such as message length, complexity, and information novelty, to evaluate collaboration efficacy. Dialogue acts like Inform, Request, and Propose dominated the exchanges, which underscored the agents' tendency to frequently share and seek information, essential for progressive problem-solving.

The dialogue act transition graphs elucidate how agents move from data exchange phases to proposing and negotiating decisions, impacting the collaboration's fluidity and resolution, particularly in negotiation-heavy settings (Figure 4).

Figure 4: Dialogue act transition graph for exchange economy and recommendation system environments.

Early Stopping in Collaboration

The research proposes mechanisms for optimal stopping in collaborations to maximize efficiency. Techniques such as monitoring information difference and language patterns provide insights into when to conclude decision-making processes effectively, showing improved outcomes over fixed-round approaches in multi-agent environments.

Conclusion

The RoundTable platform and its findings underscore the importance of selecting appropriate social choice methods tailored to the task's nature for effective MAS collaboration. Score-based mechanisms proved advantageous in environments requiring nuanced judgments, while one-vote systems displayed robustness in fairness and simplicity. Moving forward, linguistic indicators of collaboration present a promising avenue for adaptive early stopping rules that could enhance real-world MAS applications.