AgreeMate: Teaching LLMs to Haggle (2412.18690v1)

Abstract: We introduce AgreeMate, a framework for training LLMs to perform strategic price negotiations through natural language. We apply recent advances to a negotiation setting where two agents (i.e. buyer or seller) use natural language to bargain on goods using coarse actions. Specifically, we present the performance of LLMs when used as agents within a decoupled (modular) bargaining architecture. We demonstrate that using prompt engineering, fine-tuning, and chain-of-thought prompting enhances model performance, as defined by novel metrics. We use attention probing to show model attention to semantic relationships between tokens during negotiations.

An Analysis of "AgreeMate: Teaching LLMs to Haggle"

The paper, "AgreeMate: Teaching LLMs to Haggle," presents a compelling examination of the application of LLMs in the strategic negotiation domain, specifically within price-based bargaining contexts. The researchers propose a framework, AgreeMate, which adapts recent advances in LLMs to test their abilities as negotiation agents in a structured bargaining setting. The framework's central focus is on creating modular LLM agents for both buyers and sellers, implementing strategies like fine-tuning and chain-of-thought (CoT) prompting to enhance performance. This paper explores the potential for LLMs to function as agents within negotiation scenarios, comparing their effectiveness across different model scales, architectures, and training methodologies.

Framework and Contributions

The AgreeMate framework is notable for its systematic approach to evaluating LLM capabilities in negotiation, leveraging a decoupled modular architecture that enables role specialization. The paper makes several contributions:

1. Evaluation Framework: Development of a comprehensive framework to assess LLM performance across different model scales and training methods, with specific focus on strategic negotiation.
2. Role-Specialized Fine-Tuning: Implementation of targeted fine-tuning strategies creating specialized negotiation agents (buyer, seller, and generalist), which demonstrated the effectiveness of role-specific optimization.
3. Model Comparison: Detailed analysis of LLaMA models, from 3B to 70B parameters, revealing insights into how model size and training affect negotiation outcomes.
4. Evaluation Metrics: Establishment of robust metrics for negotiation success, such as fairness, bias, relative efficiency, and probing ratio, providing a nuanced understanding of LLM negotiation abilities.
5. Attention Probing: Use of attention head probing to gain insights into the internal dynamics of LLMs during negotiation, illustrating the contribution of specific attention mechanisms to bargaining behavior.

Experimental Findings

The experimental results in AgreeMate are extensive. The paper evaluated 21 distinct models, including baseline, CoT-enhanced, personality-driven, and fine-tuned agents, in a series of negotiation scenarios using the Craigslist and Deal or No Deal datasets. Key findings across model configurations are as follows:

• Larger Models: Higher agreement rates, indicating more consistent successes in reaching negotiation conclusions.
• Chain-of-Thought Effectiveness: Enhanced exploration and strategic reasoning but resulted in decreased fairness and increased bias in smaller models.
• Personality Influence: Personality-driven models (aggressive, fair, passive) showed that aggressive models tended to dominate negotiations, securing outcomes in the buyer's favor, while passive models encouraged more balanced dialogues.

Implications and Future Directions

The paper's implications extend to both theoretical understanding and practical applications of LLMs in autonomous negotiation:

• Strategic Improvement: The inclusion of strategic reasoning and role-specific optimization offers pathways for improving LLMs as negotiation agents, suggesting the potential for LLMs to transition from mimicking average human behavior to actively employing strategic reasoning.
• Real-World Applications: In online marketplaces, such systems could play critical roles in price discovery and ensuring equitable transactions, either by participating in negotiations directly or by offering strategic guidance to human negotiators.
• Future Research: Further exploration could investigate the identified limitations, such as the tendency for smaller models to engage in repetitive dialogue or granular negotiations, and assess how extending turn limits or model refinement might mitigate these issues.

In conclusion, the AgreeMate framework represents a significant step in understanding and improving LLMs' ability to perform complex negotiation tasks. The paper provides a substantive foundation for ongoing research into LLMs' capabilities, with promising implications for artificial intelligence applications in strategic communication scenarios across various sectors.