Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation (2403.06769v3)
Abstract: We investigate non-collaborative dialogue agents, which are expected to engage in strategic conversations with diverse users, for securing a mutual agreement that leans favorably towards the system's objectives. This poses two main challenges for existing dialogue agents: 1) The inability to integrate user-specific characteristics into the strategic planning, and 2) The difficulty of training strategic planners that can be generalized to diverse users. To address these challenges, we propose Trip to enhance the capability in tailored strategic planning, incorporating a user-aware strategic planning module and a population-based training paradigm. Through experiments on benchmark non-collaborative dialogue tasks, we demonstrate the effectiveness of Trip in catering to diverse users.
- How well can llms negotiate? negotiationarena platform and analysis. arXiv preprint arXiv:2402.05863.
- Social influence dialogue systems: A survey of datasets and models for social influence tasks. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 750–766.
- Be selfish, but wisely: Investigating the impact of agent personality in mixed-motive human-agent interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13078–13092, Singapore. Association for Computational Linguistics.
- Controllable mixed-initiative dialogue generation through prompting. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 951–966, Toronto, Canada. Association for Computational Linguistics.
- A survey on proactive dialogue systems: Problems, methods, and prospects. arXiv preprint arXiv:2305.02750.
- Prompting and evaluating large language models for proactive dialogues: Clarification, target-guided, and non-collaboration.
- Plug-and-play policy planner for large language model powered dialogue agents. arXiv preprint arXiv:2311.00262.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
- Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388.
- Resper: Computationally modelling resisting strategies in persuasive conversations. arXiv preprint arXiv:2101.10545.
- Strategies and motives for resistance to persuasion: An integrative framework. Frontiers in psychology, 6:1201.
- Ivar Frisch and Mario Giulianelli. 2024. Llm agents in interaction: Measuring personality consistency and linguistic alignment in interacting populations of large language models. arXiv preprint arXiv:2402.02896.
- Improving language model negotiation with self-play and in-context learning from ai feedback.
- Lewis R Goldberg. 1992. The development of markers for the big-five factor structure. Psychological assessment, 4(1):26.
- Decoupling strategy and generation in negotiation dialogues. arXiv preprint arXiv:1808.09637.
- Enhancing large language model induced task-oriented dialogue systems through look-forward motivated goals.
- Bayes-adaptive monte-carlo planning and learning for goal-oriented dialogues. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7994–8001.
- Evaluating and inducing personality in pre-trained language models. Advances in Neural Information Processing Systems, 36.
- Personallm: Investigating the ability of gpt-3.5 to express personality traits and gender differences. arXiv preprint arXiv:2305.02547.
- Evaluating persuasion strategies and deep reinforcement learning methods for negotiation dialogue agents. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 480–484, Valencia, Spain. Association for Computational Linguistics.
- Are llms effective negotiators? systematic evaluation of the multifaceted capabilities of llms in negotiation dialogues. arXiv preprint arXiv:2402.13550.
- Interacting with non-cooperative user: A new paradigm for proactive dialogue policy.
- Deal or no deal? end-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2443–2453.
- Legoeval: An open-source toolkit for dialogue system evaluation via crowdsourcing. arXiv preprint arXiv:2105.01992.
- One cannot stand for everyone! leveraging multiple user simulators to train task-oriented dialogue systems. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1–21.
- Shima Rahimi Moghaddam and Christopher J Honey. 2023. Boosting theory-of-mind performance in large language models via prompting. arXiv preprint arXiv:2304.11490.
- David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and brain sciences, 1(4):515–526.
- Personality traits in large language models. arXiv preprint arXiv:2307.00184.
- Neural theory-of-mind? on the limits of social intelligence in large lms. arXiv preprint arXiv:2210.13312.
- Susanne G Scott and Reginald A Bruce. 1995. Decision-making style: The development and assessment of a new measure. Educational and psychological measurement, 55(5):818–831.
- Role play with large language models. Nature, 623(7987):493–498.
- How to build user simulators to train rl-based dialog systems. arXiv preprint arXiv:1909.01388.
- Does role-playing chatbots capture the character personalities? assessing personality traits for role-playing chatbots. arXiv preprint arXiv:2310.17976.
- Persuasion for good: Towards a personalized persuasive dialogue system for social good. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5635–5649, Florence, Italy. Association for Computational Linguistics.
- Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229–256.
- Heinz Wimmer and Josef Perner. 1983. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition, 13(1):103–128.
- Improving dialog systems for negotiation with personality modeling.
- Prompt-based Monte-Carlo tree search for goal-oriented dialogue policy planning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7101–7125, Singapore. Association for Computational Linguistics.
- Let’s negotiate! a survey of negotiation dialogue systems. arXiv preprint arXiv:2402.01097.
- Ask an expert: Leveraging language models to improve strategic reasoning in goal-oriented dialogue models. arXiv preprint arXiv:2305.17878.
- Towards effective automatic debt collection with persona awareness. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 32–45, Singapore. Association for Computational Linguistics.
- Explaining agent behavior with large language models. arXiv preprint arXiv:2309.10346.
- How far are large language models from agents with theory-of-mind? arXiv preprint arXiv:2310.03051.
- Sotopia: Interactive evaluation for social intelligence in language agents. arXiv preprint arXiv:2310.11667.
- Augmenting non-collaborative dialog systems with explicit semantic and strategic dialog history.