2000 character limit reached
Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration (2311.08152v2)
Published 14 Nov 2023 in cs.CL
Abstract: LLMs have shown remarkable capabilities in general natural language processing tasks but often fall short in complex reasoning tasks. Recent studies have explored human-like problem-solving strategies, such as self-correct, to push further the boundary of single-model reasoning ability. In this work, we let a single model "step outside the box" by engaging multiple models to correct each other. We introduce a multi-agent collaboration strategy that emulates the academic peer review process. Each agent independently constructs its own solution, provides reviews on the solutions of others, and assigns confidence levels to its reviews. Upon receiving peer reviews, agents revise their initial solutions. Extensive experiments on three different types of reasoning tasks show that our collaboration approach delivers superior accuracy across all ten datasets compared to existing methods. Further study underscores the effectiveness of integrating confidence in reviews, demonstrates the superiority of feedback exchange over mere solution sharing, and highlights the role of capability and diversity in fostering successful collaboration.
- Palm 2 technical report.
- Sparks of artificial general intelligence: Early experiments with gpt-4.
- Chateval: Towards better llm-based evaluators through multi-agent debate.
- Think you have solved question answering? try arc, the ai2 reasoning challenge.
- Training verifiers to solve math word problems.
- Improving factuality and reasoning in language models through multiagent debate.
- Chain-of-thought hub: A continuous effort to measure large language models’ reasoning performance.
- Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346–361.
- Learning to solve arithmetic word problems with verb categorization. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 523–533, Doha, Qatar. Association for Computational Linguistics.
- Large language models cannot self-correct reasoning yet.
- Language models can solve computer tasks.
- Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems.
- Parsing algebraic word problems into equations. Transactions of the Association for Computational Linguistics, 3:585–597.
- Camel: Communicative agents for "mind" exploration of large language model society.
- Prd: Peer rank and discussion improve large language model based evaluations.
- Encouraging divergent thinking in large language models through multi-agent debate.
- Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158–167, Vancouver, Canada. Association for Computational Linguistics.
- Self-refine: Iterative refinement with self-feedback.
- M. Minsky. 1988. Society Of Mind. Touchstone book. Simon & Schuster.
- OpenAI. 2023. Gpt-4 technical report.
- Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies.
- Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080–2094, Online. Association for Computational Linguistics.
- Communicative agents for software development.
- Subhro Roy and Dan Roth. 2015. Solving general arithmetic word problems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1743–1752, Lisbon, Portugal. Association for Computational Linguistics.
- Reflexion: Language agents with verbal reinforcement learning.
- Corex: Pushing the boundaries of complex reasoning through multi-model collaboration.
- Challenging BIG-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13003–13051, Toronto, Canada. Association for Computational Linguistics.
- Llama 2: Open foundation and fine-tuned chat models.
- Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2609–2634, Toronto, Canada. Association for Computational Linguistics.
- Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
- Unleashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration.
- Emergent abilities of large language models. Transactions on Machine Learning Research. Survey Certification.
- Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
- The rise and potential of large language model based agents: A survey.
- Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7572–7590, Singapore. Association for Computational Linguistics.
- Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms.
- Exploring collaboration mechanisms for llm agents: A social psychology view.
- Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning Representations.
- Zhenran Xu (12 papers)
- Senbao Shi (3 papers)
- Baotian Hu (67 papers)
- Jindi Yu (3 papers)
- Dongfang Li (46 papers)
- Min Zhang (630 papers)
- Yuxiang Wu (27 papers)