· Trending Papers
Emergent Mind

Large Language Models Cannot Self-Correct Reasoning Yet

(2310.01798)
Published Oct 3, 2023 in cs.CL and cs.AI

Abstract

LLMs have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content. A contemporary methodology, self-correction, has been proposed as a remedy to these issues. Building upon this premise, this paper critically examines the role and efficacy of self-correction within LLMs, shedding light on its true potential and limitations. Central to our investigation is the notion of intrinsic self-correction, whereby an LLM attempts to correct its initial responses based solely on its inherent capabilities, without the crutch of external feedback. In the context of reasoning, our research indicates that LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction. Drawing from these insights, we offer suggestions for future research and practical applications in this field.

We're not able to analyze this paper right now due to high demand.

Please check back later (sorry!).

Sign up for a free account or log in to generate a summary of this paper:

Sign Up Now
or
Log In

We ran into a problem analyzing this paper.

Newsletter

Get summaries of trending comp sci papers delivered straight to your inbox:

Unsubscribe anytime.

YouTube
References
  1. Artificial hallucinations in chatgpt: implications in scientific writing. Cureus, 15(2)
  2. PaLM 2 Technical Report
  3. Constitutional AI: Harmlessness from AI Feedback
  4. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
  5. Sparks of Artificial General Intelligence: Early experiments with GPT-4
  6. Extracting training data from large language models. In USENIX Security Symposium, volume 6
  7. ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
  8. Teaching Large Language Models to Self-Debug
  9. Training Verifiers to Solve Math Word Problems
  10. Improving Factuality and Reasoning in Language Models through Multiagent Debate
  11. The Capacity for Moral Self-Correction in Large Language Models
  12. Rarr: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.  16477–16508
  13. CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
  14. Jie Huang and Kevin Chen-Chuan Chang. Towards reasoning in large language models: A survey. In Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics
  15. Are large pre-trained language models leaking your personal information? In Findings of the Association for Computational Linguistics: EMNLP 2022, pp.  2038–2047, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics.
  16. Language Models (Mostly) Know What They Know
  17. Language models can solve computer tasks. The ICML Workshop on Artificial Intelligence & Human Computer Interaction
  18. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213
  19. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324
  20. Multi-step Jailbreaking Privacy Attacks on ChatGPT
  21. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
  22. Let's Verify Step by Step
  23. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems
  24. Is Self-Repair a Silver Bullet for Code Generation?
  25. OpenAI. Gpt-4 technical report
  26. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744
  27. Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
  28. REFINER: Reasoning Feedback on Intermediate Representations
  29. True few-shot learning with language models. Advances in neural information processing systems, 34:11054–11070
  30. Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
  31. Learning representations by back-propagating errors. nature, 323(6088):533–536
  32. Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage
  33. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pp.  31210–31227. PMLR
  34. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems
  35. Reinforcement learning: An introduction. MIT press
  36. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp.  4149–4158
  37. Shepherd: A Critic for Language Model Generation
  38. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations
  39. Jailbroken: How Does LLM Safety Training Fail?
  40. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837
  41. Generating sequences by learning to self-correct. In The Eleventh International Conference on Learning Representations
  42. Large Language Models as Optimizers
  43. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics
  44. Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  45. Why Does ChatGPT Fall Short in Providing Truthful Answers?
  46. Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification
  47. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning Representations, 2023b.
  48. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations
  49. Universal and Transferable Adversarial Attacks on Aligned Language Models

Show All 49