Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

S$^3$c-Math: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasoners (2409.01524v1)

Published 3 Sep 2024 in cs.CL and cs.AI

Abstract: Self-correction is a novel method that can stimulate the potential reasoning abilities of LLMs. It involves detecting and correcting errors during the inference process when LLMs solve reasoning problems. However, recent works do not regard self-correction as a spontaneous and intrinsic capability of LLMs. Instead, such correction is achieved through post-hoc generation, external knowledge introduction, multi-model collaboration, and similar techniques. In this paper, we propose a series of mathematical LLMs called S$3$c-Math, which are able to perform Spontaneous Step-level Self-correction for Mathematical reasoning. This capability helps LLMs to recognize whether their ongoing inference tends to contain errors and simultaneously correct these errors to produce a more reliable response. We proposed a method, which employs a step-level sampling approach to construct step-wise self-correction data for achieving such ability. Additionally, we implement a training strategy that uses above constructed data to equip LLMs with spontaneous step-level self-correction capacities. Our data and methods have been demonstrated to be effective across various foundation LLMs, consistently showing significant progress in evaluations on GSM8K, MATH, and other mathematical benchmarks. To the best of our knowledge, we are the first to introduce the spontaneous step-level self-correction ability of LLMs in mathematical reasoning.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Step-Level Value Preference Optimization for Mathematical Reasoning.
  2. Training Verifiers to Solve Math Word Problems.
  3. LM vs LM: Detecting Factual Errors via Cross Examination. https://arxiv.org/abs/2305.13281v1.
  4. Improving Factuality and Reasoning in Language Models through Multiagent Debate.
  5. Measuring Mathematical Problem Solving With the MATH Dataset.
  6. Active Retrieval Augmented Generation. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 7969–7992. Singapore: Association for Computational Linguistics.
  7. When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs.
  8. Efficient Memory Management for Large Language Model Serving with PagedAttention.
  9. DotaMath: Decomposition of Thought with Code Assistance and Self-Correction for Mathematical Reasoning.
  10. Common 7b language models already possess strong math capabilities. arXiv preprint arXiv:2403.04706.
  11. PRD: Peer Rank and Discussion Improve Large Language Model Based Evaluations.
  12. Let’s Verify Step by Step.
  13. Large Language Models Have Intrinsic Self-Correction Ability.
  14. Augmenting math word problems via iterative question composing. arXiv preprint arXiv:2401.09003.
  15. Self-Refine: Iterative Refinement with Self-Feedback.
  16. Meta AI. 2024. Introducing Meta Llama 3: The Most Capable Openly Available LLM to Date. https://ai.meta.com/blog/meta-llama-3/.
  17. Mistral AI team. 2023. Mistral 7B. https://mistral.ai/news/announcing-mistral-7b/.
  18. Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Automated Correction Strategies. Transactions of the Association for Computational Linguistics, 12: 484–506.
  19. Are NLP Models Really Able to Solve Simple Math Word Problems?
  20. REFINER: Reasoning Feedback on Intermediate Representations. In Graham, Y.; and Purver, M., eds., Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 1100–1126. St. Julian’s, Malta: Association for Computational Linguistics.
  21. Fine-Tuning with Divergent Chains of Thought Boosts Reasoning Through Self-Correction in Language Models.
  22. Qwen Team. 2024. Introducing Qwen2-Math | Qwen. https://qwenlm.github.io/blog/qwen2-math/.
  23. Analysing Mathematical Reasoning Abilities of Neural Models.
  24. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.
  25. A Survey of Reasoning with Foundation Models.
  26. Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them.
  27. Evidence-Based Factual Error Correction. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 3298–3309. Online: Association for Computational Linguistics.
  28. LLMs Cannot Find Reasoning Errors, but Can Correct Them given the Error Location. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics ACL 2024, 13894–13908. Bangkok, Thailand and virtual meeting: Association for Computational Linguistics.
  29. Math-Shepherd: Verify and Reinforce LLMs Step-by-Step without Human Annotations.
  30. Self-Consistency Improves Chain of Thought Reasoning in Language Models.
  31. Chain-of-Thought Reasoning Without Prompting.
  32. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.
  33. Can We Verify Step by Step for Incorrect Answer Detection? arXiv preprint arXiv:2402.10528.
  34. Can LLMs Solve longer Math Word Problems Better? arXiv preprint arXiv:2405.14804.
  35. InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning.
  36. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models.
  37. ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search.
  38. Small Language Models Need Strong Verifiers to Self-Correct Reasoning.
  39. OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement.
  40. OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement.
  41. JiuZhang3. 0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models. arXiv preprint arXiv:2405.14365.
  42. JiuZhang3. 0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models. arXiv preprint arXiv:2405.14365.
  43. Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model. arXiv preprint arXiv:2407.10167.
  44. Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model. arXiv preprint arXiv:2407.10167.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Yuchen Yan (44 papers)
  2. Jin Jiang (17 papers)
  3. Yang Liu (2253 papers)
  4. Yixin Cao (138 papers)
  5. Xin Xu (188 papers)
  6. Mengdi Zhang (37 papers)
  7. Xunliang Cai (63 papers)
  8. Jian Shao (29 papers)