SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning (2308.00436v3)

Abstract: The recent progress in LLMs, especially the invention of chain-of-thought prompting, has made it possible to automatically answer questions by stepwise reasoning. However, when faced with more complicated problems that require non-linear thinking, even the strongest LLMs make mistakes. To address this, we explore whether LLMs are able to recognize errors in their own step-by-step reasoning, without resorting to external resources. To this end, we propose SelfCheck, a general-purpose zero-shot verification schema for recognizing such errors. We then use the results of these checks to improve question-answering performance by conducting weighted voting on multiple solutions to the question. We test SelfCheck on three datasets (GSM8K, MathQA, and MATH) and find that it successfully recognizes errors and, in turn, increases final answer accuracies.

Overview of "SelfCheck: using LLMs to zero-shot check their own step-by-step reasoning"

The paper "SelfCheck: using LLMs to zero-shot check their own step-by-step reasoning" by Ning Miao, Yee Whye Teh, and Tom Rainforth, tackles a fundamental problem in the application of LLMs to complex reasoning tasks: the verification of step-by-step reasoning outputs. Despite the advances made possible by techniques such as Chain-of-Thought prompting, current LLMs still exhibit limitations in solving difficult reasoning tasks due to propagated errors in multi-step solutions. To address this, the authors propose "SelfCheck," a zero-shot verification schema designed to allow LLMs to self-evaluate their own reasoning processes without the need for external resources or additional training data.

Main Contributions and Methodology

SelfCheck consists of a systematic approach to verify each step of a solution generated by LLMs. This approach is built on the realization that checking each step individually based on its context can potentially isolate and identify errors more effectively than evaluating the entire reasoning chain at once. The SelfCheck methodology splits the verification task into a series of stages:

1. Target Extraction: Identifying the goal of each reasoning step to enable precise verification.
2. Information Collection: Gathering requisite data from preceding steps and the problem statement to contextualize the current step.
3. Step Regeneration: Independently regenerating each step based on the extracted target and collected context, leveraging the generative capabilities of LLMs.
4. Result Comparison: Comparing the original step with the regenerated one to assess its correctness.

This framework thus transforms the verification process into a series of more manageable tasks aligned with the strengths of LLMs in generating coherent language, which aids in reducing error correlation between generation and checking phases.

Evaluation and Results

The paper evaluates SelfCheck's performance on three mathematical datasets: GSM8K, MathQA, and MATH. These datasets vary in complexity and scope, with tasks ranging from primary school arithmetic to high-school level competition mathematics. The results demonstrate that SelfCheck substantially increases answer accuracy in LLM-based solutions over simple majority voting, showcasing its ability to provide reliable confidence scores for each solution. Notably, when utilizing SelfCheck for weighted voting across multiple solutions, it consistently outperforms baseline methods, including more data-intensive approaches like few-shot verification with chain-of-thought exemplars.

Another significant finding is that the efficacy of SelfCheck persists when applied with different configurations of LLM generators and checkers, including the combination of advanced models such as GPT-4 and less computationally intensive models like GPT-3.5 as either the generator or verifier.

Implications and Future Directions

SelfCheck elucidates a methodologically elegant solution to a pervasive problem in AI: the automatic verification of machine-generated reasoning. By demonstrating the capacity for LLMs to self-verify through a structured decomposition of the verification task, this approach opens new avenues for deploying LLMs in domains where correctness and reliability are paramount, such as education, finance, and scientific computing.

Future work could explore the application of SelfCheck to more diverse reasoning tasks beyond mathematics, expanding into domains that involve logical reasoning and natural language understanding. Furthermore, integrating SelfCheck with other AI-driven diagnostic tools could enhance its robustness and applicability, providing a comprehensive framework for improving the reliability of AI reasoning systems.

Overall, SelfCheck signifies an important advancement in the field of natural language processing and AI, contributing towards building more autonomous, self-regulating systems that ascertain the trustworthiness of their reasoning without the need for extrinsic oversight or augmented training protocols.