Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling (2305.09993v2)

Abstract: We introduce Reprompting, an iterative sampling algorithm that automatically learns the Chain-of-Thought (CoT) recipes for a given task without human intervention. Through Gibbs sampling, Reprompting infers the CoT recipes that work consistently well for a set of training samples by iteratively sampling new recipes using previously sampled recipes as parent prompts to solve other training problems. We conduct extensive experiments on 20 challenging reasoning tasks. Results show that Reprompting outperforms human-written CoT prompts substantially by +9.4 points on average. It also achieves consistently better performance than the state-of-the-art prompt optimization and decoding algorithms.

• The paper introduces Reprompting, an automated method that optimizes chain-of-thought prompts using iterative Gibbs sampling to enhance AI reasoning.
• It refines initial zero-shot prompts by merging diverse reasoning strategies, reducing reliance on time-consuming human-crafted prompts.
• Performance benchmarks show that Reprompting outperforms traditional methods by adapting its prompts to different large language models for improved results.

Introduction to Reprompting

Research at Microsoft has led to the development of Reprompting, a computational approach for optimizing the way AI performs complex, multi-step reasoning tasks. At its core, Reprompting uses Gibbs sampling, a statistical algorithm, to automatically generate prompts for AI models, specifically LLMs, that guide them through a reasoning process known as Chain-of-Thought (CoT). This technique yields an innovative method for improving AI reasoning without human input.

The Challenge with Chain-of-Thought Prompts

Traditional few-shot prompting methods are effective for simple tasks but often fall short in tasks requiring intricate, multi-step reasoning. When the solution involves a sequence of logical steps, the AI needs more than just the final answer – it requires a pathway of reasoning to reach that conclusion. That's where CoT prompting comes in, which instructs the model to arrive at the answer by following a stepwise explanation. However, crafting these CoT prompts has typically depended on human expertise, placing significant limits on scalability and versatility.

Reprompting Algorithm Explained

Reprompting innovates by iterating over initial guesses at CoT recipes, which comprise stepwise reasoning paths, to develop more efficient and accurate methods of solving problems. It starts with zero-shot prompting, where no specific guidance is given, then iteratively samples from the LLM output to refine the recipes. By combining successful reasoning strategies from various initial attempts, this method converges to a set of CoT recipes that significantly improve the model’s problem-solving ability.

Performance and Implications

The performance of Reprompting has been rigorously evaluated using known benchmarks, showing considerable improvement over existing baselines, including human-written CoTs. The results also indicate that the effectiveness of CoT recipes can vary between different LLMs, emphasizing the importance of tailoring prompts to specific models for optimal results.

In summary, Reprompting presents a significant advance in automating the development of CoT prompts, enhancing AI reasoning capabilities across challenging tasks. Its ability to combine the strengths of different LLMs to improve performance holds substantial potential for the future of AI problem-solving methodologies.