Dice Question Streamline Icon: https://streamlinehq.com

Conjectured reduction of reasoning gap via specialized prompting

Establish whether specialized prompting strategies—including chain-of-thought, tree-of-thought, chain-of-code, chain-of-hindsight, program-of-thought, algorithmic skill prompting, and progressive hint prompting—reduce the reasoning gap for large language models on functional benchmarks.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper surveys prompting methods that elicit explicit reasoning steps and notes their demonstrated accuracy benefits. The authors conjecture that these methods would reduce the reasoning gap, inviting empirical or theoretical validation on functionalized benchmarks.

References

They have been shown to improve accuracy, and we would conjecture they would reduce the gap.

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap (2402.19450 - Srivastava et al., 29 Feb 2024) in Related Work, Techniques to improve reasoning in language models (Specialized prompting)