Determine the effect of optimized prompting on the reasoning gap

Determine whether and by how much optimized prompting strategies—such as chain-of-thought, tree-of-thought, and chain-of-code—reduce the reasoning gap for language models evaluated on MATH() once the benchmark is fully functionalized.

Background

The authors note that current reasoning gap measurements use simple few-shot prompting and that more sophisticated prompting could lower the gap. They plan to fully functionalize MATH() to enable a rigorous assessment of how such prompting strategies affect the gap, marking this as an open question to be resolved with complete coverage.