Formal Mathematics Statement Curriculum Learning (2202.01344v1)

Abstract: We explore the use of expert iteration in the context of LLMing applied to formal mathematics. We show that at same compute budget, expert iteration, by which we mean proof search interleaved with learning, dramatically outperforms proof search only. We also observe that when applied to a collection of formal statements of sufficiently varied difficulty, expert iteration is capable of finding and solving a curriculum of increasingly difficult problems, without the need for associated ground-truth proofs. Finally, by applying this expert iteration to a manually curated set of problem statements, we achieve state-of-the-art on the miniF2F benchmark, automatically solving multiple challenging problems drawn from high school olympiads.

• The paper introduces expert iteration in formal mathematics, interleaving proof search and learning to autonomously solve problems of increasing difficulty.
• It demonstrates state-of-the-art performance on the miniF2F benchmark by efficiently using synthetic data for curriculum-driven problem solving.
• The study addresses challenges like infinite action spaces and lack of self-play through a lean-gym interface, paving the way for advanced AI reasoning.

Formal Mathematics Statement Curriculum Learning

The paper "Formal Mathematics Statement Curriculum Learning" investigates the application of expert iteration to formal mathematics within the scope of LLMing. Expert iteration, in this context, is interpreted as a combination of proof search and learning, which significantly outperforms exclusive proof search given the same computational budget. Notably, this approach allows the model to autonomously construct and navigate a curriculum of problems of escalating difficulty without relying on predefined solutions. The authors achieved state-of-the-art results on the miniF2F benchmark by leveraging this method.

Key Contributions

1. Expert Iteration Mechanism: The core contribution of the paper is the integration of expert iteration within formal mathematics, which is demonstrated to be effective across a range of problem difficulties. This approach interleaves proof search with learning, incrementally refining the model's proficiency in solving more complex problems autonomously.
2. Efficient Problem Solving: By applying expert iteration to a manually curated dataset, the authors achieved a marked improvement in performance, demonstrating the practical impact of their approach. They successfully solved numerous complex problems from high school olympiads, contributing significantly to the miniF2F benchmark.
3. Handling Infinite Action Space and Lack of Self-Play: The paper addresses two main challenges in formal mathematics: the infinite action space and the absence of a direct self-play setup. The proposed method effectively mitigates these challenges through a curriculum compiled from auxiliary problem sets of varying difficulty.
4. Lean-Gym Interface: The implementation of lean-gym, a REPL interface for the Lean theorem prover, facilitated an efficient exploration and iteration process. This tool offers a streamlined means of interaction between the LLM and the formal environment.
5. Synthetic Data and Curriculum Learning: The generation of synthetic inequality problems allowed the researchers to test the model's ability to learn from an intrinsic difficulty gradient. The paper confirms the potential of expert iteration to foster continuous self-improvement without requiring ground-truth problem solutions.

Implications and Future Directions

This work underscores the potential of formalized problem-solving in AI, especially emphasizing the utility of curriculum learning without ground-truth reliance. The implications of these findings are substantial for both theoretical advancements and practical applications, such as in software verification where formal proofs are critical.

Looking forward, automating the generation of formal statements and scaling curriculum learning could further accelerate progress in this domain. Additionally, exploring model scaling and optimizing compute usage remain potent areas for ongoing research. The limitations noted regarding the chaining of advanced reasoning steps present an opportunity for further innovation in designing enhanced search procedures and understanding model capacity.

In summary, this paper contributes significantly to the intersection of formal mathematics and AI, providing a robust framework for advancing automated reasoning through structured learning environments. The methodologies and results presented hold promise for deeper engagement with complex, formally-defined domains.