Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset (2402.10176v2)

Published 15 Feb 2024 in cs.CL, cs.AI, and cs.LG
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset

Abstract: Recent work has shown the immense potential of synthetically generated datasets for training LLMs, especially for acquiring targeted skills. Current large-scale math instruction tuning datasets such as MetaMathQA (Yu et al., 2024) and MAmmoTH (Yue et al., 2024) are constructed using outputs from closed-source LLMs with commercially restrictive licenses. A key reason limiting the use of open-source LLMs in these data generation pipelines has been the wide gap between the mathematical skills of the best closed-source LLMs, such as GPT-4, and the best open-source LLMs. Building on the recent progress in open-source LLMs, our proposed prompting novelty, and some brute-force scaling, we construct OpenMathInstruct-1, a math instruction tuning dataset with 1.8M problem-solution pairs. The dataset is constructed by synthesizing code-interpreter solutions for GSM8K and MATH, two popular math reasoning benchmarks, using the recently released and permissively licensed Mixtral model. Our best model, OpenMath-CodeLlama-70B, trained on a subset of OpenMathInstruct-1, achieves a score of 84.6% on GSM8K and 50.7% on MATH, which is competitive with the best gpt-distilled models. We release our code, models, and the OpenMathInstruct-1 dataset under a commercially permissive license.

PDF HTML Abstract

OpenMathInstruct-1: Enhancing Mathematical Reasoning in LLMs with a Large-Scale Open Dataset

Overview of OpenMathInstruct-1

OpenMathInstruct-1 represents a significant progress in the domain of mathematical reasoning for LLMs, featuring a dataset comprising 1.8 million problem-solution pairs. This dataset is unique not only in its size—which is at least four times larger than the largest existing datasets in this domain—but also in its open licensing, facilitating unrestricted usage and contribution to future research. It addresses notable limitations in the current landscape of LLM training for mathematical reasoning by leveraging the recently released, permissively licensed Mixtral model for dataset generation.

Rationale Behind the Development

The inception of OpenMathInstruct-1 was motivated by the constraints posed by proprietary models in the existing data generation pipelines for mathematical reasoning. These constraints included legal restrictions on usage, higher generation costs, and challenges in reproducibility due to the opaque nature of closed-source models. Utilizing the Mixtral model, OpenMathInstruct-1 was compiled, aiming to bridge the performance gap observed between open-source LLMs and their closed-source counterparts in mathematical tasks.

Methodology

The generation of OpenMathInstruct-1 involved a blend of brute-force scaling and innovative prompting strategies. Notably, a novel approach using masked text solutions significantly boosted the efficiency of solution synthesis, leading to remarkable training set coverage of 99.9% for GSM8K and 93% for MATH benchmarks. This methodology showcased a leap in performance for the fine-tuned OpenMath-CodeLlama-70B model, which demonstrated competitive results against the leading gpt-distilled models on major mathematical reasoning benchmarks.

Implications and Future Directions

The creation and public release of OpenMathInstruct-1 under a permissively open license are poised to have profound implications for AI research, particularly in enhancing the mathematical reasoning capabilities of LLMs. The dataset not only sets a new benchmark for the scale and accessibility of training data in this niche but also exemplifies the potential for collaborative advancements in AI through open-source initiatives.

Additionally, the successful application of the Mixtral model for dataset generation exemplifies the narrowing performance gap between open-source and proprietary models in specialized tasks. This trend could encourage further contributions to open AI research, fostering an environment where advancements are not hindered by commercial restrictions.

The research also highlights the untapped potential of leveraging instructive data in LLM training. With OpenMathInstruct-1 now available, the AI research community is better equipped to explore novel training paradigms that could further refine the reasoning and problem-solving capabilities of LLMs.

Conclusion

OpenMathInstruct-1 represents a significant milestone in the quest for improving mathematical reasoning in LLMs. By providing an unprecedentedly large and openly accessible dataset, this research contribution not only elevates the capabilities of open-source models but also opens new avenues for collaborative research and development. As the AI field continues to evolve, resources like OpenMathInstruct-1 will be instrumental in shaping the future trajectory of intelligent systems capable of sophisticated reasoning and knowledge application.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Llemma: An Open Language Model For Mathematics.
  2. How is ChatGPT’s behavior changing over time?
  3. Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. TMLR.
  4. Training Verifiers to Solve Math Word Problems.
  5. Ronen Eldan and Yuanzhi Li. 2023. TinyStories: How Small Can Language Models Be and Still Speak Coherent English? arXiv.
  6. PAL: Program-aided Language Models. In ICML, pages 10764–10799. PMLR.
  7. ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving. In ICLR.
  8. Textbooks Are All You Need. arXiv.
  9. Measuring Mathematical Problem Solving With the MATH Dataset. NeurIPS.
  10. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580.
  11. Large Language Models Can Self-Improve. In EMNLP.
  12. Mistral 7B. arXiv.
  13. Mixtral of Experts.
  14. NeMo: a toolkit for building AI applications using neural modules. In Systems for ML Workshop, NeurIPS.
  15. Solving Quantitative Reasoning Problems with Language Models. In NeurIPS.
  16. Textbooks Are All You Need II: phi-1.5 technical report. arXiv.
  17. MARIO: MAth Reasoning with code Interpreter Output – A Reproducible Pipeline.
  18. Let’s Verify Step by Step. arXiv.
  19. TinyGSM: achieving> 80% on GSM8k with small language models. arXiv preprint arXiv:2312.09241.
  20. Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. arXiv.
  21. WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct. arXiv preprint arXiv:2308.09583.
  22. Lila: A Unified Benchmark for Mathematical Reasoning. In EMNLP.
  23. Orca: Progressive Learning from Complex Explanation Traces of GPT-4.
  24. GPT-4 Technical Report.
  25. Code Llama: Open Foundation Models for Code. arXiv.
  26. Galactica: A Large Language Model for Science.
  27. Llama 2: Open Foundation and Fine-Tuned Chat Models.
  28. MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning. In ICLR.
  29. Self-consistency improves chain of thought reasoning in language models. In ICLR.
  30. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS.
  31. WizardLM: Empowering Large Language Models to Follow Complex Instructions. arXiv.
  32. Outcome-supervised Verifiers for Planning in Mathematical Reasoning. arXiv.
  33. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models. In ICLR.
  34. Scaling Relationship on Learning Mathematical Reasoning with Large Language Models.
  35. MAmmoTH: Building math generalist models through hybrid instruction tuning. In ICLR.
  36. STaR: Bootstrapping Reasoning With Reasoning. In NeurIPS.
  37. Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification. In ICLR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Shubham Toshniwal (25 papers)
  2. Ivan Moshkov (6 papers)
  3. Sean Narenthiran (7 papers)
  4. Daria Gitman (4 papers)
  5. Fei Jia (19 papers)
  6. Igor Gitman (15 papers)
Citations (52)
View on Semantic Scholar
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - Kipok/NeMo-Skills (432 stars)