AI Research Assistant for Computer Scientists

Enhancing Mathematical Reasoning in LLMs Through InternLM-Math

Introduction

InternLM-Math emerges as a notable advancement in the domain of mathematical reasoning LLMs, building on its predecessor InternLM2. This work seeks to significantly enhance the abstract reasoning capabilities of LLMs, especially in the mathematical domain, by introducing a series of novel methodologies and integrating them into a coherent training regimen. The resulting model demonstrates superior performance across a variety of mathematical reasoning benchmarks, establishing new state-of-the-art (SOTA) metrics in the field.

Advancements in Pre-Training and Fine-Tuning

The InternLM-Math model introduces an intricate pre-training phase, leveraging a diverse data corpus that includes common crawl data, domain-specific data, and synthetic data aimed at reinforcing the model's numerical operation capabilities. This pre-training strategy not only enriches the model's understanding but also its application of mathematical concepts in diverse contexts. The use of deduplication techniques and exact formulation decontamination further refines the quality of the training data, ensuring a high degree of relevance and accuracy in the model's learning process.

Subsequently, the supervised fine-tuning (SFT) phase of InternLM-Math's development focuses on a multidimensional enhancement of the model's capabilities. This phase incorporates chain-of-thought reasoning, code interpretation, and an innovative approach to augmenting mathematical problems. These facets collectively boost the model's ability to not only solve mathematical problems but also to generate new problems and verify the correctness of its solutions, thereby supporting a self-improving mechanism within LLMs for math reasoning.

Unification of Reasoning and Coding Abilities

A noteworthy innovation in InternLM-Math is the unification of reasoning and coding abilities under a unified seq2seq format, termed Reasoning Interleaved with Coding (RICO). This approach enables the model to interleave mathematical reasoning with coding sequences, offering a more natural and human-like problem-solving process. The integration of formal reasoning, through the use of the LEAN theorem prover, further distinguishes InternLM-Math by enabling it to tackle formal mathematical statements, bridging the gap between informal natural language reasoning and formal mathematical verification.

Leveraging Reward Modeling

Another significant aspect of InternLM-Math is its incorporation of reward modeling for improving the selection of reasoning paths and solutions. By employing both outcome reward models (ORM) and process reward models (PRM), InternLM-Math can more accurately identify and prioritize correct reasoning processes and solutions. This method not only enhances the model's performance on benchmark tasks but also aids in the generation of high-quality, verifiable data for self-improvement.

Practical Implications and Future Directions

InternLM-Math’s advancements present several practical implications for the field of AI-driven mathematical reasoning. Its ability to generate new mathematical problems and verify solutions opens up avenues for automated curriculum development and evaluation in educational contexts. Additionally, the model’s integration of formal reasoning capabilities suggests potential applications in automated theorem proving and formal verification, areas of significant importance in computer science and logic.

Looking forward, InternLM-Math sets the stage for future explorations into the uncharted territories of AI capabilities in mathematics. Its innovative methodologies and impressive performance lay a foundation for further research into self-improving systems, possibly leading to LLMs that can autonomously expand their knowledge and reasoning abilities across various domains of mathematics and beyond.

Conclusion

InternLM-Math represents a significant step forward in the pursuit of advanced mathematical reasoning abilities within LLMs. Through its comprehensive approach to pre-training and fine-tuning, along with the integration of code interpretation, formal reasoning, and reward modeling, InternLM-Math offers a glimpse into the future of AI-driven mathematics education, research, and application. The remarkable performance across multiple benchmarks underlines the efficacy of these innovations, paving the way for further advancements in the domain of AI and mathematical reasoning.