Are Your LLMs Capable of Stable Reasoning? (2412.13147v4)

Abstract: The rapid advancement of LLMs has shown remarkable progress in complex reasoning tasks. However, a significant disparity exists between benchmark performances and real-world applications. We attribute this gap primarily to current evaluation protocols and metrics, which inadequately capture the full spectrum of LLM capabilities, especially in complex reasoning tasks where both accuracy and consistency are essential. In this paper, we introduce G-Pass@$k$, a novel evaluation metric that continuously assesses model performance across multiple sampling attempts, quantifying both the model's performance potential and its stability. Through extensive experiments on various public and newly constructed benchmarks, we employ G-Pass@$k$ in conjunction with state-of-the-art LLMs to provide comprehensive insights into their potential capabilities and operational consistency. Our findings reveal a significant opportunity to enhance the realistic reasoning abilities of LLMs, underscoring the necessity for more robust evaluation metrics.

• The paper introduces G-Pass@, a novel metric evaluating LLM consistency across multiple reasoning attempts.
• The paper presents LiveMathBench, a dynamic benchmark featuring challenging multilingual mathematical problems.
• The paper reveals that increasing model size does not guarantee improved stability, highlighting a gap in current evaluation metrics.

An Insightful Overview of "Are Your LLMs Capable of Stable Reasoning?"

The paper "Are Your LLMs Capable of Stable Reasoning?" addresses a critical issue in the evaluation of LLMs: the gap between performance on standard benchmarks and real-world applications, especially in complex reasoning tasks. The authors identify current evaluation metrics as a limiting factor in assessing LLM capabilities, particularly regarding both accuracy and consistency in reasoning. This work introduces two novel contributions to address these limitations: the G-Pass@ metric and the LiveMathBench benchmark.

Key Contributions and Findings

1. G-Pass@: A New Evaluation Metric

The G-Pass@ metric extends beyond traditional evaluation metrics, which often fail to capture the stability of an LLM's reasoning over multiple sampling attempts. While metrics like Greedy Accuracy and Pass@K focus on single-instance evaluation or peak performance across multiple runs, they do not adequately assess output stability. G-Pass@ provides a more comprehensive measure by evaluating model performance under varying thresholds of correctness. This approach allows for a continuous assessment, capturing both the model's potential and its consistency, thus offering a more nuanced understanding of an LLM’s capabilities in complex reasoning tasks.

1. LiveMathBench: A Dynamic Benchmark

The authors present LiveMathBench, a dynamic evaluation framework comprising challenging mathematical problems. This benchmark is designed to reduce the risks of data leakage and ensure relevance with contemporary mathematical challenges. It evaluates models on both English and Chinese problems from widely recognized mathematical competitions, such as the China National Mathematical Olympiad and the American Mathematics Competition.

1. Experimental Insights

Through extensive experiments using G-Pass@ on various LLMs, including general-purpose and mathematics-specialized models, the research highlights significant room for improvement in LLMs' "realistic" reasoning capabilities. Notably, the paper reveals that these models exhibit substantial instability when tackling challenging reasoning tasks. For many models, performance drops exceed 50% when evaluated with stricter G-Pass@ thresholds, underscoring the inadequacy of current metrics in capturing the full spectrum of LLM performance.

Additionally, merely increasing model size does not correspond to improved stable reasoning capabilities. The findings indicate a gap between potential capabilities, as measured by G-Pass@ with low thresholds, and actual stability when faced with high thresholds, emphasizing the need for further research to develop more robust reasoning and evaluation frameworks.

1. Implications and Future Directions

The implications of this research are both practical and theoretical. Practically, it suggests that for applications requiring reliable outcomes, new metrics that balance response diversity with stable performance are crucial. Theoretically, it opens avenues for research into methods that can better harness the potential capabilities of LLMs in a stable and reliable manner. Future developments in AI could focus on enhancing the inherent stability of reasoning mechanisms within LLMs and refining evaluation frameworks to provide a balanced view of model performance across both potential and consistency dimensions.

In summary, this paper makes significant strides in addressing the disconnect between LLM evaluation and real-world application. By introducing G-Pass@ and LiveMathBench, the authors provide valuable tools for assessing and improving the robustness and consistency of reasoning in LLMs. This work lays a foundation for future research aimed at bridging the gap between experimental benchmarks and practical applications in AI.