EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Models (2511.10201v1)

Abstract: LLMs with Chain-of-Thought (CoT) prompting achieve strong reasoning but often produce unnecessarily long explanations, increasing cost and sometimes reducing accuracy. Fair comparison of efficiency-oriented approaches is hindered by fragmented evaluation practices. We introduce EffiReason-Bench, a unified benchmark for rigorous cross-paradigm evaluation of efficient reasoning methods across three categories: Reasoning Blueprints, Dynamic Execution, and Post-hoc Refinement. To enable step-by-step evaluation, we construct verified CoT annotations for CommonsenseQA and LogiQA via a pipeline that enforces standardized reasoning structures, comprehensive option-wise analysis, and human verification. We evaluate 7 methods across 6 open-source LLMs (1B-70B) on 4 datasets spanning mathematics, commonsense, and logic, and propose the E3-Score, a principled metric inspired by economic trade-off modeling that provides smooth, stable evaluation without discontinuities or heavy reliance on heuristics. Experiments show that no single method universally dominates; optimal strategies depend on backbone scale, task complexity, and architecture.

• The paper introduces EffiReason-Bench, a unified framework that benchmarks efficient reasoning in LLMs by organizing methods into three paradigms and introducing the novel E³-Score metric.
• It evaluates multiple datasets spanning mathematics, commonsense, and logic to reveal architectural dependencies and trade-offs between reasoning accuracy and token efficiency.
• Results show that dynamic execution methods, such as Soft Thinking, offer robust performance, while blueprint approaches risk accuracy loss unless paired with targeted fine-tuning.

EffiReason-Bench: A Unified Foundation for Efficient Reasoning in LLMs

Introduction and Motivation

EffiReason-Bench introduces a rigorous and comprehensive framework for evaluating efficiency-centric reasoning strategies in LLMs. The central motivation addresses critical weaknesses in the prevailing Chain-of-Thought (CoT) paradigm: while CoT elicits improved reasoning performance via explicit step-wise rationalization, it frequently leads to excessive verbosity ("overthinking"), incurring significant computational cost and error accumulation. Moreover, the landscape of efficient reasoning research is fragmented—spanning distinct paradigms (reasoning blueprints, dynamic execution, post-hoc refinement)—lacking unified benchmarks and principled metrics for fair comparison across model scales and reasoning domains.

EffiReason-Bench tackles these challenges by (i) organizing methods into three paradigms, (ii) constructing high-quality, human-verified, standardized CoT annotations for commonsense and logic datasets to enable step-wise process evaluation, and (iii) introducing the Efficiency–Effectiveness Equilibrium Score (E$^3$-Score), a metric inspired by Constant Elasticity of Substitution economics for robust, smooth trade-off analysis.

Figure 1: Overview of EffiReason-Bench, which compares efficient reasoning methods across diverse paradigms, backbones, and datasets.

Benchmark Design and Setting

Datasets and Backbone Models

EffiReason-Bench spans four datasets: GSM8K (grade school mathematics), MATH500 (competition-level math), CommonsenseQA (commonsense reasoning), and LogiQA (advanced logical reasoning). For operational transparency, CoT annotations for CommonsenseQA and LogiQA are human-curated via a multistage pipeline enforcing explicit premises, rationale formalization, comprehensive distractor justification, and dual verification.

Six backbone models are included: Qwen2.5-Instruct {7B, 14B, 32B} and LLaMA-Instruct {1B, 8B, 70B}. Method evaluation encompasses both train-free and train-based settings, with a standardized prompt protocol for answer production.

Efficient Reasoning Methods

Seven principal methods across three paradigms are benchmarked:

• Reasoning Blueprints: (pre-processing)
  • O1-Pruner (learned length-aware fine-tuning),
  • Sketch-of-Thought (adaptive pattern routing and compression),
  • Chain-of-Draft (instruction-driven brevity).
• Dynamic Execution: (in-process)
  • Soft Thinking (latent space probabilistic concept generation),
  • Skeleton-of-Thought (two-stage structured skeleton+expansion).
• Post-hoc Refinement: (post-processing)
  • TokenSkip (semantic pruning of completed rationales).

E$^3$-Score: Principled Efficiency–Effectiveness Metric

Prior measures (e.g., ACU, AES) are inadequate: ACU underweights advances near strong baselines; AES introduces parameter instability and discontinuities. The E$^3$-Score builds on CES (Constant Elasticity of Substitution), harmonizing accuracy ($A$) and efficiency (output tokens $T$) via:

$$r_{\text{acc}} = \frac{A}{1-A} / \frac{A_0}{1-A_0},\quad r_{\text{tok}} = \frac{T_0}{T}$$

$$\mathrm{E^3}(A,T) = \left(w \cdot r_{\text{acc}}^{\rho} + (1-w)\cdot r_{\text{tok}}^{\rho}\right)^{\frac{1}{\rho}}$$

Where $w=A_0$ weights accuracy in strong baseline regimes; $\rho=-1$ (weighted harmonic mean) penalizes asymmetric improvements and heavily penalizes dual degradation. The resulting metric enables robust and interpretable Pareto analysis across backbone scales and tasks.

Experimental Analysis Across Reasoning Domains

Mathematical Reasoning (GSM8K, MATH500)

Key findings:

• Blueprint methods: O1-Pruner (train-based) reliably preserves or improves accuracy with modest compression, while train-free methods (SoT, CoD) achieve maximal compression at the expense of accuracy.
• Dynamic execution: Soft Thinking achieves near-identical accuracy to standard CoT across scales and tasks, indicating exceptional robustness. Skeleton-of-Thought exhibits unstable performance, degrading accuracy on complex tasks.
• Post-hoc refinement: TokenSkip is highly sensitive to backbone architecture; catastrophic accuracy degradation observed on Qwen backbones for MATH500, but favorable trade-offs on LLaMA.

Commonsense Reasoning

Compression is highly robust:

• CoD and SoT compress up to 90% of tokens with minimal accuracy loss on Qwen; performance diverges on LLaMA where accuracy deteriorates.
• O1-Pruner enhances accuracy on weaker LLaMA baselines.
• TokenSkip is universally benign, maintaining or improving accuracy and reducing tokens across architectures.

Logical Reasoning

Logical inference is more fragility-prone:

• Aggressive blueprint compressions (CoD, SoT) typically degrade accuracy.
• O1-Pruner and TokenSkip markedly improve weak baseline performance—especially for LLaMA 1B and 8B. Soft Thinking offers the highest accuracy but little efficiency gain.

Few-shot and Cross-domain Reasoning

Few-shot experiments elucidate scaling and robustness properties:

• SloT scales favorably with increased shots; structured skeleton-based learning enhances accuracy, outpacing CoT on several datasets at higher shot counts.
• Blueprint methods (CoD, SoT) exhibit rigid compression, making them highly sensitive and fragile to noise or cross-domain exemplar settings.

  Figure 2: Few-shot performance (accuracy on top, tokens on bottom) of four reasoning methods on Qwen2.5-7B across four datasets, comparing clean vs. noisy settings.

Under noisy supervision, SloT demonstrates resilience while CoD/SoT are brittle. In cross-domain transfer:

Figure 3: Comparison of few-shot transfer from MATH500 to Commonsense, GSM8K, and LogiQA, in terms of accuracy (top) and tokens (bottom).

Figure 4: Comparison of few-shot transfer from LogiQA to Commonsense, GSM8K, and MATH500 in terms of accuracy (top) and tokens (bottom).

CoT maintains stability; SloT depends critically on shot count; blueprints transfer poorly and unreliably, especially with far-domain settings.

Theoretical and Practical Implications

EffiReason-Bench provides evidence for strong architectural and domain sensitivity in reasoning efficiency optimization—for instance, TokenSkip’s dichotomous effects between Qwen and LLaMA. Furthermore, dynamic execution strategies (Soft Thinking) consistently outperform other paradigms in terms of robustness, favoring adaptive concept-level token generation over parallel skeleton expansion. Blueprint methods offer maximal efficiency but at tangible risk to reasoning accuracy and transfer robustness unless supported by learned length-sensitive fine-tuning or careful architecture selection.

The E$^3$-Score metric sets a new standard for evaluating joint accuracy–efficiency performance without the confounds of heuristic hyperparameters or discontinuity, enabling reproducible and comparable assessment as models and reasoning paradigms continue to diversify.

Future Directions

EffiReason-Bench establishes a reproducible, evaluation-centric foundation for both method development and downstream benchmark analysis in efficient reasoning. The revealed architectural dependencies and paradigm trade-offs motivate continued research into backbone-aware efficiency strategies, more resilient blueprint mechanisms, and advanced process-aware CoT annotation pipelines. Future extensions may include multimodal reasoning tasks, broader model families, and robustification under adversarial or contaminated supervision.

Conclusion

EffiReason-Bench and E$^3$-Score furnish the research community with a unified, rigorous platform for principled evaluation of efficient reasoning methods in LLMs. The extensive empirical analysis demonstrates that optimal strategies are highly contingent on backbone architecture, domain, and the operational regime. Train-based blueprint compression and post-hoc refinement deliver notable accuracy improvements for weaker backbones; dynamic execution methods offer stability and robustness, while blueprint methods suffice for safe, high-efficiency reasoning principally on compatible architectures. The framework enables systematic development and comparison, setting the agenda for future advances in adaptive efficient reasoning systems.