EffiReason-Bench: Efficient Reasoning in LLMs

• EffiReason-Bench is a unified benchmark that defines efficient reasoning in large language models with a focus on reducing redundant token generation.
• It categorizes methods into Reasoning Blueprints, Dynamic Execution, and Post-hoc Refinement, offering cross-paradigm comparisons across varied datasets and backbones.
• The benchmark introduces the E³-Score to jointly evaluate effectiveness and efficiency, highlighting architecture-dependent performance trade-offs.

EffiReason-Bench is a unified benchmark for evaluating efficient reasoning in LLMs, with the explicit goal of comparing methods that reduce the cost of Chain-of-Thought (CoT) reasoning without sacrificing accuracy (Huang et al., 13 Nov 2025). It is motivated by the observation that CoT often yields unnecessarily long and redundant reasoning traces, increasing inference cost and, in some cases, degrading correctness. The benchmark addresses a methodological gap in prior work: efficiency-oriented approaches were typically assessed under fragmented conditions, with different models, tasks, and metrics, making cross-paradigm comparison unreliable. EffiReason-Bench standardizes the comparison space across three method categories—Reasoning Blueprints, Dynamic Execution, and Post-hoc Refinement—while also introducing verified CoT annotations for CommonsenseQA and LogiQA and a trade-off metric, E$^3$-Score, for joint evaluation of effectiveness and efficiency (Huang et al., 13 Nov 2025).

1. Motivation and conceptual scope

EffiReason-Bench is framed around a specific failure mode of contemporary CoT prompting: strong reasoning performance is often accompanied by excessive token generation rather than calibrated inference. In the benchmark’s formulation, the problem is not simply answer quality, but the efficiency–accuracy trade-off under standardized evaluation. This positions efficient reasoning as a first-class benchmarking target rather than a secondary implementation concern (Huang et al., 13 Nov 2025).

The benchmark evaluates methods that intervene at different points in the reasoning pipeline. This organizational choice reflects the paper’s claim that efficient reasoning is not a single technique but a family of approaches distinguished by when they compress, constrain, or restructure reasoning. EffiReason-Bench therefore serves not merely as a leaderboard instrument, but as a common experimental substrate for comparing train-free and train-based methods under aligned prompting, formatting, backbones, and datasets (Huang et al., 13 Nov 2025).

A central conclusion of the benchmark is that efficient reasoning is not governed by a universal best method. Instead, the best-performing strategy depends on backbone scale, task difficulty, and architecture. This suggests that efficiency interventions interact with model internals and task structure in a highly non-uniform way, rather than expressing a stable Pareto improvement across conditions (Huang et al., 13 Nov 2025).

2. Method taxonomy and benchmarked paradigms

EffiReason-Bench organizes efficient reasoning methods into three paradigms, corresponding to the stage at which the intervention occurs (Huang et al., 13 Nov 2025).

Paradigm	Intervention point	Methods listed
Reasoning Blueprints	Before or at the start of reasoning	Chain-of-Draft (CoD), Sketch-of-Thought (SoT), O1-Pruner
Dynamic Execution	During inference	Soft Thinking, Skeleton-of-Thought (SloT)
Post-hoc Refinement	After rationale generation	TokenSkip

Reasoning Blueprints are methods that shape reasoning from the outset. O1-Pruner is described as a train-based approach that fine-tunes with a length-aware reward so the model learns to prune or shorten reasoning while preserving correctness. Sketch-of-Thought (SoT) uses a lightweight router to map inputs to high-level reasoning sketches, encouraging compressed templates. Chain-of-Draft (CoD) uses prompt instructions that force very short intermediate drafts. The benchmark notes that CoD and SoT are especially aggressive at compression and often trade away accuracy, whereas O1-Pruner is more conservative and tends to preserve performance better, though at the cost of additional training (Huang et al., 13 Nov 2025).

Dynamic Execution methods alter the reasoning process during decoding. Soft Thinking reasons in a continuous concept space through “concept tokens,” described as soft, probability-weighted mixtures of token embeddings. Skeleton-of-Thought (SloT) first generates a high-level skeleton and then expands each part, which introduces a more structured, potentially parallelizable inference process. Within the reported results, Soft Thinking is characterized as the most robust dynamic method, while SloT often achieves substantial compression but is more unstable on difficult tasks (Huang et al., 13 Nov 2025).

Post-hoc Refinement is represented by TokenSkip, which removes tokens with low semantic contribution from a completed rationale. Unlike the blueprint and dynamic categories, TokenSkip does not modify reasoning generation upstream. Its reported behavior is notably architecture-sensitive: it is highly effective on commonsense and logic tasks and often strong on LLaMA backbones, yet it can be harmful on some Qwen mathematical settings, particularly on MATH500 (Huang et al., 13 Nov 2025).

The reference condition is Standard CoT, yielding a seven-method comparison suite in total: CoD, SoT, O1-Pruner, Soft Thinking, SloT, TokenSkip, and Standard CoT (Huang et al., 13 Nov 2025).

3. Benchmark construction and verified CoT annotations

EffiReason-Bench evaluates four datasets: GSM8K, MATH500, CommonsenseQA, and LogiQA (Huang et al., 13 Nov 2025). These cover mathematics, commonsense reasoning, and logical reasoning. A major construction challenge arises from the fact that CommonsenseQA and LogiQA do not natively provide reasoning traces. To support step-by-step evaluation, the benchmark introduces verified CoT annotations for these two datasets via a three-stage pipeline (Huang et al., 13 Nov 2025).

The first stage imposes a standardized reasoning structure with four components:

1. Premise identification
2. Relationship formalization
3. Condition evaluation
4. Option mapping

This standardization is intended to regularize annotations across questions and annotators, preventing free-form explanations from obscuring logical structure (Huang et al., 13 Nov 2025).

The second stage performs option-wise comparative analysis. For each multiple-choice item, the solution must analyze all answer options, not just justify the selected one. The annotation must explain why the correct option is correct, why distractors are wrong, and how the reasoning aligns with the ground-truth answer. This design explicitly forces comparative reasoning instead of post hoc rationalization of the selected option (Huang et al., 13 Nov 2025).

The third stage is dual human verification, used to remove logical inconsistencies, unsupported inferences, and mismatches between the reasoning trace and the gold answer. The benchmark emphasizes this because automatically generated CoT traces may contain contradictions or logical gaps even when they sound plausible (Huang et al., 13 Nov 2025).

The experiments also standardize output formatting. Models must produce a single-line final answer beginning with Final Answer:. For multiple-choice tasks, the answer is a single option letter (A/B/C/D), while for open-ended mathematics it is a number or short span. In few-shot settings, exemplars are drafted by GPT-4o and human-audited; a fixed global seed is used; the same exemplar lists are reused across methods, backbones, and regimes; and each exemplar follows the pattern Question → Reasoning (numbered steps) → Final Answer (Huang et al., 13 Nov 2025).

The appendix further specifies noise-injection variants for robustness testing. These include reasoning-step noise—deletion, permutation, and repetition of steps—and semantic noise. For mathematics, semantic noise consists of small numeric perturbations; for commonsense and logic, it includes logical connective reversals such as “therefore $\rightarrow$ however,” “thus $\rightarrow$ but,” “because $\rightarrow$ despite,” “implies $\rightarrow$ contradicts,” and “leads to $\rightarrow$ prevents” (Huang et al., 13 Nov 2025).

4. Experimental protocol and comparison design

The benchmark evaluates six open-source instruction-tuned backbones:

• Qwen2.5-Instruct 7B
• Qwen2.5-Instruct 14B
• Qwen2.5-Instruct 32B
• LLaMA-Instruct 1B
• LLaMA-Instruct 8B
• LLaMA-Instruct 70B

This spans a scale range from 1B to 70B, enabling analysis of how efficient reasoning interacts with model size and architecture (Huang et al., 13 Nov 2025).

The evaluation includes both train-free and train-based regimes. Under this protocol, each efficient reasoning method is compared against Standard CoT across all six backbones and all four datasets, using consistent formatting and prompting rules. This standardized matrix is a core contribution of EffiReason-Bench, since the paper explicitly identifies fragmented prior evaluation as a barrier to fair comparison (Huang et al., 13 Nov 2025).

The empirical results support a non-uniform view of efficient reasoning. On GSM8K, many methods reduce token counts, but some incur accuracy loss. On MATH500, trade-offs become sharper, indicating that more demanding mathematical reasoning is less tolerant of aggressive compression. Soft Thinking is reported as the most robust dynamic method and usually matches or exceeds CoT accuracy. TokenSkip performs well on LLaMA backbones but can catastrophically harm Qwen, especially on MATH500. O1-Pruner tends to preserve accuracy better than heuristic compression methods, but it is resource-intensive and caused OOM on larger models in the reported setup (Huang et al., 13 Nov 2025).

For CommonsenseQA, the benchmark reports substantially greater compression tolerance. CoD and SoT can reduce tokens by over 90%. TokenSkip is especially strong in this regime, often preserving or improving accuracy while shortening outputs, and Soft Thinking remains stable and often matches Standard CoT. O1-Pruner is reported to help weaker backbones, especially LLaMA 8B (Huang et al., 13 Nov 2025).

For LogiQA, compression is again more delicate. CoD tends to hurt accuracy despite extreme compression. Soft Thinking remains strong on accuracy but yields limited token savings, while TokenSkip is reported as the best-balanced method overall, improving or preserving accuracy with moderate token reduction (Huang et al., 13 Nov 2025).

The few-shot analysis adds further structure to these findings. SloT scales well with more demonstrations in some tasks and transfer settings. By contrast, CoD and SoT are described as rigid: they often learn only “be short” and do not benefit substantially from additional shots. Under noisy exemplars, SloT is more robust than CoD and SoT. In cross-domain transfer, CoT is the most stable baseline, while blueprint methods transfer poorly (Huang et al., 13 Nov 2025).

5. E$^3$-Score and efficiency–effectiveness evaluation

EffiReason-Bench introduces E$^3$-Score—the Efficiency–Effectiveness Equilibrium Score—to evaluate the trade-off between accuracy and token efficiency (Huang et al., 13 Nov 2025). The metric is inspired by the Constant Elasticity of Substitution (CES) formulation from economics and is designed to avoid two problems identified in prior efficiency metrics: over-rewarding token savings and introducing discontinuities or heuristic thresholds.

Let $A_0, T_0$ denote baseline accuracy and token count, and $A, T$ denote the method’s accuracy and token count. The paper defines the normalized ratios as

$\rightarrow$0

Here, $\rightarrow$1 measures relative accuracy improvement under odds normalization, and $\rightarrow$2 measures relative token efficiency improvement (Huang et al., 13 Nov 2025).

The E$\rightarrow$3-Score is then defined as

$\rightarrow$4

The parameter $\rightarrow$5 controls the weight assigned to accuracy versus efficiency, and the paper sets $\rightarrow$6, so stronger baselines place greater emphasis on accuracy. The parameter $\rightarrow$7 controls substitutability; with $\rightarrow$8, the two components behave more like complements, meaning token savings cannot fully compensate for accuracy degradation. The benchmark uses $\rightarrow$9 by default, which makes the score a weighted harmonic mean and yields a deliberately conservative aggregation (Huang et al., 13 Nov 2025).

The paper contrasts E$\rightarrow$0-Score with ACU and AES. ACU (Accuracy per Computation Unit) is said to undervalue small but meaningful gains when baselines are already strong, while AES (Accuracy–Efficiency Score) is characterized as piecewise and dependent on manual thresholds and heuristic hyperparameters. By construction, E$\rightarrow$1-Score is intended to be smooth, stable, principled, and less heuristic-dependent (Huang et al., 13 Nov 2025).

A sensitivity theorem is also provided. With $\rightarrow$2, $\rightarrow$3, and $\rightarrow$4, the paper gives the bound

$\rightarrow$5

equivalently,

$\rightarrow$6

and

$\rightarrow$7

These expressions are used to justify that E$\rightarrow$8 is Lipschitz-stable in $\rightarrow$9, with a bound independent of $\rightarrow$0 (Huang et al., 13 Nov 2025). This suggests that the metric is less sensitive to parameter perturbations than piecewise heuristic alternatives.

6. Empirical patterns, limitations, and significance

The principal empirical conclusion is that no single method dominates everywhere (Huang et al., 13 Nov 2025). The benchmark repeatedly shows that the effect of an efficiency intervention depends on task complexity, model scale, and architecture. CommonsenseQA is reported as compression-tolerant; LogiQA is more fragile; and MATH500 is the hardest setting, exposing the weaknesses of aggressive compression most clearly (Huang et al., 13 Nov 2025).

Several method-specific patterns recur across the reported experiments. Soft Thinking is the most robust dynamic method, especially when preserving accuracy is the primary concern. TokenSkip is often the strongest post-hoc method on commonsense and logic tasks, but its performance is architecture-sensitive and can be harmful on some Qwen mathematics settings. O1-Pruner is accurate and conservative but training-heavy and resource-intensive. CoD and SoT achieve extreme compression, yet they are also the methods most likely to sacrifice accuracy on harder reasoning problems (Huang et al., 13 Nov 2025).

Model architecture matters independently of scale. A major insight of the benchmark is that the same method can behave very differently on Qwen and LLaMA backbones. The paper identifies TokenSkip as the clearest example: excellent on LLaMA, but harmful on Qwen in some mathematical settings (Huang et al., 13 Nov 2025). This suggests that efficient reasoning interventions are not plug-compatible abstractions but architecture-contingent procedures.

The benchmark also yields practical guidance for future evaluation. It suggests that efficient reasoning methods should be assessed across multiple backbones rather than a single host model; that both accuracy and token cost should be reported; that architecture-specific behavior should be treated as central rather than incidental; that robustness to noisy and cross-domain exemplars should be included; and that trade-off assessment should use a stable metric such as E$\rightarrow$1 rather than ad hoc heuristics (Huang et al., 13 Nov 2025).

A plausible implication is that EffiReason-Bench shifts efficient reasoning from an informal optimization objective into a reproducible benchmarking problem. Its contribution is not only the evaluation of seven methods across six backbones and four datasets, but the establishment of a standardized cross-paradigm framework in which efficiency is measured jointly with correctness, under aligned reasoning traces and a stable trade-off metric (Huang et al., 13 Nov 2025).