HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs (2511.18760v1)

Abstract: Informal mathematics has been central to modern LLM reasoning, offering flexibility and enabling efficient construction of arguments. However, purely informal reasoning is prone to logical gaps and subtle errors that are difficult to detect and correct. In contrast, formal theorem proving provides rigorous, verifiable mathematical reasoning, where each inference step is checked by a trusted compiler in systems such as Lean, but lacks the exploratory freedom of informal problem solving. This mismatch leaves current LLM-based math agents without a principled way to combine the strengths of both paradigms. In this work, we introduce Hermes, the first tool-assisted agent that explicitly interleaves informal reasoning with formally verified proof steps in Lean. The framework performs intermediate formal checking to prevent reasoning drift and employs a memory module that maintains proof continuity across long, multi-step reasoning chains, enabling both exploration and verification within a single workflow. We evaluate Hermes on four challenging mathematical reasoning benchmarks using LLMs of varying parameter scales, from small models to state-of-the-art systems. Across all settings, Hermes reliably improves the reasoning accuracy of base models while substantially reducing token usage and computational cost compared to reward-based approaches. On difficult datasets such as AIME'25, Hermes achieves up to a 67% accuracy improvement while using 80% fewer total inference FLOPs. The implementation and codebase are publicly available at https://github.com/aziksh-ospanov/HERMES.

• The paper introduces Hermes, a hybrid LLM agent that couples step-wise chain-of-thought generation with Lean4 verification for robust mathematical proofs.
• It demonstrates a 67% accuracy improvement, 80% reduction in FLOPs, and 4x lower token usage on challenging benchmarks.
• The modular design with an integrated memory system enables real-time error detection and correction during multi-step reasoning.

Hermes: A Verifiable, Efficient Multi-Modular Agent for Mathematical Reasoning with LLMs

Introduction and Motivation

LLMs have demonstrated the capability to solve challenging mathematical problems when equipped with step-wise reasoning strategies. Nevertheless, their informal, natural-language reasoning chains (Chain-of-Thought, CoT) are highly susceptible to hallucinations, logical drift, and error accumulation, particularly for multi-step and nontrivial mathematical proofs. While recent advances in formal reasoning systems (Lean, Coq, Isabelle) have achieved machine-verified solutions at Olympiad-level performance, these systems lack the flexibility and informal inventiveness required for exploratory human-like reasoning. Neither paradigm alone suffices for both correctness and adaptability.

The paper "HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs" (2511.18760) introduces Hermes, a modular, Lean4-integrated agent that tightly couples step-wise LLM reasoning with immediate, in-context formal verification. This hybrid, neuro-symbolic architecture validates each critical reasoning step at inference time, leveraging formal signals and memory-based context continuity to minimize hallucination propagation and enhance both accuracy and resource efficiency.

Figure 1: Overview of the Hermes framework: a plug-and-play Lean4-driven agent that combines LLM step generation, translation to Lean, symbolic verification, and feedback for robust mathematical reasoning.

Hermes Framework: System Architecture and Design

Hermes implements a multi-stage pipeline consisting of four main components: (1) an LLM step generator, (2) a translation module (autoformalization of NL steps to Lean4), (3) a prover module for symbolic Lean verification (including both the original claim and its negation), and (4) a feedback/memory block that maintains proof-state and validation history. At each high-level step, the LLM pauses to invoke external tool modules. Translation first converts the current natural-language step into a canonical Lean4 proposition with built-in consistency checks (syntactic compilation and semantic backtranslation equivalence). On validation, the statement is subjected to proof-search via both whole-proof generation models and Lean's native solvers.

If the proof attempt or its negation succeeds, Hermes feeds back a high-confidence correctness/incorrectness signal to the LLM. If translation or proof fails (due to insufficient formalizability or resource/time limits), a VERIFICATION FAILURE status is returned, and the LLM is prompted to self-correct, revise, or re-explore alternate proof directions.

Figure 2: Complete Hermes pipeline with modular interface—LLM reasoning, autoformalization, Lean symbolic proof-search, and feedback for iterative reasoning and repair.

The structured memory block stores all Lean-verified claims as embeddings for context-aware retrieval. This guarantees continuity in multi-step proofs: previously validated claims are readily available as hypotheses for future translation/proof steps. The design is fully modular; future advances in translation or proving can be immediately swapped into Hermes without retraining.

Experimental Evaluation: Accuracy, Efficiency, and Scalability

Quantitative Results and Baseline Comparisons

Hermes is rigorously evaluated on four mathematical reasoning benchmarks (MATH500, AIME’25, CollegeMath, HardMath2) with LLMs covering small to state-of-the-art model scales (Qwen3-8B, o3-mini, DeepSeek-V3.1). Strong baselines span zero-shot CoT, majority vote, state-of-the-art reward models (PRMs / ORMs), and existing Lean-based verifiers (Safe, Safe*). The results show that Hermes consistently dominates all baselines in step-critical accuracy, with especially large margins on the most difficult datasets and LLMs.

Notably, on the AIME’25 benchmark with DeepSeek-V3.1, Hermes yields a 67% absolute accuracy improvement, with up to 80% reduction in inference FLOPs and at least 4x lower token usage compared to reward-model selection-based methods.

Figure 3: Token usage per problem: Hermes achieves comparable efficiency to zero-shot CoT while outperforming reward-based BoN methods by 4–6x in token usage.

Figure 4: Average TeraFLOPs per problem: Hermes incurs only marginal additional cost over LLM-only approaches, orders of magnitude less than BoN reward-based selection.

Scaling and BoN Investigation

Analysis of Best-of-N candidate generation (BoN) with growing N reveals that reward-based methods (ORMs/PRMs) can only surpass Hermes’s accuracy for small LLMs if allowed up to 28x more generation—an unreasonable efficiency-accuracy trade-off. For large LLMs, Hermes at single-shot inference outperforms any practical BoN approach.

Figure 5: Scaling BoN: Hermes achieves top accuracy in single-shot mode versus reward model selection at large N, presenting an orders-of-magnitude efficiency advantage.

Ablations and Sensitivity

Systematic ablation demonstrates that both the integrated memory and Lean-based prover module are critical: removal of either reduces accuracy (on AIME, up to a 25% absolute drop). Detailed sweeps over translation and prover budgets ($K_t$, $K_p$) show that relatively moderate settings are sufficient for strong reliability, with diminishing returns past $K_p=8$ or $K_t=4$.

Figure 6: Distribution of Hermes failures with fixed translation budget—translation errors dominate at low $K_t$, but diminish rapidly as the sampling budget increases.

Figure 7: Distribution of Hermes failures with fixed prover budget—unresolved proof failures dominate at low $K_p$.

Qualitative Behavior: Error Correction and Hallucination Mitigation

Hermes’s feedback mechanism directly detects and mitigates hallucinated intermediate steps, guiding model reasoning away from spurious directions—a frequent deficiency in LLM-only CoT generation.

Figure 8: Hermes detects and corrects a spurious logical claim during LLM-generated chain-of-thought, enforcing proof-state discipline.

Implications and Future Prospects

The empirical evidence validates that hybrid, formally-grounded LLM agents can surpass both pure reward-model and verification-after-generation strategies on resource and reliability axes. Unlike post-hoc or scalar-valued reward models, Hermes provides interpretable, step-level feedback, enabling both transparency and online repair. Integration with memory and context retrieval further stabilizes long-horizon, multi-branch proofs.

Practically, Hermes enables deployment of mathematical LLM agents in settings where robustness and correctness are mandatory, including educational automation, mathematical research assistants, and high-confidence symbolic computation. The modular architecture anticipates rapid advances in both translation and Lean4 theorem proving technology.

Theoretically, Hermes connects neuro-symbolic learning with trusted formal verification, setting a precedent for hybrid LLM systems in other scientific and engineering domains where multisource rigorous reasoning is essential. Hermes points towards a framework where interpretability, correctness, and sample efficiency are unified, not opposed, and future research may generalize this architecture to capture broader classes of logical, program, and scientific reasoning.

Conclusion

Hermes establishes a new reference paradigm for tool-augmented, formally verifiable mathematical reasoning in LLMs. By synchronizing interpreter-driven step validation, memory-based context maintenance, and neural generation, Hermes consistently improves accuracy, robustness, and efficiency. The approach demonstrates that close-coupled, agentic integration of LLMs with symbolic proof tools is an effective and scalable solution to the limitations of purely informal or purely formal AI reasoners.