- The paper presents a novel two-phase protocol that integrates deterministic verifier feedback through specialized agents to enhance zero-shot theorem proving in Lean 4.
- It achieves higher solve rates and improved sample efficiency by utilizing structured signals such as syntax errors, type mismatches, and goal progress during proof search.
- Experimental results on miniF2F and CombiBench benchmarks show that VERITAS outperforms traditional sampling methods, effectively solving complex multi-step theorems.
Introduction
The paper "VERITAS: Verifier-Guided Proof Search for Zero-Shot Formal Theorem Proving" (2606.19399) introduces VERITAS, a zero-shot automated theorem prover for Lean 4 that tightly integrates verifier feedback signals—syntax errors, type mismatches, partial goal progress, and proof completion—into all stages of proof search. Existing LLM-based formal provers typically reduce rich verifier outputs to a binary scalar, collapsing away intermediate feedback that could improve search efficiency and strategy, especially for multi-step theorems with dependent tactics. VERITAS advances the field by routing these structured signals throughout a two-phase protocol, achieving higher solve rates than traditional sampling approaches and offering a robust testbed for verifier-guided search.
VERITAS Framework
VERITAS employs four specialized agents: Strategist, Tactician, Critic, and Retriever. Proof search is orchestrated through a shared proof state with Lean's structured verification signals. The search protocol consists of two distinct phases:
Phase 1 (Best-of-N Sampling): Direct sampling of N tactics using Claude Sonnet followed by batched Lean validation. This efficiently solves theorems amenable to flat sampling.
Phase 2 (Critic-Guided MCTS): For theorems unsolved in Phase 1, VERITAS activates a Critic-guided MCTS loop, injecting prior tactic failures and Lean error messages as negative examples to the Tactician. The Critic (Claude Haiku) replaces random rollouts with a value estimate integrating four-way Lean signals, and search is biased toward strategy-relevant subspaces defined by the Strategist. The Retriever fetches relevant Mathlib premises based on weighted keyword overlap.
The design enforces an internal monotonicity guarantee: all Phase 1 solves are preserved and any additional solves in Phase 2 are attributable to explicit feedback-driven exploration, not stochastic overlap.
Experimental Results
VERITAS is evaluated on two Lean 4 benchmarks: miniF2F (244 Olympiad-level theorems) and VERITAS-CombiBench (55 combinatorics theorems, released as a new benchmark). Methodological baselines include heuristic portfolios, flat Best-of-1/-5 sampling with Claude, and ablated variants (VERITAS-Heuristic, VERITAS-MCTSonly).
Main Metrics:
- miniF2F: VERITAS achieves 40.6% solve rate, outperforming Best-of-5 (36.9%), Best-of-1 (29.1%), and Portfolio (26.2%).
- VERITAS-CombiBench: VERITAS reaches 7.3% versus Best-of-5 (1.8%) and Portfolio (3.6%), revealing that unguided sampling is counterproductive when lemma names need repair based on verifier feedback.
Phase-wise decomposition indicates VERITAS uniquely solves 11 miniF2F theorems that are unreachable by the Best-of-5 baseline, mainly multi-step problems requiring iterative correction informed by Lean’s feedback. VERITAS also demonstrates sample efficiency, converting ≈6,500 Lean calls (27/theorem) into the highest solve rate at comparable API cost, while flat sampling requires substantially more calls to match performance.
Failure analysis on the 244 theorems shows that most non-solves manifest as syntax errors (32%), type mismatches (17%), or partial goal progress (10%)—all cases where verifier signals delineate actionable state transitions for further agent development.
Design Principles and Implications
The key principle evidenced is that deterministic verifier signals should be routed into generation and search decisions, not collapsed into terminal reward functions. The structured signal flow amplifies LLM-based tactic generation without requiring fine-tuned trace models. VERITAS’s architecture clarifies vital distinctions:
- Synergy between LLM and Search: Neither deep search alone nor increased LLM sampling achieves comparable success; the integration is essential for multi-step theorems, reflecting human proof strategy.
- Monotonicity Guarantee: The two-phase protocol isolates easy solves up front, preserving sample efficiency, and exposes the residual hard set for intensive MCTS, ensuring rigorous attribution.
- Sample Efficiency: VERITAS dominates the inference-only compute frontier, yielding higher solve rates per unit cost and Lean calls.
VERITAS advances over prior art by leveraging intermediate verifier signals at inference, as opposed to trace-based training. Comparable systems (e.g., GPT-f [Polu & Sutskever, 2020], PACT [Han et al., 2022], ReProver [Yang et al., 2023]) focus on tactic proposal and retrieval, but do not exploit intermediate verifier feedback in real-time. Recent reinforcement learning and search-based provers (HyperTree [Lample et al., 2022], Thor [Jiang et al., 2022], DeepSeek-Prover-V1.5 [Xin et al., 2025], COPRA [Thakur et al., 2024]) utilize proof assistant feedback, but VERITAS’s explicit role decomposition and critic-guided state expansion differentiates its approach.
The methodology aligns with emerging trends in rich process supervision (SDPO [Hübotter et al., 2026], Reflexion [Shinn et al., 2023]), but operates purely at inference time and with deterministic feedback rather than learned opinions or trace distillation.
Theoretical and Practical Impact
The study demonstrates that structured intermediate feedback significantly boosts inference-time formal reasoning, especially in multi-step, dependent-tactic settings. The release of CombiBench offers a new evaluation suite tailored to combinatorics, furthering reproducibility and benchmarking rigor.
Practically, VERITAS suggests that interactive theorem provers should expose rich, structured feedback to downstream agents, maximizing the utility of proof assistant signals. Theoretically, the design motivates future research in agent orchestration, adaptive retrievers, and learned critics capable of exploiting partial progress signals. The principle is generalizable to other domains—SMT unsat cores, Coq/Agda goal states, symbolic execution traces, and model-checking counterexamples—where verifier feedback can drive generation and search.
Future Directions
Potential extensions include training adaptive retrievers to reduce syntax hallucinations, enhancing critic value estimates via trace distillation, and scaling the two-phase protocol for larger or more creative theorem sets. The integration of feedback-driven search with interactive or hybrid symbolic-neural provers could substantially increase the reach and robustness of LLM-based methods. VERITAS provides a formal baseline for verifier-in-the-loop search design and sets a clear direction for future exploration in zero-shot and inference-driven formal reasoning.
Conclusion
VERITAS establishes that deterministic verifier signals routed through specialized agents and structured search protocols can substantially improve zero-shot automated theorem proving. The framework not only advances sample efficiency and solve rates over existing baselines, but also clarifies the role of interactive feedback in proof search. These results provide a foundation for scalable inference-time theorem provers and delineate a trajectory for future integration of structured signal routing in formal reasoning agents, with broader applicability to symbolic verification and mathematical AI research.
