Why Agentic Theorem Prover Works: A Statistical Provability Theory of Mathematical Reasoning Models

Abstract: Agentic theorem provers -- pipelines that couple a mathematical reasoning model with library retrieval, subgoal-decomposition/search planner, and a proof assistant verifier -- have recently achieved striking empirical success, yet it remains unclear which components drive performance and why such systems work at all despite classical hardness of proof search. We propose a distributional viewpoint and introduce statistical provability, defined as the finite-horizon success probability of reaching a verified proof, averaged over an instance distribution, and formalize modern theorem-proving pipelines as time-bounded MDPs. Exploiting Bellman structure, we prove existence of optimal policies under mild regularity, derive provability certificates via sub-/super-solution inequalities, and bound the performance gap of score-guided planning (greedy/top-(k)/beam/rollouts) in terms of approximation error, sequential statistical complexity, representation geometry (metric entropy/doubling structure), and action-gap margin tails. Together, our theory provides a principled, component-sensitive explanation of when and why agentic theorem provers succeed on biased real-world problem distributions, while clarifying limitations in worst-case or adversarial regimes.

• The paper formalizes agentic theorem proving as a finite-horizon reachability MDP over measure-valued proof states and proves that optimal deterministic Markov policies exist under compactness and continuity assumptions.
• The paper develops sub- and super-solution certificates plus score-guided planning bounds, including worst-case regret of at most 2∑ε_b and faster expected rates under favorable proof-state margin conditions.
• The paper explains practical success without overturning complexity theory: structured problem distributions, short effective proofs, low-dimensional state geometry, strong verifier feedback, and useful retrieval reduce statistical difficulty, while its assumptions remain empirically unvalidated.

Overview and motivation

Agentic theorem provers—pipelines coupling an LLM policy with library retrieval, subgoal decomposition/search planning, and a proof-assistant verifier—have achieved substantial empirical success on formal mathematics benchmarks. The paper under review addresses a conceptual tension: proof search is classically hard (NP-completeness of 3-SAT, undecidability of first-order logic), yet stochastic generators with search loops succeed in practice. The authors' resolution is not to dispute complexity-theoretic lower bounds but to shift the analytic object: real theorem instances are drawn from highly structured, biased distributions concentrated on repetitive idioms, canonical proof patterns, and library lemmas, and success should be measured distributionally rather than worst-case.

The central contribution is the notion of statistical provability: the finite-horizon success probability of reaching a verified proof, averaged over an instance distribution $Q$. The paper formalizes agentic provers as time-bounded reachability MDPs and develops Bellman-structure tools—existence results, sub-/super-solution certificates, and regret bounds for score-guided planning—that make explicit how each pipeline component (verifier feedback, retrieval, test-time search, representation geometry) affects success probability (2602.10538).

Formalization as a finite-horizon reachability MDP

The state space is built from a measure-valued encoding of proof states. Syntactic goals are embedded via $\phi:\mathcal G_0\to K$ into a compact metric space $(K,d_K)$; a proof state containing goals $g_1,\dots,g_n$ is represented by the empirical measure $\mu(x)=\sum_i\delta_{\phi(g_i)}$, so the solved state is the zero measure and goal count is total mass. Truncating mass at $W$ yields the compact state space $\mathcal X = M_{\le W}(K)$ under the bounded-Lipschitz metric—a compactness result proved via Arzelà–Ascoli and Prokhorov arguments. This encoding is permutation-invariant and supports variable goal counts naturally.

Provability is then defined as the optimal time-bounded reachability probability,

$$V_B^*(\mu) := \sup_\pi \Pr^\pi_\mu(\exists t \le B:\ \mu_t \in G),$$

with the distributional counterpart $\mathbb E_{\mu_0\sim Q}[V_B^*(\mu_0)]$. This is precisely the object agentic systems implicitly optimize: success probability on a biased problem stream under fixed compute, not worst-case solvability.

Existence of optimal policies and provability certificates

Under mild regularity—compactness of $\mathcal X$ and action space $\phi:\mathcal G_0\to K$0, a Feller-type weak-continuity assumption on the transition kernel, and closedness of the goal set—the paper proves by backward induction (Portmanteau theorem plus measurable selection) that each finite-horizon value $\phi:\mathcal G_0\to K$1 is Borel-measurable, suprema over actions are attained, and an optimal deterministic Markov policy exists.

More practically significant is the certificate machinery. Using monotonicity of the Bellman operator $\phi:\mathcal G_0\to K$2, any super-solution sequence $\phi:\mathcal G_0\to K$3 with $\phi:\mathcal G_0\to K$4 upper-bounds $\phi:\mathcal G_0\to K$5, and any sub-solution sequence lower-bounds it. This yields a provability certificate interval $\phi:\mathcal G_0\to K$6 whose gap quantifies residual uncertainty about provability given approximate surrogates—an evaluation tool that does not require solving the MDP.

Regret bounds for score-guided planning

The reasoning model is modeled as producing score functions $\phi:\mathcal G_0\to K$7 approximating the one-step optimal value $\phi:\mathcal G_0\to K$8, driving greedy, top-$\phi:\mathcal G_0\to K$9, beam, or rollout planners. Under uniform approximation error $(K,d_K)$0 on a relevant domain $(K,d_K)$1, greedy selection incurs per-step regret at most $(K,d_K)$2, giving the worst-case bound

$(K,d_K)$3

with certificates obtainable directly from clipped score bounds. The more interesting result is a fast rate under a margin condition: if the occupancy distribution $(K,d_K)$4 over depth-$(K,d_K)$5 states satisfies a Mammen–Tsybakov-style tail bound $(K,d_K)$6 on the $(K,d_K)$7-gap, then top-$(K,d_K)$8 and beam policies achieve expected regret $(K,d_K)$9. The mechanism is transparent: mis-ranking requires the gap to be smaller than twice the score error, which is rare under the margin tail. When margins fail ($g_1,\dots,g_n$0) or exact ties carry positive mass, fast rates are unavailable from plug-in ranking alone, and the analysis must shift to set-coverage arguments—an honest delineation of where ranking-based selection is intrinsically fragile.

High-probability control of approximation error

The paper supplies sufficient conditions for the uniform approximation assumption to hold with high probability, covering offline pretraining on expert traces, verifier-in-the-loop rollouts, and adaptive beam-search data collection. Two complementary routes are developed:

• Adaptive uniform deviation: a covering-number argument combined with Azuma–Hoeffding controls martingale-difference deviations for non-i.i.d., adaptively collected data.
• Net-coverage estimation: assuming doubling dimension $g_1,\dots,g_n$1 for the reachable domain and Lipschitz hypothesis classes, Monte Carlo rollout labels (solve-or-not within remaining budget) yield the rate

$g_1,\dots,g_n$2

up to polylog factors. Here geometry enters directly: small doubling dimension accelerates statistical convergence, while retrieval-augmented pretraining reduces the approximation term $g_1,\dots,g_n$3 by exposing relevant lemmas and proof schemas.

Variable goal counts are handled compactness-preservingly: if overflow beyond mass $g_1,\dots,g_n$4 occurs with probability at most $g_1,\dots,g_n$5, all bounds transfer to the untruncated system with additive $g_1,\dots,g_n$6 corrections.

Representation learning via certificate-gap minimization

A distinctive methodological proposal treats the embedding $g_1,\dots,g_n$7 itself as a decision variable. Within a restricted function family, tightest sub-/super-solutions define optimal certificates $g_1,\dots,g_n$8 and $g_1,\dots,g_n$9, and representations are trained to minimize the expected certificate gap plus a complexity penalty on doubling dimension, Lipschitz constants, and distortion. This makes explicit the tradeoff that better representations tighten certificates but overly complex geometries loosen them statistically. Notably, this objective is proposed rather than validated empirically; its tractability and practical benefit remain open.

Easy versus hard instances

The framework distills instance difficulty into five interpretable parameters: effective proof length $\mu(x)=\sum_i\delta_{\phi(g_i)}$0, approximation error, statistical error governed by geometric complexity, margin parameters $\mu(x)=\sum_i\delta_{\phi(g_i)}$1, and overflow tails. A schematic lower bound reads:

$\mu(x)=\sum_i\delta_{\phi(g_i)}$2

Instances are easy when proofs are short, margins large, reachable-state geometry low-dimensional, and verifier feedback prunes aggressively; they are hard when proofs are long, near-ties abound, covering numbers explode, or the domain is out-of-distribution. Crucially, computationally hard subproblems embedded in a proof inflate $\mu(x)=\sum_i\delta_{\phi(g_i)}$3 and flatten margins, so classical hardness re-emerges through these parameters rather than being contradicted. This is the paper's answer to its motivating paradox: no transcendence of complexity theory occurs; biased distributions simply place typical instances in favorable regimes of these parameters.

Limitations and open questions

Several caveats deserve emphasis. First, the theory is qualitative-to-semi-quantitative: the headline bounds involve constants and schematic terms ("beam/top-$\mu(x)=\sum_i\delta_{\phi(g_i)}$4 bias terms") that are not instantiated against concrete systems or benchmarks; no experimental validation of the predicted parameter sensitivities is provided. Second, the margin condition is assumed rather than measured—whether real proof-state occupancies exhibit polynomial gap tails with nontrivial $\mu(x)=\sum_i\delta_{\phi(g_i)}$5 is an open empirical question. Third, the high-probability estimation results rely on coverage assumptions (doubling dimension, net access with conditionally unbiased rollout labels) that may fail when reachable domains are poorly covered by training data, exactly the OOD regime the framework identifies as hard. Fourth, the certificate-gap minimization objective is defined but not shown to be efficiently optimizable. Finally, the bridge between classical complexity barriers and statistical complexity remains partial: the paper positions its parameters as the right vocabulary but does not prove formal correspondences between, say, proof-theoretic cut structure and margin behavior.

Conclusion

This paper supplies a principled, component-sensitive account of why agentic theorem provers succeed: modeling proving as finite-horizon reachability over measure-valued states, it derives existence of optimal Markov policies, sub-/super-solution provability certificates, worst-case and margin-accelerated regret bounds for score-guided planning, and geometrically controlled estimation rates. The resulting theory explains empirical effectiveness on biased, structured problem distributions without contradicting classical hardness, and identifies verifiable quantities—effective horizon, approximation error, doubling dimension, margin exponents, overflow tails—through which design choices act. Its main open problems are empirical calibration of the margin and geometry assumptions and practical realization of certificate-driven representation learning.