Modular Hybrid & Sketch-Based Proof Synthesis

• Modular hybrid and sketch-based proof synthesis is a novel approach that integrates high-level proof planning with low-level tactic refinement using LLMs and trusted checkers.
• It employs a domain-specific language to extract and refine proof sketches, enabling efficient local repairs and modular reuse of proof components.
• Experimental benchmarks demonstrate state-of-the-art success rates on systems like Isabelle and Lean, highlighting robust verification and improved scalability.

Modular hybrid and sketch-based proof synthesis encompasses new approaches to automated theorem proving that integrate LLM–driven proof generation, explicit intermediate proof representations ("sketches"), and lightweight or classical proof checkers. By decomposing the proof process into higher-level structural planning and lower-level tactic or inference step refinement, these systems increase success rates and verification trust, and address limitations inherent to both monolithic and serial LLM proof synthesis. The central innovation lies in representing proofs as modular sketches in a domain-specific language (DSL), which serve as interfaces between LLMs and trusted checking/refinement modules. This synthesis paradigm has led to state-of-the-art performance in major formal verification benchmarks for systems such as Isabelle and Lean (Hu et al., 21 May 2025, Kommuru et al., 7 Apr 2026).

1. Hybrid Architectures for Proof Synthesis

Hybrid proof synthesis frameworks interleave two complementary LLM-based strategies: whole-proof generation and tactic-level, stepwise generation. HybridProver (Hu et al., 21 May 2025) formalizes this pipeline by first using an LLM to propose entire proof candidates in the native language of the target theorem prover (Isar/apply scripts for Isabelle), then extracting proof sketches by systematically replacing concrete tactic applications with placeholders (e.g., “sorry”). These incomplete sketches preserve global proof structure and subgoal hierarchies, ready for further stepwise tactic refinement by a second LLM, which is specialized for local subgoal completion and coordinated with external tools like Sledgehammer.

ProofSketcher (Kommuru et al., 7 Apr 2026) generalizes this principle to a broader workflow: LLMs generate typed proof sketches in a compact custom DSL, expressing major proof moves (e.g., induction, split, rewrite) and explicit subgoals; a small trusted kernel then parses the sketch, extracts proof obligations as sequents, and attempts discharge natively or via external solvers whose certificates can be checked by minimal trusted code. A local repair mechanism iteratively edits only the parts of the sketch corresponding to failed obligations, leveraging modularity for efficient reuse and correction.

2. Proof Sketch Extraction and DSL Representation

Proof sketches serve as modular, partially specified proof objects that bridge global reasoning and local refinement. In HybridProver, a proof sketch is defined as a tree structure where every tactic node is replaced with a “sorry” placeholder. Formally, starting from a proof tree $P = (N, E, root, label)$ with tactic and subgoal nodes, its sketch $S(P)$ is $(N, E, root, \ell')$ where $\ell'(n) = \text{tactic}(\texttt{"sorry"})$ if $\ell(n) = \text{tactic}(t)$, else $\ell(n)$. This operation can be implemented by traversing the proof AST and substituting all tactics.

ProofSketcher's DSL occupies the spectrum between text and tactics: each sketch node contains (i) a goal formula, (ii) a method tag (e.g., $\texttt{rewrite}$, $\texttt{split}$, $\texttt{induction}$), (iii) zero or more references (lemmas, hypotheses), and (iv) typed subgoal holes ($h:\psi$). Consistency is enforced via typing/well-formedness judgments at both hole and node level. The DSL enables structured extraction of sequents to be discharged, and allows fine-grained local repair.

DSL Grammar Example

$S(P)$0

where each hole $S(P)$1 must be well-typed in the parent context.

3. Kernel and Refinement Engines

The trusted kernel or checker constitutes the core of modular hybrid pipelines, responsible for parsing proof sketches, extracting obligations, and verifying or refining steps with strong guarantees. ProofSketcher’s kernel comprises: (i) a parser for the sketch DSL, (ii) a deterministic obligation extractor generating sequents from methods (e.g., splitting $S(P)$2 yields two subcases), and (iii) a prover engine capable of both native (small-step) and certificate-gated external discharge. Only the kernel and certificate checker need to be trusted, concentrating the verification TCB.

In HybridProver, the refinement stage parses sketches with “sorry” placeholders, instantiates subgoals, and generates candidate tactics via tactic-LM and Sledgehammer. It iteratively substitutes “sorry” by applying the first tactic whose application passes Isabelle’s internal checker. This process is modular over the subgoals and can flexibly leverage ATPs, custom automation, or further LLM-inference depending on domain and goal complexity.

4. Modularity, Caching, and Reuse

Modularity is central: each sketch node or tactic hole can be treated as an independent module, uniquely identified by its (goal, tag, refs, holes) signature (commonly via a content hash). In ProofSketcher, proof cacheing is at the node level; incremental edits or LLM corrections only trigger obligation extraction/discharge for affected nodes. This enables efficient local repair strategies, wherein failures (such as a missing lemma or a bad rewrite instantiation) lead to LLM editing solely of failing subtrees—substantially reducing redundant proof search.

Sketched modules can be parameterized (e.g., with induction variable, lemma selection, case predicate), composed as first-class macros, and mixed with retrieval hints per node. This architecture supports reuse, compositionality, and adaptation to diverse proof libraries or external engines.

5. Integration with Automated Reasoning Tools

Hybrid pipelines maximize the capabilities of both LLMs and classical theorem proving by integrating ATPs and solvers at targeted points. In HybridProver, Sledgehammer is invoked with each subgoal (fixed 30s ATP timeout), and standard Isabelle tactics (“simp,” “auto,” “blast,” “presburger,” etc.) are available for tactic discovery. The refinement loop greedily selects the first tactic (whether from LLM or ATP) that yields proof object acceptance. In ProofSketcher, external solvers are certificate-gated and only accepted steps are incorporated in the final proof object.

Obligation granularity (how fine/sketched individual subgoals are) directly impacts the tractability and scalability of this integration. Too coarse a granularity yields large solver obligations and increased certificate overhead, while too fine transfers greater burden to LLMs—a subject of ongoing empirical optimization.

6. Experimental Benchmarks and Ablation Results

HybridProver achieves state-of-the-art performance on the Isabelle/miniF2F benchmark: 59.4% success rate at $S(P)$3 sampling, surpassing previous SOTA (56.1%) (Hu et al., 21 May 2025). Ablation studies reveal that neither full-proof nor tactic-only LLM yields more than 38% on its own; the sketch-extraction and refinement pipeline adds approximately 18 percentage points, with a further 3 points accruing from full system integration. Sampling temperature ($S(P)$4 in 0.6–0.9) and rate (@128) are critical. Data quality and learning rate profoundly affect results; LoRA rank has negligible impact.

ProofSketcher reports kernel-accepted proof rates of 92.21% (miniF2F-test), 58.25% (LeanDojo-test), and 44.62% (ProofNet-test), consistently outperforming prior baselines (Kommuru et al., 7 Apr 2026). Mean LLM calls per theorem are close to 1.3, as most theorems require only one or two LLM edits before verification. The bulk of runtime is spent in external certificate checking, not in the kernel proper.

System	miniF2F (%)	LeanDojo (%)	ProofNet (%)
ProofSketcher	92.21	58.25	44.62
DeepSeek-Prover-V2	88.93	—	37.10
ReProver	—	51.20	—

7. Strengths, Limitations, and Research Directions

The modular hybrid/sketch paradigm offers strong soundness (all proofs are kernel-checked with certificate validation for external steps), efficient local repair (failures isolated per node), and enables scalable proof engineering via caching and node-level modularity. Most failures arise from missing lemma retrieval or bad LLM instantiations, which are efficiently localized.

Trade-offs include overhead in certificate checking for large obligations, dependency on quality retrieval mechanisms, and careful management of sketch granularity. Future advances may include dynamic granularity control per node/domain, optimized certificate formats (e.g., smaller LFSC-style proofs), tight coupling between LLM generation and kernel verification, and domain-specific drivers (e.g., geometry engines for mathematical domains). These directions seek to further reduce proof search iteration depth, strengthen guarantees, and extend applicability to more expressive logics and larger proof corpora (Hu et al., 21 May 2025, Kommuru et al., 7 Apr 2026).