AutoReal-Prover in Industrial Isabelle

• AutoReal-Prover is a 7B fine-tuned Isabelle theorem prover employing LLM-driven, chain-of-thought proof generation for industrial-scale verification.
• It integrates context augmentation by explicitly providing proof-relevant definitions and lemmas, enabling effective handling of long-horizon, context-dependent proofs.
• Empirical validation on seL4 and AFP projects shows success rates around 51-54%, highlighting its potential for secure, on-premise deployment in complex industrial settings.

AutoReal-Prover is a 7B fine-tuned Isabelle theorem prover derived from the broader AutoReal methodology for LLM-driven theorem proving on real-world industrial-scale verification, with seL4-Isabelle as the primary case study and additional evaluation on security-related Isabelle projects from the Archive of Formal Proofs (AFP). It operates in a theorem-plus-context setting: the model receives a target theorem together with theorem-relevant proof context, generates a complete Isabelle proof script along with step-aligned natural-language rationales, and counts as successful only when Isabelle accepts the proof, all subgoals are discharged, and no placeholders such as sorry or oops remain (Zhang et al., 9 Feb 2026).

1. Industrial verification setting

AutoReal-Prover is defined against a markedly different target domain from mathematics-oriented benchmarks such as miniF2F. The motivating environment is the seL4 microkernel verification in Isabelle/HOL, where proofs are context-dependent, long-horizon, interactive and state-sensitive, and maintenance-heavy. The seL4 proof base is reported as about 8.7 KLOC of C, roughly 600 lines of assembly, and around 200,000 lines of Isabelle scripts, and the original verification required about 20 person-years of proof effort. In this setting, theorem proving is not treated as isolated tactic prediction over short mathematical lemmas, but as proof construction inside a large, evolving verification project with strong dependence on local theory context and project-specific proof idioms (Zhang et al., 9 Feb 2026).

The system is also motivated by deployment constraints. Prior work on industrial verification is described as often relying on closed-source models such as GPT-4 or other very large commercial systems, which are problematic for local deployment because of security and confidentiality concerns, API usage costs, and the practical requirement for on-premise, lightweight deployment. AutoReal-Prover addresses this by using a compact 7B model rather than a hundreds-of-billions-parameter hosted system. A common misconception is that the work is primarily a miniature adaptation of benchmark-oriented theorem proving; the paper instead frames it as an attempt to move LLM-driven proving into industrial-scale Isabelle developments (Zhang et al., 9 Feb 2026).

2. AutoReal methodology and proof-generation model

The paper distinguishes between AutoReal and AutoReal-Prover. AutoReal is the methodology: a pipeline comprising proof CoT data construction, CoT-based proof training, context augmentation, and proof generation with formal checking. AutoReal-Prover is the concrete prover obtained by fine-tuning Qwen2.5-Coder-7B under that methodology for industrial Isabelle theorem proving (Zhang et al., 9 Feb 2026).

At inference time, the system takes a target theorem and a proof context consisting of theorem-relevant definitions and lemmas. Unlike stepwise interactive provers, AutoReal-Prover is trained to generate the whole proof in a single inference, together with step-by-step rationales. The paper motivates this as an efficiency decision: repeated theorem-prover interaction can be costly in developments such as seL4. The formal viewpoint is organized around Isabelle subgoals of the form

$$A_1 \Longrightarrow \cdots \Longrightarrow A_n \Longrightarrow C,$$

with proof-state evolution written as

$$G_0 \xrightarrow{a_1} G_1 \xrightarrow{a_2} \cdots \xrightarrow{a_T} G_T,$$

and the structured reasoning trace as

$$(x, G_0) \Rightarrow (z_1, a_1) \Rightarrow \cdots \Rightarrow (z_T, a_T).$$

Here $a_t$ denotes a proof command and $z_t$ the aligned rationale for that command (Zhang et al., 9 Feb 2026).

This architecture implies another important clarification. AutoReal-Prover is not a search-based interactive prover in the style of tactic-by-tactic best-first expansion. Proof verification occurs after generation: a candidate proof script is inserted in place of the original proof and then checked by Isabelle. The system allows up to 256 proof attempts per theorem, and a theorem is considered solved if at least one attempt verifies. The paper does not provide a separate retrieval score, ranking formula, or pass@$k$ estimator; the operational success criterion is simply verifier acceptance within the bounded attempt budget (Zhang et al., 9 Feb 2026).

3. Chain-of-thought proof training

The central methodological novelty is step-level chain-of-thought supervision extracted from real Isabelle proof traces. The source traces come from FVELER, which records per-step proof states before and after each proof command. For each individual proof step, the construction pipeline packages four ingredients: the source lemma, the proof state before applying the step, the proof step itself, and the proof state after applying the step. A model is then prompted to explain the reasoning of that single step in a concise technical paragraph, focusing on methods used, proof-state change, and the role of the step in the larger proof (Zhang et al., 9 Feb 2026).

The resulting dataset contains about 200k step-level CoT instances built from seL4 proofs. These instances are then folded into full-proof instruction-tuning data whose input is the theorem statement plus proof context and whose output is the entire proof sequence interleaved with rationales. The intended effect is twofold: first, the model is exposed to why a command is applicable in a given proof state rather than merely seeing final proof text; second, the resulting prover can emit explanations aligned with the formal derivation (Zhang et al., 9 Feb 2026).

The paper also imposes a strict anti-leakage protocol. Proof steps associated with evaluation theorems are excluded from CoT data construction, so the CoT dataset contains no proof steps from the evaluation targets. This is significant because a superficial reading might suggest that the model is trained directly on the very proofs it is later evaluated against. The paper explicitly rules that out for the benchmark theorems (Zhang et al., 9 Feb 2026).

4. Context augmentation

The second major component is context augmentation. In large Isabelle developments, the theorem statement alone frequently underdetermines the proof strategy because essential information resides in local definitions, refinement lemmas, or project-specific infrastructure. AutoReal therefore supplies the model with a compact proof context consisting of theorem-relevant definitions and lemmas rather than entire transitive theory closures (Zhang et al., 9 Feb 2026).

This context is not automatically retrieved in the current implementation. It is explicitly provided by the verifier, and in evaluation it is kept consistent with the lemmas and definitions used in the corresponding human-written proofs. That design isolates theorem proving from premise-selection noise, but it also means the pipeline is not yet end-to-end automated. A common misconception is that AutoReal-Prover solves industrial Isabelle proving together with premise retrieval; the paper instead evaluates the theorem-plus-context problem, with context discovery left open (Zhang et al., 9 Feb 2026).

The role of context is illustrated by the running example lec_ex_cap_to[wp], whose proof depends on the definition lookup_extra_caps_def and the traversal lemma mapME_set. With that local context available, the model can unfold the function and apply weakest-precondition reasoning. Without it, the paper argues, both humans and LLMs would struggle to recover the proof structure. Context augmentation is therefore not presented as optional prompt engineering, but as a structural requirement for proving in large Isabelle verification projects (Zhang et al., 9 Feb 2026).

5. Empirical performance

The main evaluation covers two settings: seL4 Important Theories and three security-related AFP projects. On seL4, the benchmark contains 660 theorems drawn from one designated theory in each of the 10 seL4 proof categories: access-control, asmrefine, bisim, capDL-api, crefine, drefine, infoflow, invariant-abstract, refine, and sep-capDL. On AFP, the evaluation covers 451 theorems across CRYSTALS-Kyber_Security, RSAPSS, and Elliptic_Curves_Group_Law (Zhang et al., 9 Feb 2026).

Evaluation set	Theorems	Success rate
seL4 Important Theories	660	51.67%
AFP security-related projects	451	53.88%
Syscall_AI.thy ablation, base model	105	0.95%
Syscall_AI.thy ablation, CoT only	105	22.86%
Syscall_AI.thy ablation, context only	105	5.71%
Syscall_AI.thy ablation, full method	105	54.29%

The headline seL4 result is 341/660 = 51.67%. Category-wise, the system reaches 89/123 = 72.36% on capDL-api / KHeap_DP.thy, 57/105 = 54.29% on invariant-abstract / Syscall_AI.thy, 101/233 = 43.35% on infoflow / Noninterference.thy, 13/46 = 28.26% on crefine / Refine_C.thy, 1/1 = 100% on asmrefine / SEL4GraphRefine.thy, and 0/1 = 0% on sep-capDL / AbstractSeparation_SD.thy. The paper compares the overall seL4 result to a previously reported 27.06% from Selene, while noting that the comparison is not fully apples-to-apples because Selene used 340 sampled seL4 theorems, whereas AutoReal-Prover evaluates all theorems in the selected Important Theories (Zhang et al., 9 Feb 2026).

The attempt-budget sensitivity shows that the success rate saturates well before the maximum budget: on seL4 the system reaches 37.12% with 5 attempts, 46.36% with 64 attempts, 50.00% with 128 attempts, and 51.67% with 256 attempts. On AFP, the corresponding success rates are 36.36%, 51.00%, 52.55%, and 53.88%. The ablation on Syscall_AI.thy is particularly diagnostic: the base model solves 1/105 = 0.95%, CoT-based proof training alone reaches 24/105 = 22.86%, context augmentation alone reaches 6/105 = 5.71%, and the full method reaches 57/105 = 54.29%. This makes the complementarity of CoT training and context augmentation one of the clearest empirical claims in the paper (Zhang et al., 9 Feb 2026).

6. Generated proofs and explanation fidelity

AutoReal-Prover is not evaluated only by theorem-level success. The paper also analyzes the structure of the successful proofs and the quality of the aligned rationales. Among the 341 successfully proved seL4 theorems, 93.55% of generated proofs differ from the original human-written proofs, while the average number of applied methods per proof is similar: 6.018 for generated proofs versus 6.123 for human proofs. The system therefore usually does not reproduce the original script verbatim; it often constructs an alternative proof with comparable apparent tactical complexity (Zhang et al., 9 Feb 2026).

The qualitative examples reinforce that point. For thread_set_has_no_reply_cap, the generated proof unfolds has_reply_cap_def, explicitly applies hoare_pre, and then uses wpsimp with general Hoare/WP lemmas, whereas the human proof starts with clarsimp and uses the more specialized lemma thread_set_cte_wp_at_trivial. Both scripts verify, but they expose different decompositions of the same obligation. This supports the interpretation that the model is synthesizing Isabelle proofs rather than merely replaying memorized tactic sequences (Zhang et al., 9 Feb 2026).

The rationales were manually analyzed over 670 proof steps and aligned rationales from the successful seL4 proofs. The major source of explanation error is the terminal command done: 84 done rationales are reported as irrelevant or incorrect. Outside done, only 10 additional inaccurate rationales are observed, yielding a reported non-done rationale inaccuracy rate of 1.49%. The remaining errors involve minor mismatches for by simp, overly generic explanations for apply auto, and spurious references under apply wpsimp+. This matters because it shows that explanation generation is largely accurate for substantive proof steps, but not uniformly reliable for semantically underspecified terminal automation (Zhang et al., 9 Feb 2026).

7. Relation to surrounding automated proving and open issues

AutoReal-Prover occupies a distinct design point within recent automated proving systems. In Isabelle/HOL, AUTO2 is an embedded saturation-based heuristic prover implemented as an Isabelle tactic, with best-first search over derivable propositions and user-specified proof steps rather than LLM-based proof generation (Zhan, 2016). In Lean 4, REAL-Prover is a retrieval-augmented stepwise theorem prover with best-first search over proof states and semantic premise selection over Mathlib, reporting 23.7% on ProofNet and 56.7% on FATE-M (Shen et al., 27 May 2025). ImProver addresses a different task—automated proof optimization—by rewriting existing Lean proofs to improve criteria such as length or readability rather than proving industrial Isabelle theorems from scratch (Ahuja et al., 2024). In Rust verification, AutoVerus is a verifier-in-the-loop proof-synthesis system for Rust+Verus that proves 137 of 150 tasks (91.3%) by staged generation, refinement, and debugging (Yang et al., 2024). In Coq, QEDCartographer guides search with a learned state-value heuristic and proves 21.4% of the CoqGym test set (Sanchez-Stern et al., 2024). Against that background, AutoReal-Prover is characterized by industrial Isabelle/seL4 evaluation, whole-proof generation in a single inference, step-aligned CoT training, and explicit proof context supplied by the verifier rather than interactive state-space search (Zhang et al., 9 Feb 2026).

Two misconceptions are particularly important. First, AutoReal-Prover is not an end-to-end autonomous industrial prover, because theorem-relevant proof context is not automatically extracted in the current implementation. Second, it is not an interactive stepwise system: the proof is generated as a whole and only then checked by Isabelle. These choices help explain both the strengths and the current limits of the method (Zhang et al., 9 Feb 2026).

The open problems are correspondingly concrete. The paper identifies automating theorem-relevant proof-context extraction as a major future direction, because the current theorem-plus-context setting leaves premise discovery outside the automated pipeline. It also leaves substantial headroom in proof coverage, with success rates of 51.67% on seL4 and 53.88% on AFP. Finally, while the model is argued to support lightweight local deployment because it is a compact 7B model, the paper does not provide detailed runtime, latency, or memory measurements. A plausible implication is that AutoReal-Prover presently demonstrates that industrial Isabelle proof generation is feasible under explicit context supply, but not yet that industrial-scale proof maintenance has become fully automatic (Zhang et al., 9 Feb 2026).