AutoReal: LLM-Driven Industrial Theorem Prover

• AutoReal is an LLM-driven theorem proving method that leverages chain-of-thought training and context augmentation to tackle complex, context-rich industrial verification tasks.
• It employs a four-stage pipeline—proof trace extraction, CoT-based training, context augmentation, and multi-attempt proof evaluation—to maximize proof success rates in Isabelle.
• AutoReal-Prover, a compact 7B-scale model, achieves over 50% success on seL4 and AFP benchmarks, highlighting its potential for practical, open-source verification.

Searching arXiv for papers on AutoReal and related seL4 theorem proving. AutoReal is an LLM-driven theorem proving method for real-world industrial-scale systems with support for lightweight local deployment. Introduced by Zhang et al. in the context of the seL4-Isabelle verification project, it combines two core components—chain-of-thought (CoT)-based proof training and context augmentation—to address large, context-rich proof obligations that are costly to discharge with conventional formal-methods workflows. On this basis, the authors fine-tune a compact 7B-scale model, AutoReal-Prover, and report a 51.67% proof success rate on 660 theorems from seL4-designated Important Theories across all 10 seL4 proof categories, as well as 53.88% on 451 theorems from three security-related Archive of Formal Proofs developments (Zhang et al., 9 Feb 2026).

1. Problem setting and design objective

AutoReal is framed around a central constraint of industrial formal verification: formal methods are reliable but costly to apply, often requiring years of expert effort in industrial-scale projects such as seL4, especially for theorem proving. The method is positioned against a literature in which most prior work emphasizes mathematics-oriented benchmarks such as miniF2F, while the few studies targeting industrial-scale verification mostly rely on closed-source models with hundreds of billions of parameters. AutoReal instead targets real-world verification with a compact, open-source, locally deployable model architecture (Zhang et al., 9 Feb 2026).

The representative case study is the seL4-Isabelle verification project. In this setting, proof obligations are not only numerous but also highly context dependent: even simple theorems may depend on definitions and auxiliary lemmas scattered across many theory files. AutoReal therefore treats industrial theorem proving not as isolated next-step prediction but as a task requiring explicit reasoning traces and theorem-specific contextualization.

A plausible implication is that AutoReal’s contribution is less about introducing a new proof assistant backend than about redefining the data interface between LLMs and existing proof developments. The method operates within Isabelle acceptance criteria: a proof is successful only if Isabelle accepts it without errors or placeholder commands.

2. Framework and end-to-end pipeline

The AutoReal framework is presented as a four-stage pipeline. First, proof CoT data construction extracts step-level proof traces from existing seL4 Isabelle proofs via FVELER, with each trace recording the proof state before and after a single proof command. A CoT generation model, Qwen2.5-Coder-7B, then produces a concise natural-language explanation for each step, yielding approximately 200,000 (step, rationale) pairs. Second, CoT-based proof training instruction-tunes a 7B base model on inputs consisting of a lemma and relevant context, and outputs a single interleaved sequence of proof commands and rationales. Third, context augmentation pairs each target theorem with exactly those definitions and auxiliary lemmas that appear in the human-written proof. Fourth, proof evaluation attempts up to 256 proof generations per theorem (Zhang et al., 9 Feb 2026).

The interleaved training target has the form:

apply a₁ rationale z₁ apply a₂ rationale z₂ … done

This formulation is intended to internalize proof-state evolution during training rather than through repeated proof-checker queries at generation time. In the paper’s formal description, for lemma context $x$ inducing an initial goal state $G_0$ and proof commands $A=(a_1,\dots,a_T)$, the intermediate chain $Z=(z_1,\dots,z_T)$ contains step-aligned natural-language justifications, and the training objective can be understood as maximizing the log-likelihood of the interleaved sequence:

$$\max_\theta \sum_{(x,G_0)\to(z_1,a_1)\dots(z_T,a_T)} \log p_\theta\bigl([z_1,a_1,\dots,z_T,a_T]\mid x\bigr).$$

This setup suggests that AutoReal treats rationale generation not as an auxiliary explanatory layer but as part of the learned proof trajectory itself.

3. Chain-of-thought proof training

The first of AutoReal’s two principal innovations is CoT-based proof training. The key idea is to supervise the model with explicit, step-aligned natural-language explanations of why each proof command transforms one proof state into the next. Rather than repeatedly querying Isabelle during generation to recover intermediate states, the model is trained to internalize the progression of the proof script and its local justification structure (Zhang et al., 9 Feb 2026).

The training data are constructed only from proofs outside the evaluation theorems. Each instance combines the lemma statement and its supplied proof context, the proof step as an Isabelle command, and the CoT rationale generated by the CoT model. Training is cast as instruction tuning: the prompt asks the model to emit the full proof interleaved with explanations.

The empirical role of CoT supervision is made explicit in the ablation study on the Syscall_AI.thy subset of 105 theorems. Starting from Base Qwen2.5-Coder-7B at 0.95% success (1/105), adding CoT-based proof training only yields 22.86% (24/105), while context augmentation only yields 5.71% (6/105). Full AutoReal, with both components, reaches 54.29% (57/105). The result is presented as evidence that CoT supervision and context augmentation are complementary rather than interchangeable.

A further qualitative claim concerns the fidelity of generated rationales. Except for underspecified uses of done, fewer than 1.5% of rationales outside done are reported as inaccurate. This suggests that the rationale channel is not merely decorative; it is aligned closely enough with proof behavior to support inspection and explanation.

4. Context augmentation and model instantiation

The second principal innovation is targeted context augmentation. In seL4, entire theory files would introduce substantial token overhead and much irrelevant material. AutoReal therefore supplies only the subset of definitions and auxiliary lemmas needed to understand and discharge a particular theorem, mirroring how human verifiers work. This is explicitly described as avoiding the token-bloat of entire theory files and focusing the model’s attention on semantically essential context (Zhang et al., 9 Feb 2026).

Based on this methodology, the authors fine-tune Qwen2.5-Coder-7B to obtain AutoReal-Prover, described as a compact 7B-scale prover for industrial-scale theorem proving. The Appendix E hyperparameters are:

• Precision: bfloat16
• Max sequence length: 8192 tokens
• Effective batch size: 32 (1 per-device, grad-accum 32)
• Epochs: 10
• Optimizer: paged 8-bit AdamW, [weight decay](https://www.emergentmind.com/topics/weight-decay) = 0.05
• Learning rate: 5×10⁻⁶, cosine schedule with warmup ratio 0.03
• Grad-norm clip: 0.3

No separate loss term is introduced for proof script versus rationale. The model is trained to maximize the likelihood of the interleaved sequence as a single output object. This design choice implies a unified generative view of proof construction and proof explanation.

A notable operational limitation is that theorem-relevant context is currently manually aligned with the human proofs. The paper identifies automation of this extraction as a future direction, which indicates that context selection remains a substantive part of the system’s effectiveness.

5. Evaluation on seL4 and AFP developments

To reflect real-world usage, the seL4 evaluation avoids cherry-picking. The authors follow the official seL4 repository’s ten proof categories and, for each, take the first Important Theory listed in the README, yielding 660 theorems across access-control, asmrefine, bisim, capDL-api, crefine, drefine, infoflow, invariant-abstract, refine, and sep-capDL. Each theorem is attempted up to 256 times under a uniform prompt template. The overall success rate is reported as $\frac{341}{660}\times 100\% = 51.67\%$ (Zhang et al., 9 Feb 2026).

Performance varies across categories. The reported range is from 0% in sep-capDL (0/1) to 100% in asmrefine (1/1), with 43.35% in infoflow (101/233) and 72.36% in capDL-api (89/123). The paper contrasts this with Selene (GPT-4), which reported 27.06% over a sampled set of 340 seL4 theorems.

Generalization is evaluated on three security-related Archive of Formal Proofs developments: CRYSTALS-Kyber_Security (64 theorems), RSAPSS (278 theorems), and Elliptic_Curves_Group_Law (109 theorems). With theorem dependencies mirroring the human scripts and the same inference protocol of up to 256 attempts, AutoReal-Prover achieves $\frac{243}{451}\times 100\% = 53.88\%$, with project-level performance of 50% in Kyber, 59.6% in RSAPSS, and 43.1% in EC group law.

Evaluation setting	Theorems	Success rate
seL4 Important Theories	660	51.67%
AFP security projects	451	53.88%
Syscall_AI.thy full AutoReal ablation	105	54.29%

These figures indicate that the method is not confined to a single verification subdomain. A plausible implication is that the combination of step-aligned proof supervision and tightly scoped context transfer is portable across systems proofs, cryptographic arguments, and mathematical developments, provided the relevant local context can be supplied.

6. Interpretation, limitations, and terminological scope

The paper’s broader interpretation is that AutoReal advances LLM-driven theorem proving in real-world industrial-scale verification by combining open-source local deployability with comparatively high success on unfiltered proof workloads (Zhang et al., 9 Feb 2026). Two additional findings reinforce this interpretation. First, 93.6% of generated proofs differ in structure from their human counterparts, which the authors present as evidence of proof synthesis rather than rote memorization. Second, the method remains transparent enough to emit step-wise explanations alongside proofs.

The stated limitations define the present scope of the method. Context extraction is not yet automated; extensions to other proof assistants such as Coq or Lean are future work; and the paper identifies richer interaction or reinforcement learning as possible routes beyond the approximately 54% success ceiling. These caveats matter because they delimit what AutoReal currently solves: it is an end-to-end methodology for context-rich Isabelle proving, not a fully autonomous verifier across all formal ecosystems.

The term “AutoReal” also appears in a different sense in the supplied materials. In the structured summary of “DAARIA: Driver Assistance by Augmented Reality for Intelligent Automobile,” DAARIA is described as “a concrete instance of an ‘AutoReal’ solution,” referring to a tightly fused augmented-reality assistance system combining heterogeneous real-time sensor streams, calibration and synchronization on a moving platform, and an egocentric AR metaphor responsive to environment and driver state (George et al., 2012). This usage is distinct from AutoReal as a theorem proving methodology. The coexistence of these senses suggests that “AutoReal” functions more as a descriptive label than as a uniquely stabilized term across domains.

Within the verification literature represented here, however, AutoReal most specifically denotes the Zhang et al. methodology and its instantiated model, AutoReal-Prover: a compact 7B-scale, CoT-trained, context-augmented prover evaluated on seL4 and AFP security developments under Isabelle acceptance criteria.