Goedel-Code-Prover-8B Overview

Updated 3 July 2026

Goedel-Code-Prover-8B is an 8-billion parameter Transformer specialized for formal code verification and Lean 4 theorem proving using a two-stage, decomposition-guided proof search.
It combines supervised fine-tuning with hybrid reinforcement learning on a large-scale autoformalized dataset to achieve state-of-the-art performance in code verification tasks.
The model demonstrates robust empirical gains with a 2.6× improvement over previous systems, efficiently decomposing complex verification goals into atomic sub-problems.

Goedel-Code-Prover-8B refers to a class of 8-billion-parameter Transformer-based LLMs specialized for formal code verification and higher-order theorem proving in Lean 4, with particular emphasis on hierarchical proof search and large-scale dataset autoformalization. Distinct from previous single-pass provers, Goedel-Code-Prover-8B implements integrated decomposition-guided proof search, combines supervised and hybrid reinforcement learning on a corpus of formalized code-verification tasks, and exhibits strong empirical performance that substantially exceeds the capabilities of much larger prior neural provers (Li et al., 18 Mar 2026, Lin et al., 11 Feb 2025).

1. Model Architecture and Formalization Pipeline

Goedel-Code-Prover-8B utilizes a GPT-style decoder-only Transformer with approximately 8 billion parameters. The architecture is initialized from DeepSeek-Prover-V1.5-Base and fine-tuned via expert-iteration, although the released literature does not specify all component hyperparameters or layer counts; a typical 8B configuration would be 24–32 layers, hidden size 2048–4096, 16–32 attention heads, and MLP inner size 8192–11008 (Lin et al., 11 Feb 2025, Li et al., 18 Mar 2026). Input to the model is generally a (natural-language statement, Lean 4 formalization) pair, with the model generating Lean proof scripts.

Dataset construction is centered on autoformalization of natural language math problems—starting from nearly 1 million informal statements, dual formalizer models are fine-tuned (Qwen2.5-Coder-32B SFT variants) on human-labeled and LLM-generated formalization pairs. Outputs are filtered for compiling correctness and semantic faithfulness using both compiler checks and large model judgments, producing the Goedel-Pset-v1 dataset (1.64M statements) and a solved subset of over 900K with Lean proofs via expert-iteration. The formalization quality is summarized in the following table (Lin et al., 11 Feb 2025):

Metric	Formalizer A	Formalizer B
Compile@1	76.7%	88.5%
Faithfulness@1	48.1%	80.4%
Compile+Faith@8	82.3%	95.8%

Supervised training and expert iteration are performed solely on Lean-4 compatible formal code and proofs, with sequence-packing to saturate hardware utilization.

2. Hierarchical Proof Search and Decomposition Framework

Goedel-Code-Prover-8B implements a two-stage hierarchical proof-search system within Lean 4:

Stage 1 (Lemma Decomposition): For a verification theorem $G$ , the model proposes helper lemmas $L_1,\ldots,L_k$ plus a reconstruction proof for $(L_1 \land \dots \land L_k) \implies G$ . Each candidate decomposition is verified by Lean reconstruction and by random counterexample search (QuickCheck), with only structurally productive splits retained.
Stage 2 (Lemma Completion): The set of atomic goals is closed via tactic-by-tactic proof generation, continually interacting with the Lean kernel for subgoal generation and error feedback.

At every iteration, the model selects the open goal with the maximal operator footprint (AST complexity) and applies recursive decomposition until all branches are sufficiently atomic or compute budgets are exhausted (Li et al., 18 Mar 2026).

3. Decomposition Score and Training Objectives

Central to the search is the decomposition score $S(L_1,\ldots,L_k;G)$ , defined as:

$v(L_1,\ldots,L_k;G)$ : validity indicator using Lean proof reconstruction and QuickCheck,
$r(L_1,\ldots,L_k;G) = \max(1 - \bar d/d(G), 0)$ with $\bar d$ a LogSumExp-aggregated subgoal size,
$S = v \cdot r$ , a continuous reward in $[0,1]$ .

This score forms both the RL training reward (dense, aligning optimization and deployment) and the inference-time ranking metric. Training follows a hybrid schema:

Supervised fine-tuning: Cross-entropy on decomposition and completion trajectories.
Hybrid RL: Group Relative Policy Optimization (GRPO) using $S$ for decomposition, and concurrent supervised replay for tactic-generation stability; overall objective $L_1,\ldots,L_k$ 0 with $L_1,\ldots,L_k$ 1.

Subproblem curricula result from incorporating solved decompositions and completions back into the training buffer, iteratively growing the model's capacity (Li et al., 18 Mar 2026).

4. Empirical Benchmarks and Scaling Properties

Goedel-Code-Prover-8B sets a new state-of-the-art for machine-checked code verification in Lean 4 on benchmarks including Verina, Clever, and AlgoVeri (427 tasks).

Benchmark	Goedel-Code-Prover-8B	Best Prior (GPT-5.3/BFS-Prover)
Verina	68.8%	19.7%
Clever	54.0%	23.6%
AlgoVeri	62.3%	17.4%
Total	62.0%	23.8%

This represents a 2.6× improvement over prior best neural systems, including models with up to 84× the parameter count (Li et al., 18 Mar 2026). Inference-time scaling remains robust: deeper decomposition iterations monotonically reduce subgoal complexity, while parallel sampling (pass@ $L_1,\ldots,L_k$ 2) yields continually increasing success rates with no observed early saturation up to $L_1,\ldots,L_k$ 3.

5. Relations to Predecessor and Contemporary Models

Goedel-Code-Prover-8B is conceptually and methodologically distinct from the Goedel-Prover series focused on mathematical (not code) theorem proving (Lin et al., 11 Feb 2025, Lin et al., 5 Aug 2025). Whereas Goedel-Prover-V2-8B employs scaffolded curriculum synthesis, verifier-guided self-correction, and model averaging to reach 84.6% pass@32 on MiniF2F (outperforming much larger open-source provers), Goedel-Code-Prover-8B targets structured program verification with explicit decomposition-guided search. Both lines employ some form of iterative improvement (expert iteration or RL), but Goedel-Code-Prover-8B aligns its optimization directly with the structural reduction of code-verification goals. No retrieval, external memory, or tool-use chains are integrated at inference time in the code-prover, a design choice contrasting with recent agentic approaches (Chung et al., 9 Apr 2026).

6. Implementation, Limitations, and Future Directions

The model is implemented on top of Qwen-3-8B, with custom Lean tactics (operatorcount, quickcheck) augmenting the search. Distributed verification can scale to 512 Lean 4 workers. Training is compute-intensive but manageable for the parameter count (e.g., RL phase with batch=64 prompts × 8 samples, RL learning rate $L_1,\ldots,L_k$ 4, and prompt/context length up to 10,240 tokens) (Li et al., 18 Mar 2026). Direct model architecture details (e.g., layer counts, vocab size) are underreported due to non-disclosure in base releases (Lin et al., 11 Feb 2025).

Key limitations include lack of explicit retrieval, domain adaptation (underperformance on ProofNet for non-code theorems), and reliance on strong problem-specific autoformalizers. Future work is expected to target:

Integration with stepwise or interactive search for greater generalization.
Incorporation of explicit Σ-reflection mechanisms and higher-order logic support, as seen in formalizations of Gödel’s incompleteness theorems using hereditarily finite set theory (Paulson, 2021).
Binder-aware internal syntax layers and certificate-length induction to systematically handle abstractions and automate reflection proofs.

7. Theoretical Foundations and Lessons from Mechanized Proofs

The design of Goedel-Code-Prover-8B is informed by formalizations of Gödelian meta-mathematics over hereditarily finite (HF) set theory, as exhibited in the Isabelle/HOL mechanization of the incompleteness theorems (Paulson, 2021). Fundamental techniques—such as encoding of syntax via HF codes, Σ-reflectivity, recursive substitution/abstraction, binders using de Bruijn or nominal representations, and certificate-induction schemes—can be internalized in future large-scale code-provers. A self-verifying architecture should integrate a high-level meta-logic for reflection and derivability, a low-level HF engine for syntactic calculations, binder-aware representations, and a library of Σ-definable predicates and generic single-valuedness lemmas.

In conclusion, Goedel-Code-Prover-8B establishes a new baseline for Lean-4-based code verification by unifying decomposition-guided proof planning, hybrid RL, and large-scale autoformalization. Its core advances lie in aligning structural decomposition rewards with inference-time search and achieving competitive or superlative empirical results despite moderate model scale (Li et al., 18 Mar 2026, Lin et al., 11 Feb 2025). Its methodological lineage, empirical scaling, and theoretical underpinnings position it as a foundation for subsequent advances in neural-augmented mechanized verification and formal mathematics.