---
title: Goedel-Code-Prover-8B Overview
url: https://www.emergentmind.com/topics/goedel-code-prover-8b
type: topic
---

# Goedel-Code-Prover-8B Overview

Goedel-Code-Prover-8B refers to a class of 8-billion-parameter Transformer-based language models specialized for formal code verification and higher-order theorem proving in Lean 4, with particular emphasis on hierarchical proof search and large-scale dataset autoformalization. Distinct from previous single-pass provers, Goedel-Code-Prover-8B implements integrated decomposition-guided proof search, combines supervised and hybrid reinforcement learning on a corpus of formalized code-verification tasks, and exhibits strong empirical performance that substantially exceeds the capabilities of much larger prior neural provers [2603.19329, 2502.07640].

## 1. Model Architecture and Formalization Pipeline

Goedel-Code-Prover-8B utilizes a GPT-style decoder-only Transformer with approximately 8 billion parameters. The architecture is initialized from DeepSeek-Prover-V1.5-Base and fine-tuned via expert-iteration, although the released literature does not specify all component hyperparameters or layer counts; a typical 8B configuration would be 24–32 layers, hidden size 2048–4096, 16–32 attention heads, and MLP inner size 8192–11008 [2502.07640, 2603.19329]. Input to the model is generally a (natural-language statement, Lean 4 formalization) pair, with the model generating Lean proof scripts.

Dataset construction is centered on autoformalization of natural language math problems—starting from nearly 1 million informal statements, dual formalizer models are fine-tuned (Qwen2.5-Coder-32B SFT variants) on human-labeled and LLM-generated formalization pairs. Outputs are filtered for compiling correctness and semantic faithfulness using both compiler checks and large model judgments, producing the Goedel-Pset-v1 dataset (1.64M statements) and a solved subset of over 900K with Lean proofs via expert-iteration. The formalization quality is summarized in the following table [2502.07640]:

| Metric            | Formalizer A | Formalizer B |
|-------------------|-------------:|-------------:|
| Compile@1         | 76.7%        | 88.5%        |
| Faithfulness@1    | 48.1%        | 80.4%        |
| Compile+Faith@8   | 82.3%        | 95.8%        |

Supervised training and expert iteration are performed solely on Lean-4 compatible formal code and proofs, with sequence-packing to saturate hardware utilization.

## 2. Hierarchical Proof Search and Decomposition Framework

Goedel-Code-Prover-8B implements a two-stage hierarchical proof-search system within Lean 4:

- **Stage 1 (Lemma Decomposition):** For a verification theorem $G$, the model proposes helper lemmas $L_1,\ldots,L_k$ plus a reconstruction proof for $(L_1 \land \dots \land L_k) \implies G$. Each candidate decomposition is verified by Lean reconstruction and by random counterexample search (QuickCheck), with only structurally productive splits retained.
- **Stage 2 (Lemma Completion):** The set of atomic goals is closed via tactic-by-tactic proof generation, continually interacting with the Lean kernel for subgoal generation and error feedback.

At every iteration, the model selects the open goal with the maximal operator footprint (AST complexity) and applies recursive decomposition until all branches are sufficiently atomic or compute budgets are exhausted [2603.19329].

## 3. Decomposition Score and Training Objectives

Central to the search is the decomposition score $S(L_1,\ldots,L_k;G)$, defined as:
- $v(L_1,\ldots,L_k;G)$: validity indicator using Lean proof reconstruction and QuickCheck,
- $r(L_1,\ldots,L_k;G) = \max(1 - \bar d/d(G), 0)$ with $\bar d$ a LogSumExp-aggregated subgoal size,
- $S = v \cdot r$, a continuous reward in $[0,1]$.

This score forms both the RL training reward (dense, aligning optimization and deployment) and the inference-time ranking metric. Training follows a hybrid schema:
- **Supervised fine-tuning:** Cross-entropy on decomposition and completion trajectories.
- **Hybrid RL:** Group Relative Policy Optimization (GRPO) using $S$ for decomposition, and concurrent supervised replay for tactic-generation stability; overall objective $J = J_\mathrm{GRPO}(S) + \lambda J_\mathrm{SFT}$ with $0 < \lambda < 1$.

Subproblem curricula result from incorporating solved decompositions and completions back into the training buffer, iteratively growing the model's capacity [2603.19329].

## 4. Empirical Benchmarks and Scaling Properties

Goedel-Code-Prover-8B sets a new state-of-the-art for machine-checked code verification in Lean 4 on benchmarks including Verina, Clever, and AlgoVeri (427 tasks).

| Benchmark | Goedel-Code-Prover-8B | Best Prior (GPT-5.3/BFS-Prover) |
|-----------|----------------------:|-------------------------------:|
| Verina    | 68.8%                | 19.7%                          |
| Clever    | 54.0%                | 23.6%                          |
| AlgoVeri  | 62.3%                | 17.4%                          |
| Total     | 62.0%                | 23.8%                          |

This represents a 2.6× improvement over prior best neural systems, including models with up to 84× the parameter count [2603.19329]. Inference-time scaling remains robust: deeper decomposition iterations monotonically reduce subgoal complexity, while parallel sampling (pass@$k$) yields continually increasing success rates with no observed early saturation up to $k=128$.

## 5. Relations to Predecessor and Contemporary Models

Goedel-Code-Prover-8B is conceptually and methodologically distinct from the Goedel-Prover series focused on mathematical (not code) theorem proving [2502.07640, 2508.03613]. Whereas Goedel-Prover-V2-8B employs scaffolded curriculum synthesis, verifier-guided self-correction, and model averaging to reach 84.6% pass@32 on MiniF2F (outperforming much larger open-source provers), Goedel-Code-Prover-8B targets structured program verification with explicit decomposition-guided search. Both lines employ some form of iterative improvement (expert iteration or RL), but Goedel-Code-Prover-8B aligns its optimization directly with the structural reduction of code-verification goals. No retrieval, external memory, or tool-use chains are integrated at inference time in the code-prover, a design choice contrasting with recent agentic approaches [2604.08388].

## 6. Implementation, Limitations, and Future Directions

The model is implemented on top of Qwen-3-8B, with custom Lean tactics (operatorcount, quickcheck) augmenting the search. Distributed verification can scale to 512 Lean 4 workers. Training is compute-intensive but manageable for the parameter count (e.g., RL phase with batch=64 prompts × 8 samples, RL learning rate $5\times10^{-6}$, and prompt/context length up to 10,240 tokens) [2603.19329]. Direct model architecture details (e.g., layer counts, vocab size) are underreported due to non-disclosure in base releases [2502.07640].

Key limitations include lack of explicit retrieval, domain adaptation (underperformance on ProofNet for non-code theorems), and reliance on strong problem-specific autoformalizers. Future work is expected to target:
- Integration with stepwise or interactive search for greater generalization.
- Incorporation of explicit Σ-reflection mechanisms and higher-order logic support, as seen in formalizations of Gödel’s incompleteness theorems using hereditarily finite set theory [2104.14260].
- Binder-aware internal syntax layers and certificate-length induction to systematically handle abstractions and automate reflection proofs.

## 7. Theoretical Foundations and Lessons from Mechanized Proofs

The design of Goedel-Code-Prover-8B is informed by formalizations of Gödelian meta-mathematics over hereditarily finite (HF) set theory, as exhibited in the Isabelle/HOL mechanization of the incompleteness theorems [2104.14260]. Fundamental techniques—such as encoding of syntax via HF codes, Σ-reflectivity, recursive substitution/abstraction, binders using de Bruijn or nominal representations, and certificate-induction schemes—can be internalized in future large-scale code-provers. A self-verifying architecture should integrate a high-level meta-logic for reflection and derivability, a low-level HF engine for syntactic calculations, binder-aware representations, and a library of Σ-definable predicates and generic single-valuedness lemmas.

---

In conclusion, Goedel-Code-Prover-8B establishes a new baseline for Lean-4-based code verification by unifying decomposition-guided proof planning, hybrid RL, and large-scale autoformalization. Its core advances lie in aligning structural decomposition rewards with inference-time search and achieving competitive or superlative empirical results despite moderate model scale [2603.19329, 2502.07640]. Its methodological lineage, empirical scaling, and theoretical underpinnings position it as a foundation for subsequent advances in neural-augmented mechanized verification and formal mathematics.

Source: https://www.emergentmind.com/topics/goedel-code-prover-8b