---
title: 'FormalEvolve: Neuro-Symbolic Autoformalization'
url: https://www.emergentmind.com/topics/formalevolve
type: topic
---

# FormalEvolve: Neuro-Symbolic Autoformalization

Searching arXiv for recent papers on "FormalEvolve" and closely related uses of the term to ground the article in the cited literature.
FormalEvolve is a compilation-gated neuro-symbolic evolutionary framework for autoformalization that treats the translation of natural-language mathematics into compilable, machine-checkable statements as a budgeted, test-time search over semantically consistent repertoires rather than a one-shot generation problem [2603.19828]. Its central premise is that semantic faithfulness does not imply prover effectiveness: two semantically consistent formalizations can differ substantially in proof-search cost and success rate under a fixed prover and time budget. FormalEvolve therefore optimizes not only for compilation and semantic adequacy, but also for diversity, coverage, and downstream prover utility by combining LLM-driven mutation and crossover, bounded repair, symbolic Abstract Syntax Tree rewrites, and archive-based evolutionary selection.

## 1. Problem setting and formal objectives

FormalEvolve is defined for a fixed-budget, test-time regime in which each problem is allotted at most \(T\) generator calls and \(R\) prover attempts [2603.19828]. Within this regime, a candidate formalization \(c\) is filtered by a hard compilation gate and a soft semantic gate:
- \(C(c)\in\{0,1\}\), where \(C(c)=1\) iff \(c\) compiles.
- \(J(c)\in\{0,1\}\), where \(J(c)=1\) iff \(c\) passes an LLM-based semantic judge.

The output is the deduplicated repertoire
\[
\tilde G_T=\{c \mid C(c)=1 \wedge J(c)=1\},
\]
which is the set of compilation-feasible, semantically accepted candidates generated within the budget.

The framework explicitly optimizes three objectives. **Coverage** is the fraction of problems for which \(\tilde G_T\neq\varnothing\). **Uniformity** seeks to avoid concentrating semantic successes on a small subset of easy problems. **Prover utility** evaluates how useful the resulting repertoire is to a downstream prover when it is allowed to run on up to \(R=64\) candidates per problem, measured by pass@R, complete@R, and theorem-complete@R [2603.19828].

This formulation reframes autoformalization as repertoire construction under resource constraints. A plausible implication is that the framework is designed less as a pure generation model than as an adaptive search procedure whose target is a robust set of prover-facing formulations.

## 2. Archive structure, parent selection, and evolutionary search

FormalEvolve begins with **Seedbank Initialization**. A domain-specialized seed model \(M_{\rm seed}\) is called \(T_{\rm seed}\) times to draft Lean 4 statements from scratch; compilation-feasible outputs are inserted into a compilable archive \(\mathcal A\) [2603.19828]. The default seed model is Kimina-7B.

The main loop performs **Iterative Evolution** until the total number of debited LLM calls reaches \(T\). The archive is partitioned into \(K\) semi-isolated islands, with default \(K=2\), to encourage exploration. From a chosen island \(\mathcal A_I\), a parent \(c_p\) is sampled with probability
\[
p_i \propto \sigma(\lambda z_i)\cdot u_i,
\]
where
\[
z_i=\frac{\alpha_i-\operatorname{median}(\alpha)}{\operatorname{MAD}(\alpha)}, \qquad
\alpha_i=C(c_i)(1+J(c_i))\in\{0,1,2\},
\]
and
\[
u_i=\frac{1}{1+(1+\beta)n_i}.
\]
Here \(u_i\) penalizes over-used parents; the implementation uses \(\lambda=10\) and \(\beta=0.05\) [2603.19828].

After parent selection, the algorithm samples up to \(m\) inspirations from the island archive, mixing top-scoring and random context. Variation is then produced by a patch model \(M_{\rm patch}\), specifically Qwen3-30B-A3B, through three edit modes:
- **Full patch**: rewrite the entire statement given the informal input and parent context.
- **Diff patch**: propose a minimal edit.
- **Cross patch**: condition on one or more inspiration candidates.

The archive update is compilation-gated: each new compilation-feasible candidate is inserted into its island archive, exact duplicates are rejected, and semantic scores are recorded. Periodic migration across islands is used to reduce local collapse; the reported implementation uses migration rate \(0.1\) every 10 generations [2603.19828].

## 3. Repair operators and symbolic AST rewrites

A distinctive feature of FormalEvolve is that proposal generation is coupled to **bounded patch repair** [2603.19828]. If a proposal fails to compile, the system may invoke up to \(R_{\rm comp}=2\) compile-repair calls, each debited against the same global budget. If a candidate compiles but fails the semantic judge, the system may invoke up to \(R_{\rm sem}=2\) semantic-repair calls, also debited. The compile-repair prompt receives the original code and compiler errors and asks for minimal edits to restore compilation; the semantic-repair prompt receives judge rationale and asks for corrections to semantic mismatches.

When LLM patching repeatedly stalls through duplicates or compile failures, FormalEvolve falls back to **EvolAST**, a zero-call symbolic rewrite mechanism applied only to the theorem’s binder types and goal type, with imports and preamble unchanged [2603.19828]. The rewrite set is conservative and includes:
- commutativity of \(\wedge\): \(P\wedge Q \to Q\wedge P\),
- associativity/distributivity: \((P\wedge Q)\vee R \to (P\vee R)\wedge(Q\vee R)\),
- symmetry of equations: \(a=b \to b=a\),
- dual relations: \(a<b \to b>a\), \(a\le b \to b\ge a\),
- hypothesis reordering,
- negation normalizations such as \(\neg\neg P \to P\).

Each rewrite is applied in a bounded sequence, then compiled and semantically judged. Accepted outputs enter the archive without consuming LLM calls. This design creates a hybrid search process in which neural edits supply open-ended variation and symbolic rewrites inject low-cost structural diversity.

## 4. Metrics and empirical results

FormalEvolve evaluates candidates through both per-problem and cross-problem metrics [2603.19828]. Let \(E_t\) denote the set of all evaluated candidates up to \(t\) debited calls. The **semantic hit rate** at budget \(T\) is
\[
SH(T)=\mathbb I[\exists c\in E_T : C(c)=1 \wedge J(c)=1].
\]
Cross-problem concentration is summarized by the **Gini coefficient** over \(N\) test problems. If \(s_j(T)\) is the number of deduplicated semantic successes for problem \(j\), then
\[
\mathrm{Gini}(T)=
\frac{\sum_{i=1}^N\sum_{j=1}^N |s_i(T)-s_j(T)|}
{2N\sum_{j=1}^N s_j(T)+\varepsilon},
\]
with \(\varepsilon\to 0\) guarding division by zero. Lower Gini indicates more uniform distribution of semantic successes across problems.

The reported evaluation uses ProofNet test split (\(N=186\)) and CombiBench (\(N=100\)), with \(T=100\) generator calls per problem and \(R=64\) prover attempts per problem. The model stack is Kimina-Autoformalizer-7B for seeding, Qwen3-30B-A3B for patching and repair, CriticLean-Qwen3-14B as semantic judge, and Goedel-Prover-V2-32B as prover [2603.19828].

| Benchmark | Semantic coverage and uniformity at \(T=100\) | Downstream prover utility at \(R=64\) |
|---|---|---|
| ProofNet | CH@100 = 0.973; SH@100 = 0.849 vs 0.780 baseline; Gini 0.443 vs 0.555 | pass@64: 127/186 vs 119/186; complete@64: 52/186 vs 50/186; theorem-complete@64: 45/186 vs 46/186 |
| CombiBench | CH@100 = 1.000; SH@100 = 0.580 vs 0.460 baseline; Gini 0.759 vs 0.813 | pass@64: 44/100 vs 40/100; complete@64: 27/100 vs 23/100; theorem-complete@64: 13/100 vs 8/100 |

The reported budget-sweep curves show that FormalEvolve outperforms sampling baselines across call budgets, with the largest gains under strict budgets and domain shift, especially on CombiBench [2603.19828]. The empirical pattern supports the paper’s motivating distinction between semantic correctness and prover effectiveness: gains in semantic coverage are accompanied, on CombiBench, by improved prover-facing metrics under fixed downstream budget.

## 5. Conceptual lineage and neighboring evolutionary frameworks

The 2026 FormalEvolve paper sits within a wider cluster of work that uses evolutionary search to study or optimize formal systems, but these frameworks target different objects and fitness signals.

**EvoGPT-f** is described as a genetic-algorithm-driven framework that treats each formal mathematics language as an environment in which populations of GPT-style transformers adapt to maximize learnability [2402.16878]. Its individuals encode transformer architecture and training hyperparameters, and fitness is defined as negative held-out cross-entropy loss,
\[
F_i \equiv -L_i^v.
\]
It benchmarks five formal math corpora—Lean 3, Lean 4, Coq, HOL 4, and HOL Light—under four tokenization methods, and reports that evolution reliably reduces validation loss across generations, with Lean 4 and Coq achieving lower normalized losses than HOL 4 and Lean 3, and HOL Light the hardest to learn [2402.16878]. This suggests a broader research motif in which “evolution” operates over formal-mathematical representations, but the objective differs fundamentally from FormalEvolve’s compilation-gated repertoire search for prover-effective statements.

A more theoretical antecedent appears in work on **attribute-efficient evolvability of sparse linear functions** under Valiant’s model [1309.4132]. There, an evolutionary algorithm is specified by a representation class \(R\), a mutator, a sample size, a tolerance, and a selection rule, with BN and Opt selection defined in terms of empirical loss over sampled examples. The central results establish evolvability of sparse linear functions under \(\Delta\)-smooth \(G\)-nice or incoherent distributions with numbers of generations and sample sizes that depend on sparsity and accuracy rather than ambient dimension [1309.4132]. FormalEvolve does not inherit this formal PAC-style guarantee, but it shares the basic architecture of mutation, evaluation, and selection under explicit resource constraints.

A different extension of “formal evolution” appears in gene regulatory network models, where developmental dynamics and evolutionary dynamics are unified within a Boolean-network framework that allows gene duplication and divergence while defining phenotypes as marker-based macrostates rather than single attractors [1809.02331]. In that setting, evolution operates on adjacency matrices, thresholds, and genotype size; selection favors duplication or mutation moves that increase the basin size of a selected phenotype [1809.02331]. The commonality with FormalEvolve is methodological rather than domain-specific: both replace a single target state with a higher-level object—phenotype macrostate in one case, semantic repertoire in the other.

## 6. Scope, disambiguation, limitations, and future directions

FormalEvolve should be distinguished from two unrelated usages of “evolution” in recent arXiv literature. **ProEvolve** is a graph-based framework for programmable environment evolution in agent benchmarks; it models environments as typed relational graphs and uses graph transformations to add, remove, or modify capabilities, generating 200 environments and 3,000 task sandboxes from a single environment [2603.05910]. **“Evolution: A Unified Formula for Feature Operators from a High-level Perspective”** introduces an Evolution Function and Evolution Kernel that unify convolution, self-attention, and involution in computer vision [2305.14409]. These works share the vocabulary of evolution but not the autoformalization objective, the compilation gate, or the prover-oriented search protocol of FormalEvolve.

The limitations of FormalEvolve are explicit [2603.19828]. Its semantic gate depends on an imperfect LLM judge, reported at 79–81% accuracy on public semantic-faithfulness benchmarks. Prover utility depends on a fixed prover and prompt, so results may vary with different provers. EvolAST can sometimes hurt coverage, with dataset-dependent non-monotonic effect. The reported future directions are to integrate prover-native equivalence checks such as BEq, extend to Coq and Isabelle, refine AST-rewrite rule sets, and reduce judge noise with stronger formal verifiers.

Taken together, these properties position FormalEvolve as a neuro-symbolic evolutionary search framework specialized for test-time autoformalization under strict resource budgets. Its defining contribution is not merely to generate more semantically plausible formalizations, but to search for a diverse, compilation-feasible repertoire whose members differ in prover effectiveness and therefore materially affect downstream proof success [2603.19828].

Source: https://www.emergentmind.com/topics/formalevolve