FormalEvolve: Neuro-Symbolic Autoformalization

• FormalEvolve is a neuro-symbolic evolutionary framework that autoformalizes natural-language math into compilable, machine-checkable formal statements.
• It combines LLM-driven mutation, bounded repair, and symbolic AST rewrites to optimize both compilation and semantic fidelity under fixed resource budgets.
• Its evolutionary search process maximizes coverage, uniformity, and prover utility, leading to improved downstream proof success across benchmark tests.

Searching arXiv for papers on "FormalEvolve" and closely related uses of the term to ground the article in the cited literature. FormalEvolve is a compilation-gated neuro-symbolic evolutionary framework for autoformalization that treats the translation of natural-language mathematics into compilable, machine-checkable statements as a budgeted, test-time search over semantically consistent repertoires rather than a one-shot generation problem (Lu et al., 20 Mar 2026). Its central premise is that semantic faithfulness does not imply prover effectiveness: two semantically consistent formalizations can differ substantially in proof-search cost and success rate under a fixed prover and time budget. FormalEvolve therefore optimizes not only for compilation and semantic adequacy, but also for diversity, coverage, and downstream prover utility by combining LLM-driven mutation and crossover, bounded repair, symbolic Abstract Syntax Tree rewrites, and archive-based evolutionary selection.

1. Problem setting and formal objectives

FormalEvolve is defined for a fixed-budget, test-time regime in which each problem is allotted at most $T$ generator calls and $R$ prover attempts (Lu et al., 20 Mar 2026). Within this regime, a candidate formalization $c$ is filtered by a hard compilation gate and a soft semantic gate:

• $C(c)\in\{0,1\}$, where $C(c)=1$ iff $c$ compiles.
• $J(c)\in\{0,1\}$, where $J(c)=1$ iff $c$ passes an LLM-based semantic judge.

The output is the deduplicated repertoire

$\tilde G_T=\{c \mid C(c)=1 \wedge J(c)=1\},$

which is the set of compilation-feasible, semantically accepted candidates generated within the budget.

The framework explicitly optimizes three objectives. Coverage is the fraction of problems for which $R$0. Uniformity seeks to avoid concentrating semantic successes on a small subset of easy problems. Prover utility evaluates how useful the resulting repertoire is to a downstream prover when it is allowed to run on up to $R$1 candidates per problem, measured by pass@R, complete@R, and theorem-complete@R (Lu et al., 20 Mar 2026).

This formulation reframes autoformalization as repertoire construction under resource constraints. A plausible implication is that the framework is designed less as a pure generation model than as an adaptive search procedure whose target is a robust set of prover-facing formulations.

2. Archive structure, parent selection, and evolutionary search

FormalEvolve begins with Seedbank Initialization. A domain-specialized seed model $R$2 is called $R$3 times to draft Lean 4 statements from scratch; compilation-feasible outputs are inserted into a compilable archive $R$4 (Lu et al., 20 Mar 2026). The default seed model is Kimina-7B.

The main loop performs Iterative Evolution until the total number of debited LLM calls reaches $R$5. The archive is partitioned into $R$6 semi-isolated islands, with default $R$7, to encourage exploration. From a chosen island $R$8, a parent $R$9 is sampled with probability

$c$0

where

$c$1

and

$c$2

Here $c$3 penalizes over-used parents; the implementation uses $c$4 and $c$5 (Lu et al., 20 Mar 2026).

After parent selection, the algorithm samples up to $c$6 inspirations from the island archive, mixing top-scoring and random context. Variation is then produced by a patch model $c$7, specifically Qwen3-30B-A3B, through three edit modes:

• Full patch: rewrite the entire statement given the informal input and parent context.
• Diff patch: propose a minimal edit.
• Cross patch: condition on one or more inspiration candidates.

The archive update is compilation-gated: each new compilation-feasible candidate is inserted into its island archive, exact duplicates are rejected, and semantic scores are recorded. Periodic migration across islands is used to reduce local collapse; the reported implementation uses migration rate $c$8 every 10 generations (Lu et al., 20 Mar 2026).

3. Repair operators and symbolic AST rewrites

A distinctive feature of FormalEvolve is that proposal generation is coupled to bounded patch repair (Lu et al., 20 Mar 2026). If a proposal fails to compile, the system may invoke up to $c$9 compile-repair calls, each debited against the same global budget. If a candidate compiles but fails the semantic judge, the system may invoke up to $C(c)\in\{0,1\}$0 semantic-repair calls, also debited. The compile-repair prompt receives the original code and compiler errors and asks for minimal edits to restore compilation; the semantic-repair prompt receives judge rationale and asks for corrections to semantic mismatches.

When LLM patching repeatedly stalls through duplicates or compile failures, FormalEvolve falls back to EvolAST, a zero-call symbolic rewrite mechanism applied only to the theorem’s binder types and goal type, with imports and preamble unchanged (Lu et al., 20 Mar 2026). The rewrite set is conservative and includes:

• commutativity of $C(c)\in\{0,1\}$1: $C(c)\in\{0,1\}$2,
• associativity/distributivity: $C(c)\in\{0,1\}$3,
• symmetry of equations: $C(c)\in\{0,1\}$4,
• dual relations: $C(c)\in\{0,1\}$5, $C(c)\in\{0,1\}$6,
• hypothesis reordering,
• negation normalizations such as $C(c)\in\{0,1\}$7.

Each rewrite is applied in a bounded sequence, then compiled and semantically judged. Accepted outputs enter the archive without consuming LLM calls. This design creates a hybrid search process in which neural edits supply open-ended variation and symbolic rewrites inject low-cost structural diversity.

4. Metrics and empirical results

FormalEvolve evaluates candidates through both per-problem and cross-problem metrics (Lu et al., 20 Mar 2026). Let $C(c)\in\{0,1\}$8 denote the set of all evaluated candidates up to $C(c)\in\{0,1\}$9 debited calls. The semantic hit rate at budget $C(c)=1$0 is

$C(c)=1$1

Cross-problem concentration is summarized by the Gini coefficient over $C(c)=1$2 test problems. If $C(c)=1$3 is the number of deduplicated semantic successes for problem $C(c)=1$4, then

$C(c)=1$5

with $C(c)=1$6 guarding division by zero. Lower Gini indicates more uniform distribution of semantic successes across problems.

The reported evaluation uses ProofNet test split ($C(c)=1$7) and CombiBench ($C(c)=1$8), with $C(c)=1$9 generator calls per problem and $c$0 prover attempts per problem. The model stack is Kimina-Autoformalizer-7B for seeding, Qwen3-30B-A3B for patching and repair, CriticLean-Qwen3-14B as semantic judge, and Goedel-Prover-V2-32B as prover (Lu et al., 20 Mar 2026).

Benchmark	Semantic coverage and uniformity at $c$1	Downstream prover utility at $c$2
ProofNet	CH@100 = 0.973; SH@100 = 0.849 vs 0.780 baseline; Gini 0.443 vs 0.555	pass@64: 127/186 vs 119/186; complete@64: 52/186 vs 50/186; theorem-complete@64: 45/186 vs 46/186
CombiBench	CH@100 = 1.000; SH@100 = 0.580 vs 0.460 baseline; Gini 0.759 vs 0.813	pass@64: 44/100 vs 40/100; complete@64: 27/100 vs 23/100; theorem-complete@64: 13/100 vs 8/100

The reported budget-sweep curves show that FormalEvolve outperforms sampling baselines across call budgets, with the largest gains under strict budgets and domain shift, especially on CombiBench (Lu et al., 20 Mar 2026). The empirical pattern supports the paper’s motivating distinction between semantic correctness and prover effectiveness: gains in semantic coverage are accompanied, on CombiBench, by improved prover-facing metrics under fixed downstream budget.

5. Conceptual lineage and neighboring evolutionary frameworks

The 2026 FormalEvolve paper sits within a wider cluster of work that uses evolutionary search to study or optimize formal systems, but these frameworks target different objects and fitness signals.

EvoGPT-f is described as a genetic-algorithm-driven framework that treats each formal mathematics language as an environment in which populations of GPT-style transformers adapt to maximize learnability (Mercer, 2024). Its individuals encode transformer architecture and training hyperparameters, and fitness is defined as negative held-out cross-entropy loss,

$c$3

It benchmarks five formal math corpora—Lean 3, Lean 4, Coq, HOL 4, and HOL Light—under four tokenization methods, and reports that evolution reliably reduces validation loss across generations, with Lean 4 and Coq achieving lower normalized losses than HOL 4 and Lean 3, and HOL Light the hardest to learn (Mercer, 2024). This suggests a broader research motif in which “evolution” operates over formal-mathematical representations, but the objective differs fundamentally from FormalEvolve’s compilation-gated repertoire search for prover-effective statements.

A more theoretical antecedent appears in work on attribute-efficient evolvability of sparse linear functions under Valiant’s model (Angelino et al., 2013). There, an evolutionary algorithm is specified by a representation class $c$4, a mutator, a sample size, a tolerance, and a selection rule, with BN and Opt selection defined in terms of empirical loss over sampled examples. The central results establish evolvability of sparse linear functions under $c$5-smooth $c$6-nice or incoherent distributions with numbers of generations and sample sizes that depend on sparsity and accuracy rather than ambient dimension (Angelino et al., 2013). FormalEvolve does not inherit this formal PAC-style guarantee, but it shares the basic architecture of mutation, evaluation, and selection under explicit resource constraints.

A different extension of “formal evolution” appears in gene regulatory network models, where developmental dynamics and evolutionary dynamics are unified within a Boolean-network framework that allows gene duplication and divergence while defining phenotypes as marker-based macrostates rather than single attractors (Borriello et al., 2018). In that setting, evolution operates on adjacency matrices, thresholds, and genotype size; selection favors duplication or mutation moves that increase the basin size of a selected phenotype (Borriello et al., 2018). The commonality with FormalEvolve is methodological rather than domain-specific: both replace a single target state with a higher-level object—phenotype macrostate in one case, semantic repertoire in the other.

6. Scope, disambiguation, limitations, and future directions

FormalEvolve should be distinguished from two unrelated usages of “evolution” in recent arXiv literature. ProEvolve is a graph-based framework for programmable environment evolution in agent benchmarks; it models environments as typed relational graphs and uses graph transformations to add, remove, or modify capabilities, generating 200 environments and 3,000 task sandboxes from a single environment (Li et al., 6 Mar 2026). “Evolution: A Unified Formula for Feature Operators from a High-level Perspective” introduces an Evolution Function and Evolution Kernel that unify convolution, self-attention, and involution in computer vision (Cai, 2023). These works share the vocabulary of evolution but not the autoformalization objective, the compilation gate, or the prover-oriented search protocol of FormalEvolve.

The limitations of FormalEvolve are explicit (Lu et al., 20 Mar 2026). Its semantic gate depends on an imperfect LLM judge, reported at 79–81% accuracy on public semantic-faithfulness benchmarks. Prover utility depends on a fixed prover and prompt, so results may vary with different provers. EvolAST can sometimes hurt coverage, with dataset-dependent non-monotonic effect. The reported future directions are to integrate prover-native equivalence checks such as BEq, extend to Coq and Isabelle, refine AST-rewrite rule sets, and reduce judge noise with stronger formal verifiers.

Taken together, these properties position FormalEvolve as a neuro-symbolic evolutionary search framework specialized for test-time autoformalization under strict resource budgets. Its defining contribution is not merely to generate more semantically plausible formalizations, but to search for a diverse, compilation-feasible repertoire whose members differ in prover effectiveness and therefore materially affect downstream proof success (Lu et al., 20 Mar 2026).