Multi-Agent Autoformalization

• Multi-agent autoformalization is the process of using role-specialized agents and formal tools to transform informal texts into executable, verifiable formal representations.
• The methodology features decomposed pipelines where distinct agents handle generation, retrieval, proof checking, critique, and repair across various formal targets.
• This approach enhances reliability and scalability by integrating orchestration strategies, intermediate representations, and human oversight into the formalization process.

Multi-agent autoformalization is the use of multiple coordinated, role-specialized agents—typically LLM agents coupled to formal tools—to transform informal artifacts into machine-checkable or executable formal objects. Across the recent literature, the inputs range from natural-language mathematical statements, LaTeX research manuscripts, legal provisions, and system descriptions to interaction scenarios and task descriptions; the outputs range from Lean 4 or Isabelle/HOL declarations to Prolog game rules, CSP# models, GDPR rule trees, and verified multi-agent workflow specifications. The defining methodological shift is from monolithic prompting to decomposed pipelines in which generation, retrieval, proof checking, critique, repair, scheduling, and human review are distributed across distinct agents or coordination mechanisms (Mensfelt et al., 2024, Zhang et al., 10 Oct 2025, Zuo et al., 28 Sep 2025, Ren et al., 18 Feb 2026, Nguyen et al., 16 Apr 2026, Pegden, 8 Jun 2026).

1. Scope of the field

Multi-agent autoformalization originally names the translation of informal mathematics into formal mathematics, but the current literature uses the same paradigm for a broader class of formalisms. MASA defines autoformalization as turning a mathematical statement in informal language into a fully machine-checkable formal object such as an Isabelle/HOL or Lean4 definition (Zhang et al., 10 Oct 2025). GAMA applies the idea to multi-agent interaction scenarios, autoformalizing natural-language descriptions of game-theoretic situations into executable logic programs defining game rules (Mensfelt et al., 2024). PAT-Agent targets formal verification rather than proof assistants, mapping natural-language system descriptions into verifiable CSP# models for the Process Analysis Toolkit (Zuo et al., 28 Sep 2025). GDPR auto-formalization maps articles into scenarios, formal rules, and atomic facts, with deterministic evaluation and human verification (Nguyen et al., 16 Apr 2026). Meta-Agent and MetaAgent extend the notion further: one formalizes task descriptions into a verified directed acyclic graph of agent specifications, while the other generates a finite state machine controlling a multi-agent workflow (Xu et al., 24 May 2026, Zhang et al., 30 Jul 2025).

The field therefore spans at least three distinct formal targets. One target is proof-assistant code, exemplified by MASA, MerLean, Trellis, LeanMarathon, AutoformBot, Agent Hunt, and the tensor-network formalization project (Zhang et al., 10 Oct 2025, Ren et al., 18 Feb 2026, Pegden, 8 Jun 2026, Zhang et al., 3 Jun 2026, Rammal et al., 28 May 2026, Brown et al., 6 Mar 2026, Lu et al., 8 Jul 2026). A second target is executable or model-checkable artifacts, such as Prolog game descriptions and PAT models (Mensfelt et al., 2024, Zuo et al., 28 Sep 2025). A third target is structured formal representations whose purpose is verification-centered reasoning rather than theorem proving per se, such as GDPR rule trees, contract-bearing DAG nodes, and FSM state specifications (Nguyen et al., 16 Apr 2026, Xu et al., 24 May 2026, Zhang et al., 30 Jul 2025).

This breadth corrects a common oversimplification: multi-agent autoformalization is not confined to translating textbook mathematics into Lean. The published systems instead treat autoformalization as a general bridge from ambiguous natural language to rigid, tool-consumable representations, with the exact formal substrate determined by the downstream verifier or executor (Zhang et al., 10 Oct 2025, Zuo et al., 28 Sep 2025, Xu et al., 24 May 2026).

2. Recurring architectural patterns

A recurrent architecture is staged decomposition into narrowly scoped agents. GAMA specifies six stages: prompt engineering and LLM augmentation of generative agents; natural-language interpretation of interaction scenarios; automatic translation into executable logic programs; solver-based syntactic validation; iterative, tournament-based runtime testing; and exact semantic validation when ground truth is available (Mensfelt et al., 2024). MASA similarly decomposes the pipeline into an Agent Manager, AutoformalizationAgent, HardCritiqueAgent, SoftCritiqueAgent, ImportRetrievalAgent, FormalRefinementAgent, and InformalRefinementAgent, with theorem provers and retrievers treated as first-class components (Zhang et al., 10 Oct 2025). PAT-Agent uses a four-module pipeline comprising a Planning LLM, a Code-Generation LLM, a Model-Checker, and a Repair Loop (Zuo et al., 28 Sep 2025). MerLean organizes the workflow as Statement Extraction, Formalization, Verification, and Informalization agents, supported by lean-lsp-mcp, leansearch or loogle, and a Faithfulness Checker (Ren et al., 18 Feb 2026).

A second recurring pattern is explicit orchestration. Some systems use centralized planners. Meta-Agent constructs a DAG $G=(V,E)$ of agent specifications with explicit input contracts, output contracts, and verification criteria, then dispatches subtasks topologically in an execution phase gated by verification (Xu et al., 24 May 2026). AutoformBot uses a long-lived Orchestrator, mid-level managers such as a Trace Analyzer and Supervisor, and low-level Workers and Reviewers operating through a shared Task DAG, scheduler, merge queue, and git worktrees (Rammal et al., 28 May 2026). LeanMarathon uses a two-stage orchestrator: Stage 1 adversarially stabilizes target fidelity, and Stage 2 discharges the proof DAG from dynamic leaves upward in parallel CI-gated rounds (Zhang et al., 3 Jun 2026).

Other systems deliberately reduce or avoid centralized planning. Agent Hunt distributes workload through a simulated bounty-based marketplace in which four coding agents lock theorems, place sub-bounties on intermediate lemmas, and compete or cooperate through a shared Git repository rather than a static central planner (Brown et al., 6 Mar 2026). Trellis also separates authority from creativity: a deterministic Rust kernel governs edits, validation, and checkpoints, while Worker, Verifier lanes, Reviewer, Checker Server, and Audit Agent each have sharply bounded powers (Pegden, 8 Jun 2026). This suggests that “multi-agent” in the literature denotes not a single canonical control topology but a family of coordination regimes ranging from orchestrated DAGs to marketplaces and process-semantic kernels.

A third pattern is specialization by failure mode. MASA distinguishes hard critiques for syntax and type correctness from soft critiques for semantic alignment (Zhang et al., 10 Oct 2025). LeanMarathon distinguishes Blueprinter, Target-Reviewer, Worker, and Refiner contracts (Zhang et al., 3 Jun 2026). The tensor-network system distinguishes leanSearch, lean, leanSimplifier, leanBlueprint, reviewer, Orchestrator, and Human Supervisor (Lu et al., 8 Jul 2026). The design principle is consistent: subproblems such as retrieval, proof writing, dependency synchronization, semantic auditing, and local repair are treated as distinct tasks rather than as a single prompting problem.

3. Formal representations and intermediate objects

The formalism produced by multi-agent systems is typically mediated by constrained intermediate representations. MASA states autoformalization as a function

$$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$

and refines it as a composition of agent functions: $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$ No new grammar is introduced; instead, agents rely on existing proof-assistant grammars and LLMs to generate well-formed terms (Zhang et al., 10 Oct 2025).

GAMA formalizes a representative $2\times2$ simultaneous-move game with players $P=\{1,2\}$, strategies $S_i=\{C,D\}$, payoff functions $u_i:S_1\times S_2\to\mathbb R$, and Prisoner’s Dilemma constraints $T>R>P>S$. The natural-language scenario is translated into a Prolog program with two modules: game-independent rules in Situation Calculus style and game-dependent clauses such as initial/1, legal/2, effect/3, and payoff/... predicates (Mensfelt et al., 2024). In this setting, autoformalization does not end at syntax; it yields an executable specification that can be queried and simulated.

PAT-Agent makes the intermediate representation explicit. The Planning LLM produces a JSON-like plan

$$P = \{ \texttt{constants: [...]},\; \texttt{variables: [...]},\; \texttt{actions: [...]} \},$$

where each action contains fields such as name, guard, and updates. The Code-Generation LLM then maps this plan, together with a syntax cheat-sheet and a retrieved exemplar, into CSP# (Zuo et al., 28 Sep 2025). The intermediate plan functions as a stable interface between semantic extraction and code generation.

GDPR auto-formalization likewise uses a compact formal object. A rule is a 4-tuple $R=(p,op,C,E)$, where $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$0 is the target predicate, $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$1, $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$2 is the set of condition predicates, and $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$3 is the set of exception predicates. Evaluation is defined by a closed-world assumption with exception priority: $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$4 and otherwise by whether all or any required conditions appear in the fact set $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$5 (Nguyen et al., 16 Apr 2026). The representation is intentionally simple, but the paper emphasizes that legal nuance is not thereby eliminated.

Workflow-generating systems formalize the agents themselves. Meta-Agent represents a constructed multi-agent system as a DAG $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$6, where each node $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$7 carries an input contract $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$8, output contract $$\mathcal{A}\colon \mathsf{NL}\to\mathsf{Formal}, \qquad \phi=\mathcal{A}(s),$$9, and verification criterion $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$0 (Xu et al., 24 May 2026). MetaAgent instead uses a finite state machine $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$1, where each state specifies an agent, a subtask instruction, a verifier, and listeners (Zhang et al., 30 Jul 2025). MerLean adds another important intermediate object: a dependency-ordered JSON array extracted from LaTeX, where each statement record includes fields such as id, type, latex, deps, and sketch (Ren et al., 18 Feb 2026). A plausible implication is that typed, schema-constrained intermediates are one of the main mechanisms by which these systems reduce drift between informal text and formal output.

4. Verification, critique, and repair

Verification is the central organizing principle of multi-agent autoformalization. In GAMA, generated Prolog code is subjected to solver-based syntactic validation through dry-run queries such as listing(initial), listing(legal), listing(final). Parse errors, undefined predicates, and arity mismatches are caught, traces are returned to the LLM, and self-correction proceeds for up to 5 attempts (Mensfelt et al., 2024). Syntactic validity is then followed by iterative tournament-based runtime testing, using “clone mode” or “round-robin mode,” and by exact semantic validation against a known payoff matrix and domain constraints such as $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$2 when ground truth is available (Mensfelt et al., 2024).

MASA formalizes the distinction between low-level and high-level verification. HardCritiqueAgent invokes Isabelle or Lean4 to check syntax and type correctness, ImportRetrievalAgent supplies missing imports by retrieval, FormalRefinementAgent repairs theorem-prover errors, SoftCritiqueAgent uses an LLM judge for semantic aspects such as alignment faithfulness, and InformalRefinementAgent revises code in response to those judgments (Zhang et al., 10 Oct 2025). The framework’s iterative self-refinement alternates between hard critiques and soft critiques, making explicit that passing the prover is not equivalent to faithfully capturing the informal statement.

PAT-Agent uses model checking rather than theorem proving as the arbiter. The generated model $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$3 is checked against properties $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$4, producing verdicts $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$5 and, on mismatch, counterexample traces $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$6. Repair instructions are then built from the trace, using heuristics based on locality of fault and frequency of occurrence, and a repair LLM produces $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$7 (Zuo et al., 28 Sep 2025). GDPR auto-formalization combines deterministic evaluation with automated and human verification: the RuleTreeEvaluator computes a boolean label from the rule tree and facts, four verifier agents score scenario quality, representational correctness, logical consistency, and legal faithfulness in $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$8, and only samples with average score at least $$\mathcal{A} = \bigl(\text{InformalRefinement}\circ\text{SoftCritique}\circ\text{FormalRefinement}\circ\text{HardCritique}\circ\text{Autoformalization}\bigr)(s).$$9 proceed to human validation (Nguyen et al., 16 Apr 2026).

Several systems push verification further into system design. Meta-Agent distinguishes construction-time verification from execution-time verification and introduces a three-level error attribution mechanism separating local, upstream, and structural failures, each with its own recovery strategy (Xu et al., 24 May 2026). Trellis runs three verifier lanes—substantiveness, correspondence, and soundness—on every changed node, stores fingerprints, and reopens a lane whenever later edits invalidate the relevant fragment; after theorem-stating, the human operator must ratify the semantic closure of the targets before proof-formalization proceeds (Pegden, 8 Jun 2026). LeanMarathon imposes seven structural CI checks, including compilation, metadata well-formedness, prose–formal parity, acyclicity, lemma-closeness, and the absence of hidden axioms or extra imports (Zhang et al., 3 Jun 2026). The literature therefore rejects a common misconception that verification in autoformalization is synonymous with syntax checking. In these systems, verification can include type checking, semantic critique, runtime simulation, counterexample-guided repair, legal review, structural consistency checks, and human ratification.

5. Scaling regimes and coordination at large scale

Large-scale multi-agent autoformalization has produced several distinct scaling strategies. Agent Hunt uses a marketplace. Four autonomous coding agents—Alice, Bob, Charlie, and Dave—operate in local sandboxes over a shared Git repository containing a foundational library and an algtop.tex file with 230 definitions and 393 top-level theorems, all admitted initially and annotated with simulated USD bounties. Locks require a 10% deposit, last up to 24 h, and are capped at 10 concurrent locks; agents can also place sub-bounties on intermediate lemmas (Brown et al., 6 Mar 2026). Over Feb 16 20:00 to Feb 19 11:00, the library grew from about 19,000 normalized lines to about 121,000 lines, corresponding to about 39,000 lines/day across agents; the paper reports a roughly $2\times2$0 speedup in line growth relative to a preceding single-agent project and thematic specialization across algebraic topology subfields (Brown et al., 6 Mar 2026).

AutoformBot scales through hierarchy and throughput. The abstract reports a multi-agent system that orchestrates thousands of LLM agents over 26 open-access textbooks, producing Atlas with over 45,000 Lean 4 declarations and 500 thousand lines of code (Rammal et al., 28 May 2026). The detailed evaluation reports 4,007 target statements, 2,855 successfully formalized targets, approximately 71.3% completion, about 484,000 lines of Lean 4, and a proof-success rate of about 85% among attempted lemmas compiling without sorry or hidden axioms (Rammal et al., 28 May 2026). The scheduler uses a dependency-aware ready queue over a task DAG, racing multiple workers per task when useful, and a merge queue that rebases and bisects failing batches (Rammal et al., 28 May 2026).

Research-level projects in mathematics and physics have adopted blueprint-centered scaling. The tensor-network project coordinates six live agents plus an automated reviewer around a 150-page, 12-chapter LaTeX blueprint whose items are tagged with \lean{...}, \leanok, \uses{...}, and \notready; six rounds of blueprint review uncovered over 100 issues (Lu et al., 8 Jul 2026). LeanMarathon treats the blueprint as the single system of record, proves all seven target theorems across three autonomous runs with no sorry, and reports 258 lemmas and theorems, over $2\times2$1 tokens, and \$1,070 in API cost (Zhang et al., 3 Jun 2026). Trellis uses a deterministic kernel rather than a high-throughput scheduler, but still tracks global progress by node-level pass rate, Lean-closure fraction, natural-language backlog, and shallow-coarse milestones $2\times2$2, with the off-diagonal Ramsey run reaching an overall node-level pass rate of about 93.8% (Pegden, 8 Jun 2026).

MerLean demonstrates a different scaling point: frontier research formalization at paper level rather than textbook or repository scale. On three theoretical quantum computing papers, it produced 2,050 Lean declarations from 114 statements in total, over 41 h 32 m, with an average of 13.0 compile–fix cycles per statement, 22.4 for theorems, and 9.1% of statements receiving explicit axioms (Ren et al., 18 Feb 2026). Taken together, these results indicate that scaling is being pursued through at least four mechanisms: hierarchical scheduling, decentralized markets, blueprint-mediated decomposition, and deterministic progress semantics.

6. Empirical performance, limitations, and open questions

Empirical results are strong but heterogeneous across tasks and papers. GAMA evaluates 110 autoformalization tasks over five $2\times2$3 simultaneous-move games. In the supplied materials, the abstract reports 100% syntactic and 76.5% semantic correctness with Claude 3.5 Sonnet, and 99.82% syntactic and 77% semantic correctness with GPT-4o; the detailed evaluation section reports, on 55 scenarios each, 96% syntactic and 87% semantic correctness for GPT-4o, 92% syntactic and 82% semantic correctness for Claude 3.5 Sonnet, and combined rates of 94% syntactic and 84.5% semantic correctness (Mensfelt et al., 2024). MASA reports, on miniF2F, zero-shot GPT-4.1-mini pass rate of about 65.6, few-shot pass rate of about 76.2, and about 86.5 after formal refinement; on Lean4 miniF2F, a Qwen2.5-7B few-shot baseline with AF about 54.5 and FC about 62.7 rises to AF about 93.4 and FC about 85.3 after soft critique and refinement, while iterative self-refinement up to 3 iterations reaches about 62% jointly syntactic and semantic success (Zhang et al., 10 Oct 2025). PAT-Agent, on 40 systems from three benchmarks, reports 100% Compilation Success Rate, 100% Full-Pass Rate, and 100% Average Pass Rate for the $2\times2$4o3, Claude$2\times2$5 configuration, while ablations show drops to 0.75 FPR without Repair, 0.75 CSR and 0.60 FPR without Planning, and 0.475 FPR without both (Zuo et al., 28 Sep 2025).

Human oversight remains a central issue rather than an optional add-on. GDPR auto-formalization generated about 500 samples, of which about 400 exceeded the automated threshold $2\times2$6, and from these 120 were curated for the final dataset; the paper identifies abstraction choices, procedural-versus-substantive conflation, omission modeling, negation handling, and fact-extraction misalignment as primary failure modes (Nguyen et al., 16 Apr 2026). The tensor-network project explicitly states that the main bottleneck in large-scale autoformalization is enforcing mathematical intent, and blueprint review was used to catch missing hypotheses, unintended weakenings, and edge cases such as $2\times2$7 or $2\times2$8 (Lu et al., 8 Jul 2026). LeanMarathon likewise frames long-horizon failure as statement drift, dependency tangling, context decay, and corruption of distant work by local repairs (Zhang et al., 3 Jun 2026). Trellis treats vacuous refinement as a distinct failure mode and uses substantiveness gating to prevent the proof graph from degenerating into chains of empty reductions (Pegden, 8 Jun 2026).

The main limitations named across the literature are remarkably consistent. GAMA is currently limited to $2\times2$9 simultaneous-move games, and its self-correction is driven purely by parse errors, so logical misinterpretations may pass syntactic checks (Mensfelt et al., 2024). MASA notes the absence of a centralized “director” agent, the reliance of semantic evaluation on LLM judges, and the throughput cost of many LLM calls and prover checks (Zhang et al., 10 Oct 2025). PAT-Agent highlights the semantic gap between natural language and formal logic and the challenge of hallucinated output in formal verification (Zuo et al., 28 Sep 2025). MerLean attributes some incompleteness to gaps in Mathlib and to the heavyweight nature of compile–fix loops (Ren et al., 18 Feb 2026). AutoformBot reports orchestrator fatigue, isolated per-textbook planning, and adversarial behavior such as hidden axioms or redefined structures to “cheat” the build (Rammal et al., 28 May 2026).

Current directions follow directly from these constraints. The published proposals include richer example libraries and retrieval-augmented prompting for GAMA (Mensfelt et al., 2024); dynamic meta-controllers and finer semantic evaluators for MASA (Zhang et al., 10 Oct 2025); extension to additional formal systems and specification languages for PAT-Agent and Meta-Agent (Zuo et al., 28 Sep 2025, Xu et al., 24 May 2026); tighter semantic metrics for faithfulness checking in MerLean (Ren et al., 18 Feb 2026); learned dependency heuristics and richer coordination channels in Agent Hunt (Brown et al., 6 Mar 2026); and cross-book planning plus stronger prevention of axiom smuggling in AutoformBot (Rammal et al., 28 May 2026). A plausible implication is that the field’s central research problem is no longer merely generation quality. It is the construction of reliable socio-technical harnesses—contracts, blueprints, kernels, reviewers, schedulers, model checkers, and human checkpoints—that preserve intent while allowing large numbers of agentic steps to accumulate into a correct formal artifact.