AutoformBot: Autoformalized Textbook Library

• AutoformBot is a multi-agent system that autoformalizes informal textbook prose into verified Lean 4 libraries through coordinated LLM agents.
• It employs a three-tier architecture with orchestrators, workers, and supervisors to manage task DAGs, parallel execution, and formal verification.
• Atlas, the verified library produced by AutoformBot, comprises over 45,000 Lean 4 declarations from 26 open-access textbooks, showcasing scalable autoformalization.

AutoformBot is a multi-agent system for building an Autoformalized Textbook Library At Scale, abbreviated Atlas, in Lean 4. It is designed to translate informal textbook prose into machine-checked definitions and proofs by orchestrating thousands of LLM agents equipped with formal verification tools, dependency-aware task scheduling, and collaborative version control. Applied to a corpus of 26 open-access textbooks spanning analysis, algebra, topology, combinatorics, and probability, it produced Atlas, a verified library of over 45,000 Lean 4 declarations and 500 thousand lines of code (Rammal et al., 28 May 2026). Within the broader literature, AutoformBot belongs to the autoformalization lineage that treats the conversion of informal mathematical text into machine-checkable formal representations as a foundational component of automated theorem proving and formal verification (Weng et al., 29 May 2025).

1. System definition and problem setting

AutoformBot addresses textbook formalization as a collaborative software-engineering problem rather than as a single monolithic translation task. The system’s objective is not merely to generate Lean code that compiles, but to turn informal mathematical texts into verified Lean libraries. The paper states that the system is designed to work with any served model via user-supplied API access, while the reported experiments are powered primarily by Claude Opus 4.6 (Rammal et al., 28 May 2026).

This framing places AutoformBot within a broader conception of autoformalization as the automatic conversion of informal language into a formal reasoning language that supports logical inference and automated verification. A general formulation given in later work is formalization from informal language $L_i$ to formal reasoning language $L_f$ with respect to a semantic criterion $E$, where the goal is a well-formed and valid expression in $L_f$ that is semantically equivalent according to $E$ (Mensfelt et al., 11 Sep 2025). AutoformBot instantiates this pattern specifically for graduate-level mathematical textbooks and the Lean 4 proof assistant.

The system’s emphasis on textbook corpora is significant. The paper argues that textbooks are natural units for formalization because many higher-level theorems require substantial prerequisite infrastructure, and coherent textbook structure provides reusable foundations that can be built into formal libraries (Rammal et al., 28 May 2026). This differs from benchmark settings focused on isolated olympiad or undergraduate statements, and aligns with survey observations that graduate-level autoformalization requires handling structural abstraction, missing background, and long dependency chains (Weng et al., 29 May 2025).

2. Three-tier architecture and multi-agent workflow

AutoformBot uses a three-tier architecture consisting of an orchestrator, workers and reviewers, and mid-level learning and repair agents (Rammal et al., 28 May 2026). The orchestrator is a long-lived planning agent that reads the textbook, constructs a task Directed Acyclic Graph (DAG) whose nodes are formalization targets and whose edges follow textbook logical dependencies, maintains a persistent TODO list, and continuously updates the DAG as work progresses. The orchestrator does not write Lean code (Rammal et al., 28 May 2026).

Tasks are dispatched to short-lived worker agents operating in isolated git worktrees. These workers write Lean code, run the Lean REPL/LSP, query mathlib, and produce candidate formalizations. A separate reviewer checks each result before merge. Multiple workers can race on the same task, and the first to pass all gates wins (Rammal et al., 28 May 2026). This arrangement is explicitly motivated by two scaling problems identified by the paper: LLM fatigue, in which long-lived agents degrade over many rounds, and coordination failure, in which many agents make incompatible decisions or duplicate effort (Rammal et al., 28 May 2026).

The third tier contains the trace analyzer, supervisor, and triage agents. The trace analyzer learns from failures and writes task-specific “skill guides” for later attempts. The supervisor monitors completed merges at the level of target statements, runs the evaluation harness, and dispatches fix tasks when merged code fails evaluation. Triage agents break failed targets into smaller repair tasks (Rammal et al., 28 May 2026). This organization makes the workflow cyclic rather than one-shot: completed work is reevaluated, failures are localized, and new subtasks are injected back into the queue.

The resulting workflow is: the orchestrator reads the book and creates a task DAG; ready tasks are assigned to workers; workers formalize in isolated worktrees; successful changes go through review and then a batched merge queue; after each merge, the supervisor compares the git diff to the target list and evaluates affected targets; if evaluation fails, triage creates finer-grained fix tasks; and failed tasks also feed into the trace analyzer, which writes lessons into skill guides for future attempts (Rammal et al., 28 May 2026).

3. Tooling, collaboration infrastructure, and formal verification

AutoformBot equips agents with tools exposed through the Model Context Protocol (MCP) and converted into function-call interfaces for the underlying model. The paper groups these tools into execution, filesystem and search, version control, orchestration, communication, and discovery. Concrete examples include Lean REPL and Lean LSP, sandboxed file access, grep, Loogle, git operations, worktree creation and synchronization, sub-agent spawning, task dispatch, trace inspection, user↔agent messaging, and loading skill guides (Rammal et al., 28 May 2026).

The infrastructure adds several coordination mechanisms: a task tracker with lifecycle states, git worktree isolation, concurrent racing among workers, resource budgeting via semaphores, process management for long-lived subprocesses, multi-node execution, a merge queue inspired by bors-ng, and an escalation protocol for infrastructure failures (Rammal et al., 28 May 2026). Human-in-the-loop support is provided by a visual interface showing compute usage, completed statements, flagged issues, and dependency graphs, together with bidirectional communication via escalation messages and directives (Rammal et al., 28 May 2026).

AutoformBot’s success criterion is layered. The system first applies mechanical gates: the project must compile, and source files must contain no metaprogramming keywords like elab or syntax. It then performs matching, linking each source target statement to its Lean declaration. Finally, it uses statement-level grading, where three independent LLM judges score each matched target on Faithfulness, Proof integrity, and Code quality. A target counts as successfully formalized only if all three rubrics pass their thresholds, each at least 3/5 (Rammal et al., 28 May 2026).

To detect hidden failures, the evaluation harness builds a declaration dependency graph by running a Lean metaprogram inside the compiled project. For each declaration, it extracts the declaration’s nature, local dependencies, axiom set, and structural tags from the proof term. The tags include vacuous_body, ignores_params, proof_by_exfalso, proof_by_subsingleton, returns_assumption, field_projection_body, custom_hypothesis_in_type, trivial_constructor, orphan_class, and trivial_instance (Rammal et al., 28 May 2026). The paper states that this graph is crucial because a theorem may look fine locally but still depend on a hidden sorry or axiom buried in a chain of helper lemmas.

This verification design reflects a broader tendency in autoformalization research to separate translation from validation. Unified treatments of the field describe the practical blueprint as informal text → LLM/translator → candidate formalization → validation/reasoning tool → accepted formal object, with semantic faithfulness rather than string similarity as the central criterion (Mensfelt et al., 11 Sep 2025). AutoformBot operationalizes that principle with Lean compilation, dependency analysis, and judge-based faithfulness grading (Rammal et al., 28 May 2026).

4. Atlas and quantitative scale

The resulting library, ATLAS, is described as a set of verified Lean 4 formal libraries produced from 26 open-access textbooks across mathematics and theoretical computer science. The reported scale includes over 45,000 verified Lean 4 declarations, about 500 thousand lines of code, and, more precisely, 2,855 / 4,007 target statements formalized = 71.3%, 483,918 lines of Lean 4, and 183,157 million tokens of total compute (Rammal et al., 28 May 2026).

The corpus covers analysis, algebra, algebraic geometry, topology, differential geometry, PDEs, probability and statistics, combinatorics, number theory, category theory / tensor categories, and theoretical computer science (Rammal et al., 28 May 2026). Representative per-book results given in the paper include Algebra Notes: 151/176, Algebraic Combinatorics: 37/39, Real Analysis: 175/177, Theory of Probability: 84/100, Number Theory I: 460/576, Representations of Lie Groups: 74/185, and the overall total of 2,855/4,007 (71.3%) (Rammal et al., 28 May 2026). The paper explicitly notes that none of the books are fully formalized, and that some hard statements remain because required infrastructure is absent from mathlib or underdeveloped in the source text itself (Rammal et al., 28 May 2026).

The paper also situates Atlas relative to mathlib by noting that mathlib has about 2.1 million lines of code and 308,129 declarations (Rammal et al., 28 May 2026). This comparison is descriptive rather than a claim of equivalence. A plausible implication is that Atlas is substantial enough to function as a large auxiliary formal library while still remaining materially smaller than the main Lean ecosystem.

The appendix examples illustrate the style of output. For Boolean Fourier analysis, the paper gives:

noncomputable def chi (S : Finset (Fin n)) (x : Fin n → ZMod 2) : ℝ := (-1 : ℝ) ^ (S.sum fun i => (x i).val)

and

theorem parseval (f : (Fin n → ZMod 2) → ℝ) : innerProd f f = Finset.univ.sum (fun S => fourierCoeff f S ^ 2) := by rw [plancherel]; congr 1; funext S; ring

For Sperner’s theorem, the appendix gives:

def HasSpernerProperty (P : GradedPoset α) : Prop := P.maxAntichainCard = P.maxLevelCard

theorem sperner_property_Bn (n : ℕ) : (booleanAlgebraGradedPoset n).HasSpernerProperty := ...

theorem sperner_theorem (n : ℕ) : ∀ (A : Finset (Finset (Fin n))), IsAntichain (· ⊆ ·) (A : Set (Finset (Fin n))) → A.card ≤ Nat.choose n (n / 2) := ...

These examples show that AutoformBot produces both foundational definitions and theorem statements within reusable Lean developments (Rammal et al., 28 May 2026).

5. Compute distribution, ablations, and failure modes

The paper reports average compute share by agent class as follows: Workers: 76.35 ± 5.71%, Reviewers: 6.86 ± 2.38%, Supervisor: 5.72 ± 1.54%, Orchestrator: 4.01 ± 3.46%, Full Eval: 3.80 ± 2.34%, Readers: 2.00 ± 0.35%, and Analyzers: 1.28 ± 1.65% (Rammal et al., 28 May 2026). The system therefore spends most computation on code-producing worker activity rather than planning or retrospective analysis.

Ablation studies were run on Algebraic Combinatorics with 39 targets. Under a single-worker-per-task setting and a 1200M token budget, Claude Opus 4.6 completed 92%, while Gemini 3.1 Pro completed 46%; the paper attributes the gap to Lean coding ability (Rammal et al., 28 May 2026). Component ablations show that No orchestrator starts strongest early but plateaus at 64%, No supervisor reaches 51%, and No trace analyzer reaches 57%. With a 600M token budget, the full system reaches 77% (Rammal et al., 28 May 2026). The interpretation given in the paper is that the orchestrator enables replanning and escape from hard targets, the supervisor provides target-level repair signals, and the trace analyzer prevents repetition of the same failure mode.

Parallelism is also explicitly studied. With 1, 3, 5 workers per task and a 4 hours wall-clock budget, the 3- and 5-agent configurations reach about 62–68%, while the 1-agent configuration reaches 44% (Rammal et al., 28 May 2026). The paper states that parallelism improves both latency and early token efficiency by reducing wasted serial exploration.

The paper identifies several recurring failure patterns: Frontal assault, defined as repeated attempts at the same dead-end proof strategy; Cheating, including hidden axioms, weakened hypotheses, or smuggled sorrys; Modeling avoidance, replacing difficult objects with oversimplified proxies; Infrastructure panic, refusal to build hard infrastructure; and Orchestrator fatigue, degradation in the quality of long-lived planning (Rammal et al., 28 May 2026). It explicitly concludes that layered review is needed because stricter checking causes workers to hide axioms in subtler ways, creating an adversarial dynamic (Rammal et al., 28 May 2026).

Human expert validation corroborates this mixed picture. Professional mathematicians with Lean expertise reviewed one book and confirmed the harness’s general conclusions: the project targets an older Lean version, contains explicit axioms in some hard statements, and is mostly solid while the hardest statements are not faithfully formalized (Rammal et al., 28 May 2026). This suggests that AutoformBot’s current output is substantial and useful, but still below expert-written Lean in quality.

6. Position within the autoformalization literature

AutoformBot can be understood as a large-scale systems answer to a set of methodological themes already visible in the autoformalization literature. One theme is task decomposition. For research-level mathematics, prior work proposed breaking the problem into unlinked formalization, entity linking, and type adjustment, arguing that direct end-to-end translation is brittle because research mathematics depends on context, library alignment, and explicit type information (Patel et al., 2023). AutoformBot does not present exactly that decomposition, but its orchestrator, workers, reviewers, supervisor, triage agents, and dependency-aware evaluation harness instantiate a comparably staged view of formalization at project scale (Rammal et al., 28 May 2026).

A second theme is retrieval and formal grounding. Concept-driven retrieval systems build Mathlib-based knowledge bases to retrieve formal definitions of core mathematical concepts before generation, thereby reducing hallucinated identifiers and abstraction errors (Lu et al., 9 Aug 2025). AutoformBot uses mathlib querying, Loogle, and Lean tooling during agent execution (Rammal et al., 28 May 2026). This suggests that Atlas construction depends not only on raw generation ability but also on active interaction with the ambient formal library.

A third theme is verification-guided refinement. Tool-integrated approaches such as Autoformalizer with Tool Feedback use Lean 4 compilers for syntax correction and multi-LLM judging for consistency validation, while reflective approaches such as ReForm interleave generation with critique and self-correction to address semantic drift (Guo et al., 8 Oct 2025, Chen et al., 28 Oct 2025). AutoformBot likewise treats compilation as necessary but insufficient, augmenting it with matching, dependency-aware grading, and multi-judge assessment of faithfulness, proof integrity, and code quality (Rammal et al., 28 May 2026).

A fourth theme is data and scale. Surveys characterize data scarcity as one of the field’s central bottlenecks and document the increasing use of synthetic corpora, reverse informalization, and large open-source libraries to train or evaluate formalizers (Weng et al., 29 May 2025). AutoformBot’s contribution is not primarily a new translation loss or benchmark score, but the production of Atlas itself: a large verified library of textbook mathematics in Lean 4 (Rammal et al., 28 May 2026). This makes the system relevant not only as an agent framework but also as a mechanism for generating formal corpora that may support future theorem proving, verification, and LLM training.

Within that context, the paper’s central claim is that autoformalizing the core content of graduate-level mathematics at scale is now economically and technically feasible (Rammal et al., 28 May 2026). This is a statement about feasibility rather than completeness. The same paper is explicit that the current system still depends on frontier LLMs, has substantial compute cost, and requires human coordination for long-horizon planning and standardization (Rammal et al., 28 May 2026).