Just Type It in Isabelle! AI Agents Drafting, Mechanizing, and Generalizing from Human Hints

Abstract: Type annotations are essential when printing terms in a way that preserves their meaning under reparsing and type inference. We study the problem of complete and minimal type annotations for rank-one polymorphic $λ$-calculus terms, as used in Isabelle. Building on prior work by Smolka, Blanchette et al., we give a metatheoretical account of the problem, with a full formal specification and proofs, and formalize it in Isabelle/HOL. Our development is a series of experiments featuring human-driven and AI-driven formalization workflows: a human and an LLM-powered AI agent independently produce pen-and-paper proofs, and the AI agent autoformalizes both in Isabelle, with further human-hinted AI interventions refining and generalizing the development.

• The paper introduces an AI-powered method for minimal type annotation, ensuring that Isabelle’s re-parsing uniquely reconstructs the original fully annotated term.
• It formalizes a reverse-greedy annotation-removal algorithm that guarantees both completeness and minimality of printed annotations.
• The work contrasts AI-led and human-led workflows, demonstrating effective autoformalization and novel abstraction through independence systems.

AI-Assisted Mechanization of Minimal Type Annotation in Isabelle

Introduction and Motivation

The paper "Just Type It in Isabelle! AI Agents Drafting, Mechanizing, and Generalizing from Human Hints" (2604.15713) rigorously formalizes the problem of minimal, sufficient type annotation for rank-one polymorphic $\lambda$-calculus terms, as required by the round-trip property of printing and parsing terms in proof assistants such as Isabelle. Beyond advancing the metatheoretical understanding of the annotation problem, the work innovates methodologically by systematically employing LLM-based agents for both informal proof drafting and formal verification via autoformalization in Isabelle/HOL, with explicit comparison to traditional human-led workflows.

Formalization of the Annotation Problem

The central technical issue is the computation of a correct printing of a fully annotated term, that is, a term whose type annotations ensure that reparsing and re-inference yields precisely the original term type with minimal redundant annotation, i.e., the annotation is minimal with respect to an appropriate partial order ($\compl$) on term decorations. The paper provides a comprehensive formal account of the type annotation lattice, term syntax with maybe-types, definitions of unambiguous and Church-typed terms, and the corresponding relations of annotation erasure, type variable coverage, and substitution.

A pivotal property is the uniqueness of well-typed completion for typable unambiguous C-terms, ensuring that minimal correct printings are well-defined and discoverable via local annotation manipulations.

The Smolka-Blanchette Algorithm and Metatheoretic Guarantees

Building on prior operational algorithms [smo_thesis, smobla_journal], the work formalizes a reverse-greedy annotation-removal algorithm. The algorithm iteratively removes type annotations from candidate positions provided their removal does not break coverage of all type variables necessary for the round-trip property. The algorithm operates by referencing a most general well-typed completion of the erasure of the original term, then testing each prospective removal for type variable coverage.

Two main theorems are formally established:

1. Completeness: The algorithm returns a correct printing, i.e., reparsing and inferring the type of the output determines the original fully annotated term uniquely.
2. Minimality: The output is minimal with respect to $\compl$ among all correct printings.

The formal proof features several technical innovations, such as a sandwich lemma for the interaction of annotation erasure, typing, and substitution, and strong congruence properties for coverage.

Workflows: Human Authorship Versus LLM Agents

The project contrasts a traditional human-driven lemma-proving workflow with an AI-powered approach based on an agentic coding environment integrating OpenCode, Claude 4.6-v1, and Isabelle/Q. The human-led development emphasized conceptual abstraction, whereas the LLM-driven process adhered more closely to the operational peculiarities of Isabelle/ML and existing implementations but, notably, converged to similar high-level concepts and correctness guarantees.

AI Paper Drafting and Revision

LLM agents iteratively drafted pen-and-paper proofs and specifications for the annotation problem, incorporating human feedback through review-based improvement cycles. While initial drafts sometimes restricted generality and omitted subtle definitions, subsequent revisions, augmented by self-review and simulated peer review, achieved high-fidelity representations of the core metatheoretic results.

AI-Authored Autoformalization in Isabelle

The autoformalization phase tasked the LLM agent with mechanizing both the AI-drafted and expert-authored pen-and-paper proofs as Isabelle/HOL developments.

Figure 1: AI-based annotation problem locale: well-formedness assumptions on the input $a$ as well as type inference in terms of $a_{star}$.

The resulting mechanizations accurately expressed the correctness and minimality theorems within Isabelle locales, making principled use of inductive predicates, explicit subterm annotation counting, and interactive proof tooling.

Human-Authored Autoformalization

The formalization of the human-led proof was characterized by slightly greater clarity in conceptual definitions and statement alignment, leading to better maintainability. Both workflows required non-trivial review to ensure faithful correspondence between paper-level statements and formal theorems, but the AI-led effort also introduced some technical misalignments, especially regarding the fixed versus abstract order of annotation positions.

Figure 2: Human-based signature locale: corresponds to the formalization of the principal type inference theorem for completions.

Generalization via Independence Systems: AI-Guided Abstraction

A final experiment provided the LLM agent with a human-supplied abstraction, reducing the annotation minimization problem to the theory of independence systems (ISs). The agent successfully mechanized the connection, leveraging standard greedy optimality theory, and automatically generated formal proofs within an existing Isabelle framework for matroidal optimization, demonstrating a substantial acceleration of the abstraction and formalization pipeline.

Figure 3: AI-generalized statement for induction proof: the proof of local minimality in the greedy algorithm required an auxiliary statement counting per-variable coverage across annotation positions.

Implications and Outlook

The paper's technical and methodological advances have broad implications:

• For interactive theorem proving: The results yield a principled and formally verified method for minimally annotating terms in Isabelle's infrastructure, impacting tooling such as Sledgehammer, proof translations, and program synthesis modules.
• For programming language metatheory: The AI-driven formalization demonstrates, in a high-value domain, that LLM agents can generate, refine, and check nontrivial metatheoretical developments aligned with best practices in formal methods.
• For autoformalization research: The documented experiments provide a concrete template for integrating LLMs in interactive proof assistant workflows, especially for domains involving intricate inductive definitions and algorithmic meta-properties.

Anticipated future directions include extension of the AI-augmented methods to richer type systems (such as System F), application to autoformalization at scale in the Archive of Formal Proofs, and further reductions in human review overhead via improved guidance and critique modeling.

Conclusion

This work rigorously advances the metatheory of minimal type annotation for polymorphic $\lambda$-terms in Isabelle, coupling formal specification, algorithmic insight, and mechanized proof. The explicit comparison between human and LLM-based authoring workflows substantiates the capacity of contemporary AI agents to complement expert mathematical and proof-engineering labor, both for drafting informal mathematics and for producing trusted formal artifacts. The successful use of abstraction hints to guide agent behavior illustrates a promising research avenue for programmatic, hint-driven autoformalization and theorem proving.