AxDafny: AI-Extended Verified Programming in Dafny
- AxDafny is an AI-extended verified programming framework that jointly synthesizes executable Dafny code and proof artifacts to meet strict verification conditions.
- It employs a generate-check-repair architecture using SMT-based verification, deterministic checks, and iterative feedback to refine invariants, assertions, and termination measures.
- Evaluated on DafnyBench and LCB-Pro-Dafny, AxDafny significantly outperforms earlier annotation-only tools by achieving higher verification accuracy and effective proof-hint repair.
AxDafny denotes an AI-extended, verifier-guided approach to verified programming in Dafny in which a model must generate not only executable code but also the proof artifacts required for the Dafny verifier to accept the program. In the literature, the term has both a broad and a specific sense: earlier work used it to characterize an emerging “AI-extended Dafny” ecosystem centered on annotation synthesis and verifier-guided data generation, whereas the 2026 system “AxDafny: Agentic Verified Code Generation in Dafny” names a concrete agentic framework for joint code-and-proof synthesis in Dafny (Poesia et al., 2024, Breen et al., 30 Jun 2026).
1. Problem setting and conceptual scope
AxDafny addresses a harder problem than conventional code generation. In Dafny, a verified method typically requires executable code, requires and ensures clauses, loop invariants, assertions, and termination arguments; verification usually fails unless sufficient invariants, assertions, and measures are given (Breen et al., 30 Jun 2026). The system therefore targets joint synthesis of code and proofs: it must satisfy a trusted specification without weakening that specification, and it must do so in a language whose verification conditions are discharged by an SMT-based backend.
The 2026 AxDafny paper defines “agentic code generation” as a multi-step process in which an LLM proposes a candidate Dafny file, external tools check it, and the system repairs it iteratively using feedback and persistent memory (Breen et al., 30 Jun 2026). This differs from earlier Dafny annotation tools that assume the implementation and top-level contracts are already given. In that earlier line of work, “AxDafny” was explicitly described as “an AI-extended Dafny,” with the main immediate target being automatic insertion of logical annotations such as assert, invariant, and decreases until the standard Dafny verifier can prove the method correct (Poesia et al., 2024).
A central difficulty is that verification and testing expose different failure modes. A Dafny program can be functionally correct with respect to its specification yet still be rejected because the proof obligations are underspecified, and conversely a program that passes runtime tests may not satisfy the full universally quantified correctness conditions encoded in the verifier (Breen et al., 30 Jun 2026). This makes AxDafny an overview framework over specifications and proof obligations, not merely over input-output behavior.
2. Agent architecture and verifier-guided repair
The concrete AxDafny system is organized around three components: a proposer, a reviewer system, and a memory or context manager (Breen et al., 30 Jun 2026). The proposer is the main LLM agent and outputs complete Dafny files; it is instructed to preserve the given specifications, fill in implementations, and repair both executable logic and proof artifacts. The reviewer system combines deterministic checks, Dafny verification, and an LLM reviewer. Deterministic checks enforce spec preservation and forbid proof-bypass patterns such as {:axiom}, {:verify false}, assume, and {:extern} in sensitive contexts. Dafny verification then runs dafny verify, optionally with counterexamples, and returns diagnostics such as unmet preconditions, violated postconditions, broken invariants, or non-termination. A secondary LLM reviewer is used to detect more subtle cheating, such as rewriting predicates into trivial truths.
The repair loop is iterative. For up to 20 iterations in the reported experiments, the proposer receives the task description, the current or previous Dafny file, verifier diagnostics, and a persistent scratchpad summary; it then emits a new complete Dafny file. If deterministic checks or verification fail, diagnostics are fed back into the next iteration, and the scratchpad is updated by another LLM pass that merges current lessons with earlier notes (Breen et al., 30 Jun 2026). The framework does not decompose responsibility into separate invariant, assertion, or termination agents; instead, a single proposer is expected to infer from verifier feedback whether it should strengthen a loop invariant, add an intermediate assert, refactor the implementation, or repair a ranking function.
The result is a generate-check-repair architecture in which the Dafny verifier supplies the environment signal. That signal is richer than ordinary test outcomes because it localizes the failing obligation and its kind, allowing the model to target loop invariants, postconditions, recursive measures, or helper assertions with some precision (Breen et al., 30 Jun 2026).
3. Benchmarks, evaluation protocol, and empirical results
AxDafny is evaluated on two distinct benchmarks. The first is DafnyBench, which measures proof-hint capability on fixed Dafny programs. The second is LiveCodeBench-Pro-Dafny (LCB-Pro-Dafny), a benchmark of 250 competition-style programming problems translated into Dafny with formal specifications and a verifier-based evaluation harness (Breen et al., 30 Jun 2026). LCB-Pro-Dafny contains 100 easy, 100 medium, and 50 hard tasks; each instance includes the original natural-language problem statement, a Dafny method signature, and a formal specification. A solution is accepted only if it verifies in Dafny, preserves the given specification, definitions, and signatures, and avoids proof shortcuts such as assume, {:extern}, and {:verify false} (Breen et al., 30 Jun 2026).
| Benchmark | AxDafny result | Comparator |
|---|---|---|
| DafnyBench | 725 / 782 = 92.7% verified | DafnyPro 86.2% |
| LCB-Pro-Dafny | 56.4% overall verified | GPT-5.5 pass@1 11.6% |
On DafnyBench, AxDafny with Gemini-3.1-Pro verified 725 of 782 tasks, or 92.7%, outperforming the strongest previously reported proof-hint baseline by 6.5 percentage points (Breen et al., 30 Jun 2026). In the same evaluation, AxDafny with GPT-5.5 medium reached 88.9%, and GPT-5.5 low reached 85.3%. On LCB-Pro-Dafny, AxDafny achieved 75.0% verification on the easy split, 52.0% on the medium split, 28.0% on the hard split, and 56.4% overall, compared with 13.0%, 14.0%, 4.0%, and 11.6% respectively for GPT-5.5 pass@1 (Breen et al., 30 Jun 2026).
The iteration profile differs across the two benchmarks. On DafnyBench, gains are concentrated in the first 5 iterations, after which the curve flattens; on LCB-Pro-Dafny, verification accuracy increases more steadily with additional iterations, reflecting the greater difficulty of end-to-end synthesis relative to proof-hint repair (Breen et al., 30 Jun 2026).
A further result is that verification success and executable competitiveness are not equivalent. For verified LCB-Pro-Dafny solutions compiled to Python and run against the original LiveCodeBench-Pro tests, the easy split produced 75 verified tasks, of which 32 passed the original tests, 39 were Time Limit Exceeded, and 4 were Memory Limit Exceeded; the medium split produced 52 verified tasks, of which 6 passed tests, 44 were Time Limit Exceeded, and 2 were Memory Limit Exceeded (Breen et al., 30 Jun 2026). The paper explicitly concludes that verification success and runtime test performance measure different aspects of generated code.
4. Antecedents in AI-assisted Dafny
The immediate prehistory of AxDafny is the shift from annotation-only assistance to joint code-and-proof generation. The 2024 system dafny-annotator addressed the annotation bottleneck by using an LLM-guided greedy search that proposes candidate annotations, enumerates syntactically valid insertion points, and keeps the first annotation-location pair that Dafny accepts (Poesia et al., 2024). On the 83 DafnyBench test methods that fail after original annotations are stripped, a base LLaMa 3.1 8B model achieved 15.7% success, while fine-tuning on DafnyBench and the synthetic dataset DafnySynth raised success to 50.6% (Poesia et al., 2024). DafnySynth itself was built by an Edit Graph pipeline with an Idea Proposer, Idea Implementer, Annotator, and Change Proposer, and in the reported $10 GPT-4o run it produced 699 compilable Dafny programs, 46 fully verified programs, and 1107 annotations, of which 923 were syntactically unique after deduplication (Poesia et al., 2024).
A second antecedent is DafnyPro, an inference-time framework focused on annotation generation under the constraint that base program logic must not change. DafnyPro combines a parser-based diff-checker that rejects modifications to executable logic, a pruner that removes non-inductive or unnecessary invariants, and a hint-augmentation system that retrieves predefined proof strategies (Banerjee et al., 8 Jan 2026). On DafnyBench, Claude Sonnet 3.5 enhanced with DafnyPro achieved 86.18%, reported in the abstract as 86%, and the paper attributes a substantial part of the gain to hint augmentation (Banerjee et al., 8 Jan 2026). AxDafny surpasses this result on the same benchmark, but it does so while tackling the broader problem of synthesizing implementations as well as proof hints (Breen et al., 30 Jun 2026).
A related but distinct line of work treats Dafny as an internal verification-aware intermediate language rather than as the user-visible output. In that architecture, an LLM first generates a Dafny specification and implementation, the verifier checks them, and only then is verified Dafny compiled to a target language such as Python (Li et al., 10 Jan 2025). This does not name itself AxDafny, but it exhibits the same general pattern of using Dafny as the oracle-bearing layer between natural-language intent and executable code.
5. Dafny substrate and enabling infrastructure
AxDafny is built on top of a mature Dafny verification substrate whose relevant features predate AI-assisted synthesis. The Dafny integrated development environment introduced continuous processing over editor snapshots, asynchronous communication, Boogie task parallelism, multiple Z3 solver instances, dependency-based caching, hover-text exposure of computed metadata, and direct integration of the Boogie Verification Debugger (Leino et al., 2014). That paper does not mention AxDafny explicitly, but it identifies several mechanisms relevant to an AxDafny-style extension: Boogie-level checksums and custom attributes for caching, an asynchronous verification pipeline, hover-text infrastructure for computed proof information, generic counterexample visualization, and on-demand re-verification selective to changed entities (Leino et al., 2014). This suggests that AxDafny is not merely an LLM wrapper around a compiler invocation; it can be viewed as a higher-level controller layered over an already incremental, metadata-rich verifier pipeline.
Dafny’s refinement features provide a second foundation. Refinement modules permit staged development in which abstract specifications, unproved lemmas, or opaque functions are later strengthened or given bodies, subject to the Principle of Semantic Refinement (Koenig et al., 2016). That paper also does not mention AxDafny by name, but it explicitly notes that an “AxDafny” system would naturally exploit refinement to introduce axioms and abstract functions in early modules and later refine them with definitions and proofs (Koenig et al., 2016). A plausible implication is that AxDafny can be understood not only as an overview system for finished verified programs, but also as a tool for navigating staged, axioms-first development within Dafny’s own refinement discipline.
6. Limitations, trust assumptions, and future directions
The AxDafny paper identifies several limitations. The 20-iteration loop with large models is expensive; performance is sensitive to prompt structure and to how verifier diagnostics are presented; the system is tailored to Dafny syntax and verification behavior; and verification remains contingent on the quality of the trusted specification, since “verified” can still be misaligned with user intent if the specification is wrong (Breen et al., 30 Jun 2026). The hard split of LCB-Pro-Dafny remains notably difficult, with only 28.0% verified, and many verified solutions remain uncompetitive under time or memory limits because the specifications generally encode functional correctness rather than complexity (Breen et al., 30 Jun 2026).
Earlier AI-assisted Dafny work names additional missing capabilities. The annotation-centric systems assume that formal specifications already exist, and they do not synthesize full method contracts, ghost methods, lemmas, or datatype definitions (Poesia et al., 2024). Future directions explicitly include specification assistance, lemma recommendation and synthesis, richer search strategies, and scaling synthetic-data pipelines (Poesia et al., 2024). AxDafny itself points to more principled search, more structured feedback than raw textual diagnostics, tighter treatment of performance constraints, adaptation to other languages and logics, and training-time use of verification feedback as a reward signal (Breen et al., 30 Jun 2026).
A separate line of research raises a trust question about the underlying toolchain. “Verified VCG and Verified Compiler for Dafny” observes that standard Dafny’s compiler and static verifier are not proved correct and have had soundness bugs, and presents a mechanized functional semantics, verified verification condition generator, and verified compiler for an imperative subset of Dafny in HOL4 (Nezamabadi et al., 4 Dec 2025). This suggests that AxDafny’s guarantees depend not only on the quality of agentic repair, but also on the trustworthiness of the verifier and compiler beneath it. In that sense, the long-term trajectory of AxDafny spans two layers: increasingly capable agentic synthesis at the top, and increasingly foundational verification and compilation guarantees at the bottom (Breen et al., 30 Jun 2026, Nezamabadi et al., 4 Dec 2025).