--- title: 'WybeCoder: Agentic Code Verification' url: https://www.emergentmind.com/topics/wybecoder type: topic --- # WybeCoder: Agentic Code Verification WybeCoder is an agentic code verification framework that implements a prove-as-you-generate development paradigm for imperative program synthesis and verification. In WybeCoder, code, invariants, and formal proofs co-evolve within a hybrid verification stack. The system unifies automatic verification condition generation, SMT-based discharge, and interactive proof construction in Lean, orchestrated through a multi-agent LLM workflow. WybeCoder establishes new state-of-the-art results on imperative verification benchmarks by scaling subgoal decomposition and agentic refinement, enabling the synthesis of large verified code corpora and demonstrating that co-evolution of code, specifications, and proofs is tractable and effective [2603.29088]. ## 1. Hybrid Verification Stack and Agentic Workflow WybeCoder is built on a closed-loop agentic architecture that decomposes verification into sequential stages: verification-condition generation (VCG), SMT-based auto-active proof search, and interactive proof completion by dedicated LLM agents. The stack operates as follows: - Imperative methods are annotated with preconditions, postconditions, and invariants in Lean4 via the Loom/Velvet plugin. - VCG translates the annotated method into first-order proof obligations (VCs). - Each VC is first discharged by the CVC5 SMT solver (auto-active phase). - Unsolved VCs become Lean subgoals, each targeted by an LLM-driven prover agent operating as an individual thread. - Proof scripts resolving subgoals are accumulated; if a subgoal is flagged as unprovable (CHANGE), a method-modification agent proposes revised code or strengthened/weakened invariants. - This process (VCG → SMT → Lean/agentic proof → feedback-driven method modification) is iterated until either all goals are discharged or a compute budget is exhausted. The agentic loop continuously co-evolves the program, its specifications, and the associated proofs, allowing specification and implementation changes to propagate adaptively in response to proof failures. ## 2. Verification Conditions and Formal Encodings WybeCoder operates over the classical Hoare logic paradigm, targeting verification tasks of the form ${\{P\} C \{Q\}}$, where $P$ is the precondition, $C$ the imperative program, and $Q$ the postcondition. The verification condition is formalized as $$ \mathrm{VC}(P,C,Q):=\forall s,s'.\bigl(P(s)\wedge\mathit{exec}(C,s,s')\bigr)\to Q(s'). $$ VCG produces obligations for sequential composition, branching, and loops: - For $C_1;C_2$, VCG generates $\mathrm{VC}(P,C_1,Q)$ and $\mathrm{VC}(Q,C_2,R)$. - For $\mathbf{while}\;B\;\mathbf{do}\;C$ with invariant $I$, the obligations are: $$ \forall s,s'.\, (I(s)\wedge B(s)\wedge \mathit{exec}(C,s,s')) \to I(s') $$ $$ \forall s.\, I(s)\wedge \neg B(s)\to Q(s) $$ In Velvet, the Lean verification DSL, users specify methods and invariants as follows: ```lean method InsertionSort (a : Array Int) return (out : Array Int) require 0 ≤ i ∧ i ≤ a.size ensures a.size = out.size ensures List.Sorted (· ≤ ·) out.toList ensures List.Perm a.toList out.toList do var i := 1 invariant 1 ≤ i ∧ i ≤ a.size + 1 invariant List.Sorted (· ≤ ·) a.toList[0:i] ... ``` Each `invariant` is translated into corresponding VCs, and the global method-level specs become top-level obligations. ## 3. Subgoal Decomposition and LLM Multi-Agent Loop The core workflow of WybeCoder is encapsulated in a subgoal-decomposition multithreaded LLM loop, formalized in Algorithm 1. ``` Algorithm 1: Subgoal-Decomposition Proof System Input: initial method M₀, LLM agent M, budget N method ← M₀ for t in 1..N: goals ← extract_goals(method) # VCG + SMT attempt results ← parallel_map(goals, λg. M.prove_subgoal(g)) if all r in results are SUCCESS: return reconstruct_proof(results) else if exists r in results with r.CHANGE: changes ← {r | r.CHANGE} method ← M.update_method(method, changes) else: return FAILURE return FAILURE ``` - `extract_goals` uses Loom/Velvet for VCG; CVC5 handles direct SMT discharge; unproven VCs are returned as Lean subgoals. - `M.prove_subgoal` invokes an LLM per goal, providing the goal context and current source; outputs are proof scripts (SUCCESS), CHANGE (unprovable given current invariants), or FAIL. - CHANGE tags initiate the method-modification agent to propose new invariants or code rewrites. - After all subgoals are discharged, proof scripts are collated via `reconstruct_proof`. This division of proof labor and refinement aligns with the prove-as-you-generate philosophy, drastically reducing bottlenecks encountered in traditional monolithic verification flows. ## 4. Benchmarks, Quantitative Results, and Case Studies WybeCoder's empirical evaluation targets two Lean verification benchmarks, "Verina-Loom" and "Clever-Loom," both mechanically or semi-automatically translated to imperative method specs in Velvet. For competitive integrity, imperative implementations are required—direct calls to pure functional reference implementations are not allowed. Evaluatee models include GPT-5, Gemini 3 Pro, and Claude 4.5 Opus, orchestrated in a $32\times16$ multi-turn agentic setting. The results are tabulated below: | Benchmark | GPT-5 | Gemini 3 Pro | Claude 4.5 Opus | |--------------|----------|--------------|-----------------| | Verina | 64.6% | 55.6% | **74.1%** | | Clever | 53.8% | 32.8% | **62.1%** | A detailed Heapsort case study decomposes the algorithm into three methods—`Heapify`, `BuildMaxHeap`, and `HeapSort_main`—each with precise pre/post-conditions for size, permutation, and heap/sortedness invariants. Agent activity included: - Heapify: 215 prover agents, 27 new invariants - BuildMaxHeap: 8 provers, 5 invariants - HeapSort_main: 134 provers, 12 invariants ~2000 lines of Lean-verified code were generated. Invariant and VC examples include: ```lean invariant ∀ j < i, a.toList[j] ≤ a.toList[parent(i)] ``` $$ \forall s,s'.\left(I(s)\wedge B(s)\wedge\mathit{exec}(\mathit{siftDown},s,s')\right)\to I(s') $$ All 27 subgoals for `Heapify` were closed within 3 refinement cycles. `HeapSort_main`’s top-level theorem required both CVC5-automated and four interactive proof steps in Lean. ## 5. Scalability and Impact on Verified Code Synthesis WybeCoder demonstrates superior scaling compared to prior systems. Unlike earlier approaches exhibiting pass@k performance plateaus (notably around pass@64), continuous improvement is observed as LLM call budgets are increased, with frontier models improving well beyond $10^4$ agentic calls. The multi-agent scaffold consistently outperforms single-agent baselines (e.g., +5–10% solve-rate for Gemini 3 Pro at fixed compute expenditure). Crucially, by automating the full pipeline—specification, implementation, invariant synthesis, and proof construction—WybeCoder enables the machine-checked generation of large corpora of verified imperative code. Such datasets are instrumental for the self-supervised or RL-based fine-tuning of formal-verification LLMs, promising to dramatically reduce the manual effort typically required to develop verified code for complex, real-world software systems. A plausible implication is the broader tractability of verified software synthesis for large-scale systems and next-generation programming language tooling [2603.29088]. ## 6. Summary and Outlook WybeCoder operationalizes a prove-as-you-generate paradigm, integrating SMT auto-active verification with interactive Lean proofs, all orchestrated by a multi-agent LLM workflow. On both "Verina" and "Clever" benchmarks of nontrivial imperative algorithms—such as selection sort, Kadane’s, and heapsort—WybeCoder solves 74% and 62% of tasks, respectively. The system demonstrates that code, invariants, and proof scripts can co-evolve stably and scalably under iterative, agentic refinement. This suggests a pathway to automated construction of massive verified codebases and enhanced formal verification capabilities for large-scale, safety-critical software. Source: https://www.emergentmind.com/topics/wybecoder