BMC-Agent: Hybrid Verification of LLM Code

• BMC-Agent is a hybrid verification framework that infers function contracts and applies bounded model checking for scalable and precise verification of LLM-generated code.
• It employs an LLM-based agent layer to propose semantic checks while a solver backend (CBMC for C and Kani for Rust) rigorously validates memory-safety and control-flow properties.
• Verification is organized compositionally by isolating each function and propagating refinements through the call graph to target both safety and functional correctness.

BMC-Agent is an instantiation of the agentic model checking paradigm introduced in "Agentic Model Checking" (Sun et al., 20 May 2026). It targets verification of LLM-generated systems code, where bugs are prevalent, formal specifications are missing, and safety contracts are encoded implicitly at call sites rather than enforced at function boundaries. The central design principle is “agents propose, solvers verify”: an LLM-based agent layer performs semantic judgments such as spec inference, check selection, counterexample classification, refinement proposal, and realism auditing, while a bounded model checking backend performs every soundness-relevant operation, including assume/assert injections, loop unrolling, memory-model encoding, and counterexample generation. In BMC-Agent, the backend is CBMC for C and Kani for Rust, and verification is organized compositionally so that each function is checked in isolation against an inferred contract (Sun et al., 20 May 2026).

1. Core architecture and verification paradigm

BMC-Agent combines a source-level analysis pipeline with a solver-backed verification loop. The high-level pipeline is: source code plus domain summary; call-graph builder followed by SCC condensation; spec generator producing DSL pre/post conditions and an optional functional_spec; flag selector producing per-function BMC flags; BMC harness synthesis and invocation of CBMC or Kani; and finally an outcome partition into Verified-clean (≤ k steps), Counterexample, or Inconclusive (Sun et al., 20 May 2026).

Its architectural commitments are explicit. First, agents handle everything requiring soft, semantic judgment, whereas the BMC solver discharges all numeric, memory-safety, and control-flow soundness decisions. Second, verification is compositional: each function is checked in isolation under assume-guarantee contracts. Third, counterexamples are not treated as immediate bug reports; instead, they enter a multi-stage validation pipeline and a CEGAR-style refinement loop at the specification level (Sun et al., 20 May 2026).

This organization is intended for codebases in which direct whole-program reasoning is costly or unstable. A plausible implication is that the decomposition by function boundary serves not only scalability but also auditability: specification changes are localized in the SpecStore, and refinements propagate structurally through the call graph rather than being handled by ad hoc suppression.

2. Specification inference and the restricted DSL

BMC-Agent infers per-function preconditions, postconditions, and optionally functional specifications in a restricted DSL designed so that every construct translates deterministically into the backend’s native assume/assert primitives (Sun et al., 20 May 2026). The grammar is given as:

$G$7

The translation is direct:

• requires E → assume(E)
• ensures E → assert(E)
• in C: __CPROVER_assume(E); __CPROVER_assert(E, “post”)
• in Rust: kani::assume(E); kani::assert(E)

An example given for kapi_file_size is:

$G$8

with the non-null branch rendered as:

$$\neg\mathsf{null}(node)\;\Rightarrow\;(\mathsf{result} \;=\;( \mathsf{node} \! \to \! \mathsf{size} ))$$

The specification mechanism is top-down: specifications are inferred from caller context rather than extracted only from local implementation structure. This suggests that BMC-Agent treats function interfaces as latent contracts recoverable from usage, while still constraining the recovered representation to a DSL whose compilation into backend primitives is deterministic and auditable.

3. Compositional checking and call-graph organization

The compositional procedure starts by building a call graph $G$, condensing SCCs, and topologically sorting layers $L_1,\dots,L_n$. For each layer, functions are processed in parallel. For a function $f$, BMC-Agent retrieves the stored spec $(\mathsf{Pre}_f,\mathsf{Post}_f)$, injects assume(\mathsf{Post}_g) for each callee $g$, injects assume(\mathsf{Pre}_f) before entry and assert(\mathsf{Post}_f) after return, encodes a harness, and runs the BMC backend under bound $k$. The outcome is recorded as VERIFIED-CLEAN, CEX, or INCONCLUSIVE (Sun et al., 20 May 2026).

The pseudocode is:

$G$9

When a counterexample arises for $f$, it is deduplicated across sibling functions by $(f,\text{property-type})$, validated, and either reported as a bug or used to drive refinement. If refinement updates $\mathsf{Pre}_f$ or $G$0, then $G$1 and all callers of $G$2 are re-enqueued for re-verification. The refinement propagation rule is stated as:

$G$3

This organization yields the paper’s stated scaling property: per-query cost scales with a single function’s state space, and refinements propagate automatically to callers (Sun et al., 20 May 2026). A plausible implication is that the method substitutes repeated bounded local proofs for a single global proof obligation, while preserving a disciplined dependency structure through the call graph.

4. Counterexample validation and refinement loop

A raw BMC counterexample for a function is not directly treated as a bug report. BMC-Agent subjects it to four validation stages (Sun et al., 20 May 2026).

Stage 1: Input Reachability. The counterexample inputs are propagated upwards through the call graph, checking at each caller whether the caller’s spec admits the witnessed values. If propagation reaches a no-caller entry function, system_entry = true; otherwise the propagation stop layer defines a weaker reachability tier.

Stage 2: Callee Feasibility. The real bodies of stubbed callees used in the original harness are inlined, and BMC is rerun over the restricted trace inputs. If the witness survives, it is symbolically feasible in the real code.

Stage 3: Dynamic Replay. A small host-side harness in C or Rust initializes concrete inputs from the counterexample, compiles, and runs under address sanitizers or signal handlers. If the harness signals SIGSEGV, SIGABRT, SIGFPE, or SIGILL, the result is upgraded to “confirmed by dynamic crash.”

Stage 4: Realism Audit. Deterministic pattern detectors first reject known model artifacts such as uninitialized globals and path-divergent unwinds. If no pattern matches, an LLM audit reviews the spec, source context, and witness, and classifies it as REALISTIC or UNREALISTIC; uncertain cases are treated as UNREALISTIC.

Only REALISTIC survivors become bug reports; UNREALISTIC outcomes trigger refinement (Sun et al., 20 May 2026). This treatment addresses a common misconception about formal bug-finding in LLM-generated code: a solver-produced witness is not necessarily an actionable defect. In BMC-Agent, the distinction between active in-tree crashes and latent public-API failures is operationalized by the validation pipeline rather than left to manual triage.

5. Functional-correctness clauses and bounded behavioural equivalence

BMC-Agent is not restricted to defensive safety properties such as panic-freeness or absence of UB. For pure algorithmic helpers, it can infer a functional_spec, which is AND-merged into the postcondition so that a single BMC run checks both defensive and behavioural obligations (Sun et al., 20 May 2026). Formally,

$G$4

The paper lists four template shapes for such clauses:

• Reference-equivalence: result = ref(inputs)
• Algebraic identities: invariant(result, inputs) = true
• Fold expressions: result = \bigl(\mathop{\mathrm{fold}_{x\in S} f(b,x)\bigr)
• Round-trip properties: decode(encode(x)) = x

An example is Rust’s align_up, whose functional specification is:

$G$5

Kani then checks these conditions under arbitrary inputs within unwind bounds (Sun et al., 20 May 2026). This extends the method from panic-freeness to bounded behavioural equivalence. A plausible implication is that BMC-Agent occupies an intermediate point between assertion-oriented bug finding and full deductive verification: it remains bounded and backend-driven, but it can validate semantically rich functional properties when those properties fit the inferred templates.

6. Empirical results, scope, and significance

BMC-Agent was evaluated on four corpora spanning LLM-generated systems code and mature hardened libraries (Sun et al., 20 May 2026).

Corpus	Reported results	Additional notes
VibeOS (15 k LOC C-kernel, ARM64)	675 functions in 37 modules → 34 confirmed realistic bugs	16 dynamic-crash confirmed (tier D), 14 system-entry confirmed (tier S), 4 formal-model confirmed (tier B)
OSS-Fuzz-hardened C libraries	Confirmed 2 new undefined‐behaviour defects in jq’s UTF-8 helpers	Positive bounded verification on DER decoders, XPath/XPointer, and streaming parsers
Realtek r8125 out-of-tree Linux driver	Found one CAP_NET_ADMIN–gated MMIO bounds‐check bypass	Triggered by per-function unsigned-overflow and pointer-overflow flags
Claudes-C-Compiler (50 k LOC Rust, AI-generated C compiler + ELF linker)	25 confirmed real bugs under threat‐model = security	24 panic-class defects and 1 functional-correctness violation

For VibeOS, the bug classes included null or memory pointer casts, integer overflows, OOB buffer access, loop-underflows, and stack buffer overflows. For the OSS-Fuzz-hardened C libraries—jq, OpenSSL, libcurl, libxml2, and protobuf-upb—the evaluation reported positive bounded verification (“clean” results) on heavily-used DER decoders, XPath/XPointer, and streaming parsers, while pattern-library filters absorbed recurring false positives such as xmlMalloc != NULL invariants (Sun et al., 20 May 2026).

For Claudes-C-Compiler, the 25 confirmed real bugs consisted of 24 panic-class defects on public-API byte-helpers, including slice OOB, integer overflow in align_up_64, and dominator-walk underflow, plus 1 functional-correctness violation in Rust align_up, described as silent misalignment on overflow caught only by behavioural spec. The same evaluation also reported bounded functional equivalence for ELF GNU hash and SysV hash using fold-style templates, align_up_64 invariants, and ELF header write layouts (phdr/shdr) (Sun et al., 20 May 2026).

The reported performance configuration was per-function BMC bound $G$6 by default and per-function timeout 120 s. Compositional parallelism was said to decouple wall-clock time from codebase size; deduplication and reusable SpecStore amortized cost across runs; and a single run over VibeOS, comprising 675 functions, completed within a few hours on a multi-core machine (Sun et al., 20 May 2026).

Taken together, these results support the paper’s five concluding claims: BMC-Agent can automatically infer function specs in C and Rust; scale BMC to large, LLM-generated systems code via compositional harnessing and parallelism; distinguish real bugs from model artifacts through a robust validation pipeline; extend from panic-freeness to bounded behavioural equivalence; and produce both confirmed defects and positive verification evidence on mature code (Sun et al., 20 May 2026). In that sense, BMC-Agent is best understood as a hybrid verification framework in which semantic proposal is delegated to an LLM agent, but every soundness-relevant decision remains under bounded model checking.