Code-Augur: Agentic Vulnerability Detection

• Code-Augur is a system for agentic vulnerability detection that synthesizes explicit in-source security invariants and validates them through runtime falsification.
• It interleaves LLM-driven hypothesis generation with greybox fuzzing to iteratively instrument, check, and refine security invariants for robust vulnerability auditing.
• Benchmarks on real-world projects demonstrate its superior capacity to uncover both known and previously unreported vulnerabilities compared to prior audit techniques.

Code-Augur is a system for agentic vulnerability detection in software, oriented around explicit specification inference and runtime falsification. It operationalizes LLM-driven auditing by requiring that every positive security judgment is explained through an in-source, executable security invariant. Code-Augur interleaves LLM hypothesis generation with greybox fuzzing, yielding a transparent, falsifiable, and empirically validated vulnerability discovery workflow. Benchmarks on real-world projects demonstrate Code-Augur’s capacity to uncover both known and previously unreported vulnerabilities, surpassing prior LLM- and fuzzer-guided audit techniques (Luo et al., 17 Jun 2026).

1. Security-Specification-First Paradigm

A foundational principle of Code-Augur is the externalization of LLM agent reasoning into explicit local invariants. For any given program $P$ and strategic program point $p$, the system defines the input space

$I = \text{InputSpace}$

as the set of attacker-controllable inputs. A local security specification or invariant is then a predicate

$\phi: I \to \{\mathit{Safe},\,\mathit{Unsafe}\}$

where $\phi(i) = \mathit{Safe}$ asserts that all paths from input $i$ reaching point $p$ respect the desired security property. Adopting the notation $\llbracket P \rrbracket(i)$ for the set of program states reachable under input $i$, and $E_p \subseteq \mathit{State}$ for error states at $p$0, soundness of $p$1 holds if

$p$2

In practice, Code-Augur synthesizes predicate invariants of the form

$p$3

where $p$4 are program state expressions and $p$5.

2. Workflow and Algorithms

Code-Augur’s “Invariant Analysis” proceeds through an iterative reasoning–instrumentation–falsification loop, implemented by interleaving LLM and fuzzer subagents. Algorithm 1 provides the overview:

$p$6

Each loop iteration involves LLM-driven hypothesis generation, security judgment evaluation, in-source assertion instrumentation, build-time assertion checking, and feedback-guided fuzzing. When a fuzzer-generated input violates an asserted invariant, the system classifies the event as either a true vulnerability or a benign counterexample warranting refinement.

Algorithm 2 governs runtime falsification:

$p$7

Feedback instrumentation ensures fuzzers can observe assertion sites, assertion violations, and progress metrics (e.g., for $p$8, expose $p$9 as a distance).

3. Architecture and LLM Integration

Code-Augur is orchestrated via a TypeScript harness managing subagents across major LLMs—Claude Sonnet 4.6 and DeepSeek V4 Pro are explicitly supported. The orchestration sequence is:

• Threat-Model Construction: The LLM parses project documentation for attacker boundaries and security-relevant state.
• Invariant Analysis: Subagents manage hypothesis/suggestion cycles, assertion placement, and LLM interaction.
• Fuzzing & Falsification: Standard fuzzers (libFuzzer, Jazzer) are invoked, augmented by feedback from program instrumentation, but without fuzzer core modification.
• Violation Triage and Bug Validation: Detection events are scrutinized by LLM subagents to distinguish genuine vulnerabilities from benign assertion violations, with proof-of-vulnerability (PoV) synthesis where possible.

A salient property is model modularity: the same specification-falsification loop applies irrespective of underlying LLM choice, though audit accuracy is empirically sensitive to model capability.

4. Experimental Results

Code-Augur was evaluated on two benchmarks:

• AIxCC (nine mature C/C++/Java codebases, 39 seeded bugs)
• OSV (nine recent open-source projects, 24 in-the-wild bugs; languages C, C++, Java, Rust)

Baselines were:

• Atlantis (AIxCC winner, fuzzing-centric with LLM steering)
• Claude Code (general-purpose LLM audit)

Metrics were:

• Existing: Known bugs rediscovered
• New: Previously unreported vulnerabilities on the analyzed revision

Results are summarized as follows (E: existing, N: new):

Benchmark	Code-Augur E+N (Sonnet/DeepSeek)	Atlantis E+N (Sonnet/DeepSeek)	Claude Code E+N (Sonnet/DeepSeek)
AIxCC	(33+26)/(29+22) = 59/51	(25+11)/(21+14) = 36/35	(32+12)/(29+7) = 44/36
OSV	(8+50)/(9+40) = 58/49	(9+20)/(8+2) = 29/10	(10+26)/(8+11) = 36/19

Code-Augur outperformed baselines by 34–63% (AIxCC) and 61–370% (OSV), even with open-weight DeepSeek.

5. Case Studies and Specification Falsification

Notable discoveries include:

• Little CMS (AIxCC): Code-Augur synthesized the invariant $I = \text{InputSpace}$0. A fuzzer input with mismatch ($I = \text{InputSpace}$1) triggered an out-of-bounds read in UnpackPixel.
• gpsd (OSV): The invariant $I = \text{InputSpace}$2 was inferred. A crafted input (count=256) violated $I = \text{InputSpace}$3, leading to an out-of-bounds access across multiple code paths and a multi-stage codebase remediation.

This suggests that externalized invariants are not only effective in surfacing subtle, long-dormant implementation flaws but also serve as actionable artifacts for regression prevention and subsequent mitigation campaigns.

6. Limitations and Future Directions

Limitations include:

• LLM error propagation, mitigated by repeated falsification and explicit proof-of-vulnerability workflow.
• Potential LLM training data leakage; however, most “new” bugs were not present in public datasets, supporting novelty of results.
• Context-window constraints in LLM inference, addressed by subagent decomposition and modularization.

Planned research directions include:

• Adversarial back-door detection via specification falsification.
• Augmenting runtime falsification with grammar-based and concolic fuzz engines.
• Enhancing invariant refinement with retrieval-augmented prompting for better drift control.
• Extending to multi-language, cross-repository global specification inference (Luo et al., 17 Jun 2026).

7. Significance in the Landscape of Automated Software Auditing

By converting natural-language security justifications into falsifiable, executable invariants, Code-Augur introduces a paradigm shift from opaque LLM judgment to transparent, verifiable security analysis. The integration of semantic LLM reasoning with feedback-driven fuzzing uncovers classes of vulnerabilities not found by prior single-modality approaches. These invariants act as durable, testable documentation for future code evolution, facilitating continuous validation and in-depth forensic analysis of security-critical codebase changes (Luo et al., 17 Jun 2026).