--- title: 'Code-Augur: Agentic Vulnerability Detection' url: https://www.emergentmind.com/topics/code-augur type: topic --- # Code-Augur: Agentic Vulnerability Detection Code-Augur is a system for agentic vulnerability detection in software, oriented around explicit specification inference and runtime falsification. It operationalizes large language model (LLM)-driven auditing by requiring that every positive security judgment is explained through an in-source, executable security invariant. Code-Augur interleaves LLM hypothesis generation with greybox fuzzing, yielding a transparent, falsifiable, and empirically validated vulnerability discovery workflow. Benchmarks on real-world projects demonstrate Code-Augur’s capacity to uncover both known and previously unreported vulnerabilities, surpassing prior LLM- and fuzzer-guided audit techniques [2606.18619]. ## 1. Security-Specification-First Paradigm A foundational principle of Code-Augur is the externalization of LLM agent reasoning into explicit local invariants. For any given program $P$ and strategic program point $p$, the system defines the input space $$ I = \text{InputSpace} $$ as the set of attacker-controllable inputs. A local security specification or invariant is then a predicate $$ \phi: I \to \{\mathit{Safe},\,\mathit{Unsafe}\} $$ where $\phi(i) = \mathit{Safe}$ asserts that all paths from input $i$ reaching point $p$ respect the desired security property. Adopting the notation $\llbracket P \rrbracket(i)$ for the set of program states reachable under input $i$, and $E_p \subseteq \mathit{State}$ for error states at $p$, soundness of $\phi$ holds if $$ \forall i \in I : \phi(i) = \mathit{Safe} \implies \left( \llbracket P \rrbracket(i) \cap E_p = \emptyset \right) $$ In practice, Code-Augur synthesizes predicate invariants of the form $$ \phi(i) = (v_1(i) \mathbin{\bowtie} v_2(i)) $$ where $v_1, v_2$ are program state expressions and $\bowtie \in \{=,\le,<,\ldots\}$. ## 2. Workflow and Algorithms Code-Augur’s “Invariant Analysis” proceeds through an iterative reasoning–instrumentation–falsification loop, implemented by interleaving LLM and fuzzer subagents. Algorithm 1 provides the overview: \[ \boxed{ \begin{aligned} &\textbf{Input:}\;P\;(\text{source}),\;T\;(\text{threat model})\\ &\textbf{Output:}\;B\;(\text{bug candidates}),\;\Phi\;(\text{surviving invariants})\\ &\Phi \leftarrow \emptyset,\; B \leftarrow \emptyset,\; C \leftarrow \emptyset\\ &\while\; \neg\text{budget\_exhausted}\;\textbf{do}\\ &\quad (h,s) \leftarrow \text{GenerateHypothesis}(P,T,\Phi)\\ &\quad (b,\phi) \leftarrow \text{EvaluateHypothesis}(P,T,h,s)\\ &\quad \If\; b \neq \bot \text{ then } B \mathrel{\cup}= \{b\}\\ &\quad \ElseIf\; \phi \neq \bot \text{ then}\\ &\quad\quad P' \leftarrow \text{Instrument}(P,\Phi \cup \{\phi\})\\ &\quad\quad \text{Check}(P',\phi)\\ &\quad\quad (b, fb, C) \leftarrow \text{Falsify}(P, P', \phi, C)\\ &\quad\quad \If\; b \neq \bot \text{ then } B \mathrel{\cup}= \{b\}\\ &\quad\quad \ElseIf\; fb \neq \bot \text{ then } \phi \leftarrow \text{RefineInvariant}(T, \phi, fb)\\ &\quad\quad \Else\; \Phi \mathrel{\cup}= \{\phi\}\\ \end{aligned} } \] Each loop iteration involves LLM-driven hypothesis generation, security judgment evaluation, in-source assertion instrumentation, build-time assertion checking, and feedback-guided fuzzing. When a fuzzer-generated input violates an asserted invariant, the system classifies the event as either a true vulnerability or a benign counterexample warranting refinement. Algorithm 2 governs runtime falsification: \[ \boxed{ \begin{aligned} &\textbf{Procedure } \Falsify(P, P', \phi, C):\\ &\quad (\mathcal{H}, C) \leftarrow \text{FuzzingPreparation}(P', C)\\ &\quad (i_{\neg \phi}, C) \leftarrow \text{SpecGuidedFuzz}(P', \mathcal{H}, \phi, C)\\ &\quad \If\; i_{\neg \phi} = \bot \text{ then } \Return (\bot, \bot, C)\\ &\quad (b, fb) \leftarrow \text{Triage}(i_{\neg \phi}, \phi, P, P')\\ &\quad \Return (b, fb, C) \end{aligned} } \] Feedback instrumentation ensures fuzzers can observe assertion sites, assertion violations, and progress metrics (e.g., for $x \le M$, expose $|x-M|$ as a distance). ## 3. Architecture and LLM Integration Code-Augur is orchestrated via a TypeScript harness managing subagents across major LLMs—Claude Sonnet 4.6 and DeepSeek V4 Pro are explicitly supported. The orchestration sequence is: - **Threat-Model Construction:** The LLM parses project documentation for attacker boundaries and security-relevant state. - **Invariant Analysis:** Subagents manage hypothesis/suggestion cycles, assertion placement, and LLM interaction. - **Fuzzing & Falsification:** Standard fuzzers (libFuzzer, Jazzer) are invoked, augmented by feedback from program instrumentation, but without fuzzer core modification. - **Violation Triage and Bug Validation:** Detection events are scrutinized by LLM subagents to distinguish genuine vulnerabilities from benign assertion violations, with proof-of-vulnerability (PoV) synthesis where possible. A salient property is model modularity: the same specification-falsification loop applies irrespective of underlying LLM choice, though audit accuracy is empirically sensitive to model capability. ## 4. Experimental Results Code-Augur was evaluated on two benchmarks: - **AIxCC** (nine mature C/C++/Java codebases, 39 seeded bugs) - **OSV** (nine recent open-source projects, 24 in-the-wild bugs; languages C, C++, Java, Rust) Baselines were: - Atlantis (AIxCC winner, fuzzing-centric with LLM steering) - Claude Code (general-purpose LLM audit) Metrics were: - **Existing:** Known bugs rediscovered - **New:** Previously unreported vulnerabilities on the analyzed revision Results are summarized as follows (E: existing, N: new): | Benchmark | Code-Augur E+N (Sonnet/DeepSeek) | Atlantis E+N (Sonnet/DeepSeek) | Claude Code E+N (Sonnet/DeepSeek) | |-----------|:-------------------------------:|:------------------------------:|:---------------------------------:| | AIxCC | (33+26)/(29+22) = 59/51 | (25+11)/(21+14) = 36/35 | (32+12)/(29+7) = 44/36 | | OSV | (8+50)/(9+40) = 58/49 | (9+20)/(8+2) = 29/10 | (10+26)/(8+11) = 36/19 | Code-Augur outperformed baselines by 34–63% (AIxCC) and 61–370% (OSV), even with open-weight DeepSeek. ## 5. Case Studies and Specification Falsification Notable discoveries include: - **Little CMS (AIxCC):** Code-Augur synthesized the invariant $\phi(i) = \texttt{fmt.channels} = \texttt{cs.channels}$. A fuzzer input with mismatch ($13 \neq 1$) triggered an out-of-bounds read in `UnpackPixel`. - **gpsd (OSV):** The invariant $\psi(i) = (\texttt{satellites\_visible} \le \mathit{MAXCHANNELS})$ was inferred. A crafted input (`count=256`) violated $\psi$, leading to an out-of-bounds access across multiple code paths and a multi-stage codebase remediation. This suggests that externalized invariants are not only effective in surfacing subtle, long-dormant implementation flaws but also serve as actionable artifacts for regression prevention and subsequent mitigation campaigns. ## 6. Limitations and Future Directions Limitations include: - LLM error propagation, mitigated by repeated falsification and explicit proof-of-vulnerability workflow. - Potential LLM training data leakage; however, most “new” bugs were not present in public datasets, supporting novelty of results. - Context-window constraints in LLM inference, addressed by subagent decomposition and modularization. Planned research directions include: - Adversarial back-door detection via specification falsification. - Augmenting runtime falsification with grammar-based and concolic fuzz engines. - Enhancing invariant refinement with retrieval-augmented prompting for better drift control. - Extending to multi-language, cross-repository global specification inference [2606.18619]. ## 7. Significance in the Landscape of Automated Software Auditing By converting natural-language security justifications into falsifiable, executable invariants, Code-Augur introduces a paradigm shift from opaque LLM judgment to transparent, verifiable security analysis. The integration of semantic LLM reasoning with feedback-driven fuzzing uncovers classes of vulnerabilities not found by prior single-modality approaches. These invariants act as durable, testable documentation for future code evolution, facilitating continuous validation and in-depth forensic analysis of security-critical codebase changes [2606.18619]. Source: https://www.emergentmind.com/topics/code-augur