Code-Augur: Agentic Vulnerability Detection via Specification Inference

Abstract: The advent of agentic vulnerability detection is already becoming a watershed moment for software security. Audits conducted entirely by autonomous LLM agents are uncovering critical vulnerabilities in fundamental software underpinning digital society. Many of these vulnerabilities remained masked for years, surfacing only now with AI agents. Yet the reasoning behind these discoveries remains alarmingly opaque and unvalidated. What assumptions did the agent make about a function's inputs when it deemed that function to be secure? Failures in reasoning and incorrect assumptions can lead to missed vulnerabilities and reduce trust in agentic analysis. We propose a security-specification-first paradigm that (1) exposes the agent's tacit assumptions explicitly as security specifications and (2) continuously refines those specifications via runtime falsification. We realize our approach in Code-Augur, a novel harness for agentic vulnerability detection. Given a codebase, Code-Augur analyzes each component of the system for vulnerable code. When it deems a component to be secure, it commits the local invariants behind that judgment as in-source assertions. In parallel, Code-Augur leverages a guided fuzzer to attempt to falsify those assumptions. When the fuzzer triggers an assertion, this either reveals a genuine vulnerability or a flawed specification to refine. In both cases, this process grounds the agent's understanding, aligning its view of code intent with how the code actually behaves. On real-world subjects, Code-Augur effectively leverages security specifications to detect more vulnerabilities than other state-of-the-art agents. Additionally, Code-Augur found 22 new vulnerabilities in key open-source projects. Compared to curated specialized models like Claude Mythos, Code-Augur offers effective agentic vulnerability detection built on widely available LLMs like Sonnet and DeepSeek.

• The paper presents a specification-first paradigm that converts implicit agentic reasoning into explicit security invariants for targeted vulnerability detection.
• It employs invariant inference combined with specification-guided fuzzing to uncover 34–370% more vulnerabilities than traditional methods.
• The approach generates durable, verifiable artifacts that support continuous security audits and efficient vulnerability validation.

Code-Augur: Agentic Vulnerability Detection via Specification Inference

Motivation and Problem Statement

Recent advances in agentic vulnerability detection, notably LLM-driven analysis, have begun surfacing latent vulnerabilities in foundational software, but the semantic opacity and lack of validation in agentic reasoning undermine its reliability. Existing paradigms rely on implicit, intuitive judgments by autonomous agents, flagging bug candidates for dynamic validation but leaving foundational security specifications unvalidated and unarticulated. As highlighted in the motivating example from Little CMS, security flaws can span complex paths in code, eluding both semantic analysis and fuzzing unless the tacit, local invariants are made explicit.

Figure 1: Code snippets illustrating how implicit agent reasoning misses a critical inconsistency, while Code-Augur’s explicit invariant commits fmt.channels = cs.channels for subsequent dynamic falsification.

Agentic systems, such as Claude Mythos, perform dynamic validation only on flagged inputs, neglecting non-suspect regions where false negatives may reside. Fuzzing serves as brute-force exploration but lacks semantic context, impeding its ability to reach deep states or recognize semantically relevant inconsistencies (Figure 2).

Figure 2: Conventional agentic vulnerability detection; validation is only performed for suspect cases, leaving the implicit specification untested.

Specification-First Paradigm and System Architecture

Code-Augur inaugurates a specification-first paradigm for autonomous vulnerability detection. The agent, guided by a threat model, explicitly commits semantic assumptions as local invariants (assertions) within code, making the reasoning tractable and falsifiable. These invariants are targeted by a specification-guided fuzzer that seeks inputs to violate them, closing the loop between semantic reasoning and dynamic edge-case exploration. Violations lead either to the surfacing of genuine vulnerabilities or to iterative refinement of erroneous agentic specifications.

Figure 3: Workflow overview: explicit security invariant inference, instrumentation, specification-guided fuzzing, violation triage, and refinement.

The system consists of four primary stages:

1. Threat Model Construction: Extraction of attack surface, trust boundaries, security-relevant variables, and context from code, documentation, and build artifacts.
2. Invariant Analysis: Agentic static reasoning hypothesizes local invariants that encode program-specific security properties and commit them as in-source assertions.
3. Invariant Falsification: Specification-guided fuzzing attempts to falsify each invariant; violations are triaged to distinguish genuine vulnerabilities from benign exceptions, prompting dynamic refinement.
4. Bug Validation: Proof-of-vulnerability construction and boundary checking ensure only reachable, attacker-relevant vulnerabilities are reported.

Implementation and Methodology

Code-Augur leverages the pi agent toolkit for orchestration and integrates off-the-shelf grey-box fuzzers (libFuzzer, Jazzer, native Go/Rust fuzzers) for runtime exploration. Invariants are instrumented using native assertion mechanisms and specialized feedback channels, facilitating distance-minimized input generation.

The iterative reason-falsify-refine loop is governed by LLM-driven static reasoning for invariant inference and refinement, while fuzzing delivers broad input coverage for dynamic specification validation. This duality substantially mitigates the risk of unsound agentic analysis.

Evaluation and Empirical Results

Effectiveness: Code-Augur was evaluated across benchmarks from AIxCC and OSV, comparing against distinguished agentic baselines (Claude Code, Atlantis). On the AIxCC benchmark, Code-Augur uncovered 34–63% more bugs on average than baselines, with superior recall both on pre-existing and novel vulnerabilities. On the OSV benchmark, Code-Augur identified 61–370% more bugs, demonstrating broad subject coverage and robust inference capability.

Component Contribution: Discovery lanes revealed that nearly half of Code-Augur’s findings originated directly from specification falsification, beyond what pure static review or fuzzing could achieve. The specification-guided approach also leverages open-weight models more efficiently compared to specialized frontier LLMs, providing cost-effective, scalable results.

Real-World Impact: In real-world audits, Code-Augur discovered 22 previously unknown vulnerabilities across critical open-source projects, with 16 already remediated or confirmed by maintainers. The findings spanned diverse classes—memory errors, access-control bypasses, SSRF, and logic flaws—many requiring counterintuitive inputs unreachably deep through classic paradigms.

Utility and Lifecycle Implications of Inferred Specifications

Code-Augur’s explicit invariants provide durable, reusable artifacts for security audits. Case studies (e.g., gpsd) demonstrate that a single invariant pins down root causes across complex bug families, guiding fix prioritization, catching incomplete remediations, and revealing undersized semantic bounds. Once committed, invariants facilitate regression testing and specification-driven reasoning by human and agentic analysts alike.

Theoretical and Practical Implications

Code-Augur formalizes the extraction and validation of agentic semantic judgments, aligning LLM-generated reasoning with real program behavior. Its approach externalizes agentic knowledge, converting intuition into actionable, checkable specifications and enabling systematic, specification-driven fuzzing. The methodology extends the lifecycle of vulnerability detection artifacts beyond point bug reports, laying groundwork for AI-centered specification management and regression prevention.

Future directions include integration with adversarial analysis for backdoor detection, extension to novel fuzzing paradigms, and operationalization of invariant repositories for continuous assurance in AI-driven development pipelines.

Conclusion

Code-Augur advances agentic vulnerability detection through explicit specification inference and dynamic falsification, harmonizing agentic reasoning and fuzzing for superior recall and actionable artifact generation. Its empirical results validate broad effectiveness, durable utility, and cross-model adaptability, offering a principled foundation for future autonomous software security practice. The specification-first paradigm positions Code-Augur not merely as a vulnerability detector but as an enabler of robust, semantically-grounded security assurance.