---
title: 'CVE-Factory: Automated Vulnerability Generation'
url: https://www.emergentmind.com/topics/cve-factory
type: topic
---

# CVE-Factory: Automated Vulnerability Generation

CVE-Factory refers to a class of systems and conceptual frameworks that automate, formalize, or model the generation, reproduction, and classification of Common Vulnerabilities and Exposures (CVEs) for purposes ranging from code agent training and evaluation to theoretical vulnerability analysis. The term encompasses: (1) practical, automated pipelines for synthesizing executable vulnerability tasks from sparse CVE metadata; (2) highly structured multi-agent workflows for benchmark and dataset construction; and (3) formal constructions, such as the Vulnerability Factory, that explore the computability and abundance of software vulnerabilities. CVE-Factory implementations have enabled large-scale, continual benchmarking of code agents and have provided the foundation for rigorous analytical tools in vulnerability theory [2602.03012, 2604.07539, 2510.15567].

## 1. CVE-Factory Agentic Reproduction Pipeline

Modern CVE-Factory systems instantiate a multi-stage, autonomous pipeline where specialized agents coordinate to convert raw CVE metadata into validated, executable security tasks. The canonical six-stage workflow is orchestrated via a central controller and follows the structure below [2602.03012]:

1. **Analyzer:** Parses CVE JSON, fetches proof-of-concept (PoC) code and associated commits, and distills all data into unified markdown artefacts.
2. **Generator:** Synthesizes logical task components (task description, tests, patch, and environment hints) from distilled documents.
3. **Builder:** Constructs the development and exploit environment (e.g., Dockerfile, docker-compose), strictly without information leakage from test or solution components.
4. **Validator:** Executes verification routines, expecting the vulnerability to be present prior to patch application (negative vulnerability test, positive functionality test), and reroutes control for remediation if environment defects are detected.
5. **Solver:** Applies the patch and expects successful mitigation of the vulnerability, again with feedback and retry logic for correction.
6. **Checker:** Enforces end-to-end validation, removes mock code, and finalizes the packaged task.

The Orchestrator manages feedback signals (**continue**, **error**, **pause**) to guarantee both correctness and process accountability. Up to three retries are allowed per stage to ensure robustness and limit resource consumption. All agents may abort or reroute at any error point, ensuring only high-fidelity environments and valid solutions propagate.

Empirical results against expert benchmarks report 95.35% solution correctness and 96.13% environment fidelity on 215 CVE tasks [2602.03012].

## 2. Dataset Construction, Benchmarking, and Scaling

CVE-Factory underpins the construction of LiveCVEBench, a continuously updated, executable benchmark that captures modern and emerging vulnerability classes. The process involves:

- **Curated Candidate Pool:** 7,152 recent CVEs are scored heuristically (presence of PoC/patch, language stack, deployment constraints).
- **Diversity Sampling:** Ensures representation across the most critical CWEs and software repositories by capping per repo/CWE.
- **Semantic LLM Filtering:** Uses large language models as judges to filter for containerizability and eliminate redundancy.
- **Automated & Manual Validation:** CVE-Factory agents process candidates, with manual spot-checking for quality control.

The current LiveCVEBench release comprises 190 tasks in 14 programming languages across 153 repositories and 74 CWE types, with 10% focused on AI-tooling vulnerabilities. All resources are publicly available, and tasks are added monthly [2602.03012].

The pipeline further supports massive parallel synthesis, sustaining 20 agents to generate over 1,000+ executable CVE tasks within 48 hours. All synthesized environments are dynamically verified and ensure patch validity at the source code level rather than version upgrade.

## 3. Model Fine-Tuning and Quantitative Evaluation

CVE-Factory datasets enable LLM fine-tuning at scale for code security tasks. Using curated trajectories comprising agent decisions and outcomes, models such as Qwen3-32B are trained using AdamW optimization, ZeRO-3 parallelism, and FlashAttention-2, targeting a sequence length of 65,536 tokens. Training is conducted for five epochs over 4,000 interaction trajectories across 1,070 tasks.

Performance is assessed on LiveCVEBench, PatchEval, and Terminal-Bench. Key results include:

- Qwen3-32B transitions from 5.3% (zero-shot) to 35.8% accuracy on LiveCVEBench when fine-tuned on CVE-Factory data.
- Comparable or superior gains are observed on PatchEval and Terminal-Bench, exceeding various baseline corpora such as SETA.
- OpenAI Claude Opus 4.5 achieves the highest mean scores (41.3% on LiveCVEBench) [2602.03012].

All model checkpoints ("Abacus-cve"), datasets ("cve_train"), and leaderboards are open-sourced.

## 4. Formal Foundations: The Vulnerability Factory

Beyond automation, the CVE-Factory concept includes foundational results in theoretical vulnerability science. The "Vulnerability Factory" is a 622-line C program designed to generate a provably countable infinity of distinct, MITRE-assigned CVEs [2604.07539]. Its structure:

- **Base Functions B:** Eleven canonical vulnerabilities each map to distinct CWE classes (e.g., CWE-121, CWE-134, CWE-190, CWE-416).
- **Generator G:** For each counter $n \in \mathbb{N}$, emits a new module $M_n$ with five unique parameterized vulnerabilities $V(M_n) = \{v_{n,1}, \ldots, v_{n,5}\}$.
- **Set-Theoretic Construction:** $V = B \cup \bigcup_{n=0}^{\infty} V(M_n)$.

Key theorems and formalizations:

- **Countable Infinity:** By mapping $f(n,i) = v_{n,i}$, the total vulnerability set $V$ is shown to be countably infinite.
- **CVE Assignability:** Each vulnerability is distinct, independently fixable, and matches MITRE's formal CVE-counting predicate.
- **Model Checking:** The transition system for the factory refutes any claim that vulnerability count is finite via CTL property violation.
- **Turing-Machine Model:** The Vulnerability Factory can be realized by a Turing machine, affirming its conceptual universality.

*Vulnerability abundance* is defined for a corpus $\Sigma$ at time $\tau$ as
$$
A_\Sigma(t, \tau) = \frac{|\{ v \in V(\Sigma, \tau) \mid \mathrm{type}(v) = t \}|}{|V(\Sigma, \tau)|}
$$
where $t$ is a CWE class. This metric mirrors the distribution of vulnerabilities analogously to chemical elemental abundance and shifts dynamically with language and software ecosystem evolution.

## 5. Theoretical Distinctions: Vulnerability Abundance and Exploit Finitude

A central CVE-Factory insight is the separation of the infinite vulnerability supply ($|V| = \aleph_0$) from the empirical finitude of exploits. Empirical studies report that only ~2.6% to ~6% of published CVEs are ever exploited in the wild, with even lower fractions in recent years [2604.07539]. The exploitation exposure for a vulnerability $v$ in software $s$ is formalized as:
$$
E(v,s) = A(t_v) \times D(s) \times P_\mathrm{exploit}(v)
$$
where $A$ is the abundance of the vulnerability class, $D(s)$ the deployment share, and $P_\mathrm{exploit}(v)$ the probability of exploitation.

This distinction underpins approaches to cyber-risk modeling, regulatory prioritization, and the economics of software security. Notably, a "small-exploit principle" emerges, positing that a small set of highly targeted exploits suffices to saturate systemic risk due to software market concentration.

## 6. Benchmarks, Formal Methods, and Open Resources

CVE-Factory outputs are foundational for both empirical and formal-mathematical software evaluation:

- **Dynamic Agent Benchmarks:** LiveCVEBench and its predecessors provide reproducible tasks to continually assess and improve code security agents.
- **Formal-Method Evaluation:** The infinite vulnerability families produced by factories like vuln_factory.c serve as stress-tests and lower bounds for static analysis and model-checking tools, proving that no finite-state analyzer can claim completeness across the vulnerability space [2604.07539].
- **Open-Source Ecosystem:** All core pipelines, agent scripts, benchmarks, datasets, and models are released with permissive licensing for reproducibility and extension (see repositories at https://github.com/livecvebench/CVE-Factory and associated HuggingFace datasets) [2602.03012].

## 7. Practical Impact and Directions

CVE-Factory pipelines offer both immediate practical applications and theoretical advances:

- **Scalable Reproduction and Evaluation:** Transition from costly manual reproduction to large-scale, agentic generation of vulnerability tasks with verified ground truth enables accelerated research in automated code security.
- **Quantitative Prioritization:** Vulnerability abundance metrics inform migration efforts and cyber-risk models, e.g., targeting memory-unsafe code (as quantified by $A_\Sigma(\text{CWE-121}, \tau)$ in C/C++) for rewrite in safer languages.
- **Theoretical Artifacts:** Formal constructions illustrate the inherent limitations of vulnerability detection, providing reusable test objects for future research.
- **Public Infrastructure:** Continuous benchmarking, model leaderboard tracking, and high-fidelity datasets benefit both academic and industrial research communities.

The CVE-Factory paradigm thus unifies automated vulnerability task synthesis, empirical code security agent assessment, and foundational research in vulnerability theory, with published implementations and proofs forming a new infrastructure layer for the security research community [2602.03012, 2604.07539, 2510.15567].

Source: https://www.emergentmind.com/topics/cve-factory