CVEFixes Dataset: Real-World Vulnerability Fixes

Updated 6 July 2026

CVEFixes is a CVE-grounded dataset that aggregates real-world vulnerability fixes from open-source projects, capturing both vulnerable and patched code with linked security metadata.
It employs an automated workflow from NVD feeds and repository mining to extract multi-level data, enabling analysis at repository, commit, file, and method levels.
The dataset supports diverse applications including vulnerability detection, classification, severity prediction, and patch pattern mining across multiple programming languages.

CVEfixes, also written as CVEFixes or CVE_Fixes in later work, is a CVE-grounded dataset of real-world vulnerability fixes in open-source software. In its automated form, it is a relational collection built from National Vulnerability Database records that reference public fixing commits, and it stores vulnerable and patched code together with programming-language annotations, CWE and CVSS metadata, and code metrics at repository, CVE, commit, file, and method levels (Bhandari et al., 2021). Subsequent studies have reused the same underlying resource to derive task-specific views, notably function-level C/C++ slices for binary vulnerability detection and larger cross-language snippet collections for language-model evaluation (Gonçalves et al., 2024, Atiiq et al., 2024).

1. Historical development and scope

The best-known form of CVEfixes originates in the 2021 automated collection pipeline that harvested all published CVEs up to 9 June 2021 when they contained public repository references with git commit hashes. That initial release covered 5,365 CVE records, 1,754 open-source repositories, and 5,495 unique fixing commits, with 27 detected programming languages and 180 CWE types (Bhandari et al., 2021). The design goal was not only to enumerate fixes, but to preserve both pre-fix and post-fix source code and to connect them to security metadata such as CVSS scores and CWE classifications.

The name also sits within a broader lineage of CVE-linked fix datasets. A manually curated SAP Security Research release from 2019 mapped 624 publicly disclosed vulnerabilities affecting 205 distinct open-source Java projects onto 1,282 fixing commits, using a four-column schema centered on vulnerability identifiers, repository URLs, commit hashes, and class labels (Ponta et al., 2019). Later CVEfixes-based studies then shifted toward broader language coverage and task-oriented representations, including a cross-language snapshot with 277,948 labeled code snippets and a refined C/C++ function subset for LLM evaluation (Atiiq et al., 2024, Gonçalves et al., 2024).

2. Collection workflow and database organization

The automated collection process begins with yearly NVD JSON feeds. These feeds are flattened and parsed to extract CVE identifiers, publication and modification dates, CWE types, reference URLs, and CVSS v2 and v3 scores and subscores. CVEs without git-fix references in their reference_json entries are filtered out. For the remaining records, the referenced repositories are cloned from GitHub, GitLab, or Bitbucket, the fix commit is checked out, and its parent commit is retrieved so that the vulnerable and patched versions can be compared directly (Bhandari et al., 2021).

Code-level mining is then performed with PyDriller. At the file level, the pipeline stores the blobs before and after the fix, the raw git diff, a parsed JSON representation of added and deleted lines, the file language as detected via Guesslang, and size and complexity metrics. At the method level, it stores method name, signature, parameters, start and end lines, token count, lines of code, cyclomatic complexity, code, and a Boolean indicator specifying whether the method snapshot is taken before or after the change. Commit-level metadata include author, timestamp, commit message, merge status, added and deleted lines, and Delta Maintainability Model metrics (Bhandari et al., 2021).

The resulting schema is normalized around the relations cve, cwe, cwe_classification, repository, fixes, commits, file_change, and method_change. This organization makes the dataset queryable across several abstraction levels. In the maintainability layer, the overall Delta Maintainability Model score is defined as

$\text{DMM}_{\text{overall}}=\frac{\text{DMM}_{\text{size}}+\text{DMM}_{\text{complexity}}+\text{DMM}_{\text{interface}}}{3}.$

This multi-level structure is central to how CVEfixes differs from corpora that expose only commit messages or diffs, because it supports repository-level, CVE-level, commit-level, file-level, and method-level analyses within a single database (Bhandari et al., 2021).

3. Labels, granularity, and cross-language representations

Although the original database is relational and multi-level, later vulnerability-detection studies commonly use snippet-level views extracted from it. In a cross-language benchmark derived from CVEfixes and updated up to 23 July 2024, the collection contains 277,948 code snippets labeled as vulnerable or non-vulnerable, with 126,599 vulnerable and 151,349 non-vulnerable instances (Atiiq et al., 2024). Each instance is linked to a CVE-ID from the NVD and to the fixing commit in an open-source repository.

The same study reports the following language-specific counts after trimming JavaScript to 100,000 samples stratified by commit timestamp (Atiiq et al., 2024):

Language	Vulnerable / Non-vulnerable	Total
JavaScript	46,802 / 53,198	100,000
PHP	4,758 / 23,499	28,257
C / C++	8,299 / 11,761	20,060
Java	3,102 / 5,285	8,387
Python	2,775 / 4,793	7,568
Go	1,880 / 4,403	6,283

In this representation, the per-instance schema includes code_before_fix, code_after_fix for vulnerable instances, label ∈ {0,1}, CVE_ID, CWE_type, CVSS_score, project_name, repo_url, commit_hash, commit_date, and a language tag, with optional fix descriptions and patch hunk coordinates (Atiiq et al., 2024). The preprocessing used for this view includes byte-for-byte deduplication of identical code snippets, language-tag validation through Linguist and file extension checks, normalization of trailing whitespace and line endings, and a temporal integrity check that excludes fix commits later than the NVD publishing date of the CVE. Train and test partitions are temporal rather than purely random, so that the model sees only past vulnerabilities during training and future vulnerabilities during testing (Atiiq et al., 2024).

This use of CVEfixes as a snippet corpus suggests that the dataset functions less as a single immutable artifact than as a reusable source database from which different vulnerability-learning tasks extract different supervised views.

A particularly detailed derivative is the C/C++ function-level subset studied in SCoPE. In that work, CVEFixes version 1.0.7 is treated as a collection of function-level code slices from real-world C/C++ projects. The reported subset contains 15,649 functions, of which 6,515 are vulnerable and 9,134 are non-vulnerable, with binary labels and metadata such as method_change_id, before_change, programming language, associated CVE and CWE identifiers, cyclomatic complexity, file_change_id, and project identifiers (Gonçalves et al., 2024).

SCoPE, the Source Code Processing Engine, applies five orthogonal transformations implemented via antlr4 parsing: programmer-defined identifier renaming into generic tokens such as VAR_1 and FUNC_1, string literal replacement with STR, whitespace normalization, comment removal, and optional tokenization into space-separated tokens (Gonçalves et al., 2024). Deduplication is performed after transformation. If the normalized code strings for two samples satisfy $C_i == C_j$ , they are considered duplicates. Duplicate groups are then categorized into three cases: code differing only in comments, truly identical code before and after, and code identical modulo renamed identifiers. Exactly one record is retained from each duplicate group, and if both labels occur in the group, the clean sample with label $0$ is preferentially kept (Gonçalves et al., 2024).

The reported processing outcome is highly specific. Of the original subset, 12,035 samples were processed without parsing errors, 3,614 were excluded due to unsupported macros, 905 duplicates were detected, and the final retained set contained 11,573 functions, including 4,612 vulnerable and 6,961 clean functions (Gonçalves et al., 2024). For model training, this refined set was down-sampled to a balanced 9,224-sample subset with 4,612 vulnerable and 4,612 clean functions. Mean token length decreased from 374 to 300 after processing. The refined schema for this derivative contains method_change_id, original_code, processed_code, label, programming_language, cve_id, cwe_id, cyclomatic_complexity, and parsing_error (Gonçalves et al., 2024).

This refinement is notable because it treats deduplication and normalization as first-class methodological issues rather than incidental cleaning. It also makes explicit that some of the apparent learning signal in raw vulnerability datasets can arise from duplicated or near-duplicated code.

5. Use in vulnerability-detection research

CVEfixes is widely used for binary vulnerability detection, but the exact evaluation setup depends on the derived view. In the refined C/C++ SCoPE subset, the train, validation, and test split is $80\%$ , $10\%$ , and $10\%$ , respectively, with stratification on the binary label to preserve the 50:50 class balance in each split (Gonçalves et al., 2024). Precision, recall, and $F_1$ are defined in the standard way:

$\mathrm{Precision}=\frac{TP}{TP+FP}, \qquad \mathrm{Recall}=\frac{TP}{TP+FN}, \qquad F_1=2\times\frac{\mathrm{Precision}\times\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}.$

On that subset, three pre-trained LLMs were fine-tuned for vulnerability detection: CodeBERT from the RoBERTa family, CodeGPT from the GPT-2 family, and NatGen from the Flan-T5 family. The reported hyperparameters are 10 epochs, batch size 16, learning rate $2\times10^{-5}$ , 200 warmup steps, and LoRA for PEFT. Best test-set results after SCoPE processing were approximately $53\%$ precision, $C_i == C_j$ 0 recall, and $C_i == C_j$ 1 $C_i == C_j$ 2 for NatGen, about $C_i == C_j$ 3 $C_i == C_j$ 4 for CodeBERT, and about $C_i == C_j$ 5 $C_i == C_j$ 6 for CodeGPT. No significant improvement in $C_i == C_j$ 7 was observed before versus after processing, and the study states that current LLMs alone remain insufficient for reliable C/C++ vulnerability detection on real-world data (Gonçalves et al., 2024).

Cross-language experiments derived from CVEfixes broaden that picture. A later study fine-tuned CodeT5, CodeBERT, UnixCoder, and DeepSeek-Coder on JavaScript, Java, Python, PHP, Go, and C/C++, using AdamW with learning rate $C_i == C_j$ 8, 10 epochs, gradient accumulation, linear warmup, and NVIDIA A100 GPUs (Atiiq et al., 2024). That study reports that vulnerability-detection performance varies significantly by language and that JavaScript exhibits the best performance, with considerably better and more practical detection capabilities than C/C++. It also computes token length, Halstead volume, difficulty, effort, cyclomatic complexity, and NLOC, then correlates language-level $C_i == C_j$ 9 with these complexity measures; all reported $0$0 with $0$1, indicating no statistically significant correlation (Atiiq et al., 2024). A plausible implication is that benchmark difficulty in CVEfixes-derived evaluations is shaped at least as much by language ecology, label construction, and context granularity as by simple code-complexity measures.

6. Strengths, limitations, and benchmark status

Several strengths recur across the literature. CVEfixes offers cross-language coverage, direct linkage to real CVE records, rich metadata including CWE and CVSS information, commit dates and project names, and before/after code connected through a relational schema that supports arbitrary queries (Atiiq et al., 2024). The collection process is repeatable, and the original release was explicitly designed to be rerun to incorporate newly published or newly linked CVEs (Bhandari et al., 2021). These properties make it useful not only for vulnerability detection, but also for vulnerability classification, severity prediction, patch-pattern mining, and automated repair (Bhandari et al., 2021).

The limitations are equally well documented. Some CVE-referenced repositories are no longer available, which produces unavoidable coverage gaps; fix commits may be incomplete or may mix vulnerability remediation with unrelated edits; and extension to other version-control systems and issue trackers remains unfinished (Bhandari et al., 2021). In cross-language LM usage, the sample sizes are strongly imbalanced by language, approximately $0$2 of NVD fixes on non-GitHub forges are not automatically collected, binary classification can be too coarse for the diversity of CWE patterns, and the common single-function framing lacks inter-procedural or data-flow context (Atiiq et al., 2024).

A recurring controversy is whether CVEfixes is heavily contaminated by public code-LLM training corpora. A 2026 perturbation-based study evaluates CVEFixes as a vulnerability-detection benchmark under consistent identifier-renaming perturbations, defines memorization advantage as $0$3, and defines per-example sensitivity as $0$4 (Euraste et al., 15 Apr 2026). In that study, CVEFixes is the only vulnerability benchmark whose median sensitivity is below $0$5 for all eight evaluated code LLMs, with reported median values from $0$6 for DeepSeek-Coder to $0$7 for StarCoder (Euraste et al., 15 Apr 2026). The paper argues that this low perturbation sensitivity challenges the assumption that performance on CVEfixes is primarily driven by direct memorization. This suggests that, despite longstanding concerns about leakage, CVEfixes remains a viable benchmark for probing learned vulnerability patterns, provided that evaluation protocols explicitly control for duplication, temporal leakage, and granularity effects.