Defects4J 1.2: Java Faults Dataset

Updated 6 April 2026

Defects4J 1.2 is a curated dataset of real-world Java faults providing paired buggy and fixed versions, comprehensive test suites, and ground-truth patches.
It categorizes 488 faults through manual control-flow and data-flow classifications, enabling precise fault localization and repair analysis.
Empirical patch analysis shows small, localized edits with dominant repair actions like method-call, conditional-branch, and assignment additions.

Defects4J 1.2 is a curated dataset of real-world Java software faults extensively used in program analysis, automated program repair (APR), and fault localization (FL) research. The dataset provides both buggy and fixed program versions, along with comprehensive test suites, ground-truth patch information, and evolves a rich tradition of empirical benchmark design rooted in the needs of reproducibility and realism in fault studies. Defects4J 1.2 comprises 488 documented faults across seven mature open-source Java projects, with rigorous classification and patch analyses enabling systematic investigation into repair, debugging, and software quality assurance (Spuy et al., 4 Feb 2025).

1. Dataset Scope, Projects, and Versioning

Defects4J 1.2 targets Java and includes seven projects: Chart (JFreeChart), Closure (Google Closure Compiler), Lang (Apache Commons Lang), Math (Apache Commons Math), Mockito (Mockito), Time (Joda-Time), and Jsoup (Jsoup). Each project contributes a set of sequentially numbered, independently reproducible buggy versions. The version distribution and project-level sizes are summarized in the following table:

Project	Faults	% of total	Project size (KLOC)
Chart	26	5.3%	96
Closure	133	27.3%	90
Lang	65	13.3%	22
Math	106	21.7%	85
Mockito	38	7.8%	11
Time	27	5.5%	28
Jsoup	93	19.1%	10
Total	488	100.0%

Bug instances are uniquely referenced in the format [Project]-[BugID], e.g., Chart-5, and each is distributed with (a) a buggy and a fixed code checkout, (b) a test suite with at least one fault-exposing test, (c) a unified patch file, and (d) metadata such as commit information and bug tracker links (Spuy et al., 4 Feb 2025, Martinez et al., 2018).

2. Fault Taxonomy: Control- and Data-Flow Graph Classification

To move beyond repair-pattern proxies, Defects4J 1.2 faults have been manually assigned to a set of semantically meaningful classes based on control-flow graphs (CFGs) and data-flow graphs (DFGs) (Spuy et al., 4 Feb 2025). Each program version is modeled as a flow graph $G = (V, E_C, E_D)$ , where $V$ are statement-vertices, $E_C \subseteq V \times V$ are CFG edges, and $E_D \subseteq V \times V$ are DFG (def-use) edges.

Control-Flow Faults (CF) [6 subclasses, summarized]:

CF1: Unconditional control-flow target faults, with subclasses:
- Statement order faults
- Jump faults (e.g., break/return/throw)
- Method-call faults
CF2: Predicate-related control-flow faults
- CF2.1: Predicate vertex faults (incorrect Boolean condition)
- CF2.2: Guard faults (missing/extraneous predicate node)
- CF2.3: Block faults (entire guarded subgraphs missing/extraneous)

Data-Flow Faults (DF) [2 subclasses]:

DF1: Definition faults, where a definition node writes a variable incorrectly (wrong value, location, or missing)
DF2: Use faults, where the use node reads from an incorrect definition

Faults often exhibit multiple classifications; across the 488 faults, the mean number of classes per fault is $\mu=2.04$ , $\sigma=1.03$ , with most faults assigned to 1–3 classes.

Fault Class	Count	% of 488
CF1.1 Order	16	3.3%
CF1.2 Jump	63	12.9%
CF1.3 Call	139	28.5%
CF2.1 Predicate	104	21.3%
CF2.2 Guard	194	39.8%
CF2.3 Block	94	19.3%
DF1 Definition	330	67.6%
DF2 Use	56	11.5%

Assignment overlaps are common: 31.6% faults are pure control-flow, 16.6% are pure data-flow, and 51.8% are mixed CF/DF.

3. Patch Structure: Size, Spreading, and Action Patterns

Empirical patch analysis reveals that Defects4J 1.2 faults are typically repaired with small, localized and syntactically simple edits (Sobreira et al., 2018). The patch size $S(p)$ , defined as the sum of added, removed, and modified lines per patch, has:

Median: 4 lines
25th percentile: 2 lines
75th percentile: 9 lines
95th percentile: 22 lines
Maximum observed: 54 lines

Patch spreading, defined as the number of files and contiguous change blocks (chunks), further quantifies locality:

92.41% of patches touch exactly one file
38.23% have no spreading (single-chunk)
50% exhibit ≤ 2 chunks

The top three repair actions—method-call addition (mcA, 61.5%), conditional-branch addition (cndA, 52.2%), and assignment addition (asgnA, 34.4%)—co-occur in 77.2% of patches. These directly inform the design of automated repair operators.

Patch repair patterns include:

Conditional block modification (42.8%)
Expression fix (32.9%)
Wrap-with construct (27.3%)
Single line changes (24.8%)
Other patterns, including wrong references, missing null-checks, copy-paste fixes, constant changes, and code moves, collectively account for 94.4% of all patches.

4. Dataset Engineering, Distribution, and Usage

The dataset is structured to support reproducible experimentation and automation. The directory organization is uniform across versions, exposing for each bug: both buggy and fixed source subtrees, a versioned test suite (JUnit), build files, and metadata. The defects4j CLI provides commands to automate checkout, compilation, testing, and access to patch and fault location information (Martinez et al., 2018).

A typical workflow:

Checkout buggy version: defects4j checkout -p [project] -v [bugID]b ...
Run and confirm regression test(s) fail.
Checkout fixed version: defects4j checkout -p [project] -v [bugID]f ...
Run tests (should all pass).
Analyze patch, test coverage, and metadata.

Source code and bug artifacts are hosted at https://github.com/rjust/defects4j, licensed under MIT.

5. Implications for Fault Localization and Automated Program Repair

The semantic classification of faults underlines the limitations of repair- or test-based taxonomies and demonstrates the intertwined nature of real-world Java faults. Definition faults (DF1) are most prevalent (67.6%), yet more than half of bugs require recognition of control-flow phenomena (particularly guard faults, 39.8%) (Spuy et al., 4 Feb 2025).

Fault Localization: Spectrum-based FL performs significantly poorer on pure control-flow faults, particularly when faults involve missing or extraneous code (guards). These findings motivate specialized FL approaches, such as guard-synthesis-aware localization or class-dependent SBFL formula selection (Spuy et al., 4 Feb 2025).

Automated Program Repair: Patch-statistics and class-frequency can prioritize repair template classes—definition edits should be attempted first due to their prevalence. Multi-class assignments reveal that robust APR tools should encompass code addition, guard synthesis, and multi-location repair, reflecting the empirical distribution in Defects4J (Sobreira et al., 2018).

A plausible implication is that joint CF/DF analysis, including the prediction of likely fault class from dynamic coverage or spectra, can reduce debugging and repair effort.

6. Extensions and Ongoing Developments

Recent work addresses the single-fault-per-version constraint in Defects4J 1.2 by constructing multi-fault variants (Defects4J-mf), where multiple previously catalogued faults are concurrently present in older versions using precise test-transplantation and fault-location backtracking (Callaghan et al., 2024). These variants average 8.3 distinct faults per program version and retain granular per-fault metadata, offering a unique substrate for evaluating multi-fault FL and APR techniques.

Multi-fault entries maintain the original workflow via an extended CLI, include new test harnesses surfacing the multiple faults, and enable reliable top-N localization metrics. The dataset is available at https://github.com/DCallaz/defects4j-mf.

7. Summary and Research Impact

Defects4J 1.2 provides a scientifically rigorous, well-characterized, and widely adopted benchmark for empirical studies in Java fault localization, program repair, and debugging tool evaluation. The comprehensive fault graph taxonomy, patch property cataloguing, and reproducible infrastructure render it indispensable for controlled, comparable evaluations of FL and APR techniques. The dataset’s empirical findings continue to inform theoretical work in fault class prediction, CF/DF-centric repair, and regression test generation, underscoring its centrality in software engineering research (Spuy et al., 4 Feb 2025, Martinez et al., 2018, Sobreira et al., 2018, Callaghan et al., 2024).