Change-Aware Defect Prediction

• Change-aware defect prediction is a set of software analytics techniques that leverage code change patterns and process metrics to accurately identify defect-prone modifications.
• It integrates fine-grained semantic embeddings, co-change graphs, hypergraph models, and commit-size stratification to enhance predictive accuracy.
• The approach guides resource allocation by prioritizing high-risk commits while addressing challenges like label persistence and cross-project generalization.

Change-aware defect prediction is a family of software analytics techniques that identify defect-prone software artifacts by modeling and learning from the patterns, semantics, and context of code changes. Rather than relying solely on static product metrics, change-aware approaches explicitly quantify the impact and structure of code edits, developer actions, and co-change histories to prioritize high-risk modifications and optimize inspection or testing effort. Recent advances integrate process metrics, graph and hypergraph models, fine-grained semantic embeddings, and robust change labeling to improve both accuracy and practical utility.

1. Foundations and Motivations

Change-aware defect prediction emerged out of recognition that static code metrics—e.g., lines of code, cyclomatic complexity, coupling—fail to capture the actual evolutionary dynamics most correlated with post-release defects. Classic studies established that process metrics, tracking how and where code evolves (e.g., commit count, developer activity, code churn, change entropy), are language-independent and often outperform product metrics for defect identification (Hrishikesh et al., 25 Apr 2025). Fine-grained defect prediction shifts the focus from entire files, classes, or modules to specific changes, typically at the commit, file-change, or even function level. This “just-in-time” (JIT) orientation delivers actionable risk scores contemporaneously with developer activity, naturally aligning with modern CI pipelines and code-review workflows (Haldar et al., 2024).

Key motivations include reducing wasted inspection effort, guiding reviewer attention to defect-inducing commits, overcoming label-persistence bias in snapshot-based evaluations, and enabling semantics- and context-aware modeling of software maintenance (Hesamolhokama et al., 29 Dec 2025). The field has expanded from basic process metrics to deep learning on structured change data, graph/hypergraph representations, and nuanced reasoning about developer intentions and refactoring semantics.

2. Core Metrics and Formal Models

2.1 Process and Change-Aware Metrics

Canonical change-aware metrics include:

• Lines Added/Deleted (LA/LD): Basic churn quantification per change instance.
• Diffusion: NS (modified subsystems), ND (directories), NF (files), and entropy of line distribution across files (ENT). Change entropy is mathematically formalized as

$H(S) = -\sum_k p_k \log p_k$

with $p_k$ the proportion of changes to file $f_k$ in the release window (Hrishikesh et al., 25 Apr 2025, Bludau et al., 2022).

• Developer and File History: NDEV (number of prior developers), AGE (recency of edit), NUC (number of unique historical changes), developer experience (EXP, REXP, SEXP).
• Review and Workflow Features: Number of review iterations, PR-specific metrics, time since last release, cumulative changes within PR, etc.

Recent advances introduce commit-size-aware vectors for each process metric, stratifying the metric by the distribution over 1–100 file commit strata, capturing the non-linear relationship between commit coarseness and defect risk (Kumar et al., 1 Apr 2026).

2.2 Co-Change Graphs and Entropy

Defect prediction is strengthened by modeling not just the frequency but structure of co-changes:

• Co-Change Graph Entropy (H₍CC₎) constructs an unweighted undirected graph $G=(V, E)$ where files are nodes, edges connect all files changed together in a commit (edges deduplicated), and node degrees power the definition of each file’s co-change probability and associated entropy:

$H'_{k} = p'_k \times H'(S)$

This captures “scattering,” or how widely a file is coupled by shared modifications (Hrishikesh et al., 25 Apr 2025).

2.3 Hypergraph and Higher-Order Centralities

Traditional pairwise graphs reduce groupwise changes to cliques, losing critical context. In the hyper co-change graph model, each commit (with up to 100 files changed) defines a hyperedge, retaining the exact commit-size semantics. Node (file) importance is decomposed via vector centralities

$\vec x_j^X = (x_{j,2}^X, \dots, x_{j,H}^X)$

aggregating classical measures (betweenness, eigenvector, etc.) per order (Kumar et al., 1 Apr 2026).

3. Datasets, Labeling, and Evaluation Protocols

3.1 Datasets and Label Quality

High-fidelity change-aware prediction demands fine-grained, accurately labeled datasets. SZZ-based heuristics (tracing defect-fixing commits back to defect-inducing ones) are ubiquitous but suffer from substantial noise unless augmented.

• ReDef (Nam et al., 11 Sep 2025): Applies explicit revert detection and multi-stage GPT-assisted adjudication to achieve 92% label precision at the function level, exceeding prior benchmarks.
• JIT-Defects4J: Refactoring-aware relabeling using Code Change Tactics (CAT) analysis improves labeling accuracy by 13.7%, correcting both spurious bug-inducing labels on pure refactorings and uncovering hidden defect-inducing edits (Niu et al., 25 Jul 2025).
• Continuous Defect Prediction (CDP) (Madeyski et al., 2017): Provides per-file, per-build metrics for over 11 million change events across 1,265 projects.

3.2 Evaluation Protocols

Effort-aware metrics are standard: Recall@20%, F1@20%, area under the cost-effectiveness curve (AUCEC). Class balance is managed by SMOTE or downsampling; evaluation is typically through stratified or time-based cross-validation (Bludau et al., 2022, Fu et al., 2017). Label-persistence bias, whereby static defect-prediction models “memorize” labels due to extreme file label consistency across versions, is a significant confounder. Change-aware file-level protocols partition samples by status transition (introduction, removal, persistence of defects) and emphasize balanced accuracy across these subsets (Hesamolhokama et al., 29 Dec 2025).

4. Advanced Modeling Techniques

4.1 Classical and Graph-based ML

Random forest and logistic regression are effective baselines, but graph-based learning over developer–file bipartite graphs captures collaboration complexity and boosts F1 up to 77.55% and MCC to 53.16%, outperforming traditional feature vectors (Bryan et al., 2021). Node2vec embeddings and topological centralities further enrich the feature space.

4.2 Sequential and Context-Aware Deep Learning

• HVSM-RNN: Concatenates per-release metric vectors for each file across multiple versions and feeds the sequence to a recurrent neural network, yielding state-of-the-art effort-aware rankings ([email protected] = 0.524 vs. next-best 0.486) (Liu et al., 2017).
• BiCC-BERT: Pre-trains a bi-modal transformer on code change and commit message pairs with the Replaced Message Identification objective, achieving an F1 of 0.478 vs. 0.431 for prior SOTA across 27,391 commits (Jiang et al., 2024).
• CompDefect: Jointly models function-level defect identification, defect type categorization, and one-statement patch generation via a GraphCodeBERT encoder, neural-tensor change fusion, and multi-task supervision. This yields F1=0.679 versus 0.414 (DeepJIT) at the identification task (Ni et al., 2022).

4.3 LLM-Driven and Agentic Multi-Agent Approaches

Static snapshot models are outperformed by change-conditioned reasoning, especially for defect status transitions. Structured LLM multi-agent debate protocols—where “analyzer,” “proposer,” “skeptic,” and “judge” agents exchange arguments about file-level changes—improve accuracy on rare “defect-introducing” or “defect-removal” transitions (D01/B10), with balanced HMB/HMD harmonic means up to 0.60/0.40 (Hesamolhokama et al., 29 Dec 2025).

4.4 Representation Learning and Encoding

Experimental analyses show compact diff-style encodings (e.g., “diff with tags,” “added→deleted blocks”) for pre-trained LLMs yield 4–5 percentage point F1 gains over whole-function formats, with the best F1=0.397 on ReDef. However, counterfactual encoding perturbations reveal that current PLMs rely on superficial cues over true edit semantics, underscoring the need for edit-sensitive pre-training (Nam et al., 11 Sep 2025).

5. Feature Importance, Best Practices, and Limitations

Feature attribution studies leveraging integrated gradients and SHAP indicate that lines added (LA), change entropy, and developer experience (EXP, REXP, SEXP) are the most robust predictors of defect risk, but importance weights vary between attribution techniques. The “Fix” flag is highly influential when available. AST-level change features, particularly counts and depth of added AST nodes, are consistently among the highest-impact features across projects (Haldar et al., 2024, Bludau et al., 2022).

Combining heterogeneous features—process, semantic, AST, workflow—yields the best predictive performance with consistent ranking in non-parametric group tests (Wilcoxon, Friedman/Nemenyi). Graph and hypergraph centralities and commit-size stratification further boost discrimination and calibration (Kumar et al., 1 Apr 2026).

Empirical limitations include dataset homogeneity (predominantly Java and C/C++), high dimensionality (hundreds to thousands of features per sample when using hypergraph centralities), incomplete refactoring-aware labeling, and the persistent challenge of cross-project generalization.

6. Open Challenges and Future Research Directions

Key future directions identified include:

• Multi-language and industrial validation: Extending benchmarks and empirical studies beyond curated open-source Java/C++ ecosystems to heterogeneous industry settings (Hrishikesh et al., 25 Apr 2025, Kumar et al., 1 Apr 2026).
• Refactoring-aware and semantic labeling: Systematically disentangling refactoring and propagation edits from bug-inducing changes, and integrating refactoring-aware metrics for all model types (Niu et al., 25 Jul 2025).
• Higher-order relational modeling: Exploring hypergraph neural networks, heterogeneous information networks, and attention over code change structure to better encode multi-file, multi-actor change semantics (Kumar et al., 1 Apr 2026).
• Semantic and intent-aware representation learning: Developing PLMs explicitly pre-trained on diff reasoning, masked edit modeling, and code–intent alignment (as in RMI or bi-modal objectives) (Jiang et al., 2024, Nam et al., 11 Sep 2025).
• Effort-aware, explainable, and granular evaluation: Incorporating finer-grained labels (e.g., hunk- or line-level), cost-effectiveness metrics, and interpretable attributions to minimize unnecessary inspection and maximize actionable insights (Bludau et al., 2022, Ng et al., 2021).
• Hybrid and unsupervised–supervised ensembles: Leveraging lightweight unsupervised metrics for recall-oriented use cases, pruning weak predictors with minimal labeled data for practical bootstrap (Fu et al., 2017). Use of OneWay (supervised metric selection wrapper) is an efficient approach where label sparsity is a concern.

7. Practical Implications and Recommendations

Change-aware defect prediction methods provide practical value for prioritizing risky commits, optimizing resource allocation in testing or inspection, and enabling continuous quality analytics. Integrated into code review or CI/CD pipelines, these models assist reviewers in triaging high-scatter or high-entropy changes, recent and infrequent file modifications, and complex, large co-change contexts. For effective deployment:

• Aggregate and stratify process metrics by commit size and history.
• Incorporate co-change graph and hypergraph centralities for structural insights.
• Employ diff-centric, intent-aware encoding for semantic modeling.
• Regularly retrain models on recent history (rolling or time-wise splits) to mitigate concept drift.
• Vet and, where possible, correct dataset labels for spurious bug-inducing attribution, particularly in the presence of large refactorings and propagations.
• Monitor both overall and subset-balanced performance (introductions/removals vs. persistence).

Limitations in scope, computational cost, and cross-project robustness remain. Ongoing research address these via improved dataset curation, innovative graph and neural architectures, and fine-grained, interpretable evaluation frameworks.