Co-Change Graph Entropy: A New Process Metric for Defect Prediction

Abstract: Process metrics, valued for their language independence and ease of collection, have been shown to outperform product metrics in defect prediction. Among these, change entropy (Hassan, 2009) is widely used at the file level and has proven highly effective. Additionally, past research suggests that co-change patterns provide valuable insights into software quality. Building on these findings, we introduce Co-Change Graph Entropy, a novel metric that models co-changes as a graph to quantify co-change scattering. Experiments on eight Apache projects reveal a significant correlation between co-change entropy and defect counts at the file level, with a Pearson correlation coefficient of up to 0.54. In filelevel defect classification, replacing change entropy with co-change entropy improves AUROC in 72.5% of cases and MCC in 62.5% across 40 experimental settings (five machine learning classifiers and eight projects), though these improvements are not statistically significant. However, when co-change entropy is combined with change entropy, AUROC improves in 82.5% of cases and MCC in 65%, with statistically significant gains confirmed via the Friedman test followed by the post-hoc Nemenyi test. These results indicate that co-change entropy complements change entropy, significantly enhancing defect classification performance and underscoring its practical importance in defect prediction.

Co-Change Graph Entropy: Advancing Defect Prediction Metrics

The paper "Co-Change Graph Entropy: A New Process Metric for Defect Prediction," authored by Ethari Hrishikesh et al., represents a refined approach to enhance the efficacy of defect prediction in software engineering through innovative metrics. Software defect prediction remains a pivotal aspect of software development, aimed at identifying modules susceptible to errors and optimizing resource allocation for quality enhancement.

Introduction of Co-Change Graph Entropy

Building upon established insights from process metrics, which trumps product metrics due to practical advantages and predictive prowess, the paper introduces Co-Change Graph Entropy, a novel metric designed to capture co-change patterns. Instead of focusing solely on the scattering of changes, known as change entropy, the authors leverage graph theory to model co-change dispersion. This approach fundamentally differentiates co-change entropy from existing measures by considering the co-change probability within a graph structure.

Experimental Analysis and Findings

Experiments were conducted on eight Apache projects using established datasets, notably the SmartSHARK dataset. The findings exhibited a significant correlation between co-change entropy and defect count, with Pearson correlation coefficients reaching as high as 0.54 under certain conditions. While change entropy showed stronger correlation, the inclusion of co-change entropy yielded notable improvements in defect classification performance.

To assess efficacy in practical scenarios, the authors introduced three distinct metric sets—P+C (including change entropy), P+Co (substituting change entropy with co-change entropy), and P+C+Co (integrating both metrics). Results demonstrated that replacing change entropy with co-change entropy enhanced AUROC measures in 72.5% of cases and achieved statistically significant improvements when both metrics were combined.

Implications and Future Research Directions

The paper implies that co-change entropy provides complementary information that strengthens the predictive capacity when combined with traditional process metrics. The statistical significance of improvements in AUROC values suggests the practical utility of integrating co-change patterns in defect prediction models.

The research opens new avenues for future exploration within AI and software quality assessment. Considering the potential of advanced graph-based constructs like heterogeneous information networks offers promising prospects for capturing nuanced software evolution patterns. Future work could extend these methodologies, applying co-change entropy to a portfolio of diverse programming languages beyond Java, thereby testing the generalizability and robustness of results across software ecosystems.

Additionally, while the performance improvements are quantitatively modest, they stress the merit of pursuing entropy-based graph measures to refine defect prediction techniques further. Such efforts would contribute to the rich tapestry of defect prediction methodologies, potentially leading to the development of more comprehensive, effective systems for identifying defect-prone modules early in the development lifecycle.

In summary, the contribution of co-change graph entropy marks a meaningful stride in the continuous endeavor of improving software defect prediction accuracy and reliability through innovative metric development. The rigor and transparency of the research, underscored by the availability of reproducible artifacts, set a benchmark in the pursuit of advancing the scientific understanding of defect-proneness in software systems.