Generalization of API Usage and Test Coverage Patterns in Java OSS

Determine whether the patterns observed in API Usage analytics and test coverage—derived from an empirical study of ten popular Java libraries and their 50-project dependent ecosystems—hold for larger samples and generalize across the entire Java ecosystem.

Background

The paper introduces community-based analytics to support open-source Java library maintainers, including API Usage analytics and two coverage-related metrics (Usage-Based API Test Coverage and Community Test Coverage). The empirical study examined ten widely used Java libraries, each with a curated set of 50 dependents aligned to the latest major version, and reported usage and coverage patterns.

In the Threats to Validity section, the authors explicitly acknowledge uncertainty regarding whether the observed patterns (e.g., the proportion of API methods used by dependents and the share of those methods covered by tests) would persist with larger samples or across the entire Java ecosystem. This raises an unresolved question about the generalizability of the study’s findings beyond the evaluated dataset.