Assess coverage generalizability beyond highly popular repositories

Determine whether the SBOM coverage figures obtained from GitHub repositories with at least 1,000 stars differ for a less popularity-skewed sample of projects and dependencies.

Background

The empirical evaluation uses GitHub repositories with at least 1,000 stars, which favors actively maintained and highly visible projects over the smaller and less curated repositories that constitute many real-world dependencies. The authors acknowledge that this selection criterion may affect the reported coverage results but do not evaluate the effect. The unresolved issue is whether the observed coverage of Syft, Trivy, and cdxgen generalizes to a less popularity-skewed population.

References

We do not measure how this affects our coverage figures, and cannot rule out that they would differ on a less popularity-skewed sample.

— Mind the Gap: How SBOM Specification Ambiguities Lead to Divergent Software Bills of Materials. An Empirical Tool Study  (2609.19920 - Prado et al., 17 Sep 2026) in Section 5.1, “Dataset selection” (Section \ref{sec:limitations})