Integrate executable validation into vulnerability-dataset construction

Integrate patch-to-proof-of-concept generation and executable validation into software-vulnerability dataset construction, while characterizing the remaining reproduction capability and publishing the cases that resist automated reproduction.

Background

The paper identifies a general shortage of independently verifiable vulnerability labels: most CVEs lack publicly available proof-of-concept exploits, most patches do not include tests, and executable benchmark construction retains only a small fraction of candidate vulnerabilities. Existing patch-to-proof-of-concept and exploit-generation systems demonstrate partial success, but they are generally developed as standalone contributions rather than as components of dataset-construction pipelines.

The authors therefore identify an unresolved problem involving both the limitations of current automated reproduction methods and their integration into data collection. They specifically motivate reporting validation yields, applying patch-to-proof-of-concept generation during collection, and retaining failed reproduction cases so that the field can characterize the vulnerabilities that remain difficult to reproduce.

References

However, the review value of the remaining findings is unknown because dynamic confirmation failed for reasons unrelated to finding validity.

— LLMSec-AV: A Vulnerability Taxonomy and LLM-Driven Software Weakness Discovery Framework for Autonomous Vehicles  (2609.09386 - Haque et al., 8 Sep 2026) in Section 7, Discussion

The open problem concerns both the remaining reproduction capability and the integration of that capability into dataset construction.

— The Data Problem in Software Vulnerability Analysis: Artifacts, Quality, and Consumption  (2609.01503 - Nong et al., 1 Sep 2026) in Section 6, “Future Directions,” subsection “Make executable validation a construction step”