Establishing Traceability Links between Release Notes & Software Artifacts: Practitioners' Perspectives

Abstract: Maintaining traceability links between software release notes and corresponding development artifacts, e.g., pull requests (PRs), commits, and issues, is essential for managing technical debt and ensuring maintainability. However, in open-source environments where contributors work remotely and asynchronously, establishing and maintaining these links is often error-prone, time-consuming, and frequently overlooked. Our empirical study of GitHub repositories revealed that 47% of release artifacts lacked traceability links, and 12% contained broken links. To address this gap, we first analyzed release notes to identify their What, Why, and How information and assessed how these align with PRs, commits, and issues. We curated a benchmark dataset consisting of 3,500 filtered and validated traceability link instances. Then, we implemented LLM-based approaches to automatically establish traceability links of three pairs between release note contents & PRs, release note contents & PRs and release note contents & issues. By combining the time proximity feature, the LLM-based approach, e.g., Gemini 1.5 Pro, achieved a high Precision@1 value of 0.73 for PR traceability recovery. To evaluate the usability and adoption potential of this approach, we conducted an online survey involving 33 open-source practitioners. 16% of respondents rated as very important, and 68% as somewhat important for traceability maintenance.