COINCIDE Framework Overview

• COINCIDE Framework is a methodological apparatus for characterizing, quantifying, and reasoning about pull requests and issues coinciding with vulnerability mitigation.
• It employs a formal 5-tuple model and six modular components—data ingestion, contribution extraction, coincidence detection, taxonomy classification, metric computation, and statistical analysis—to structure empirical studies.
• Its systematic statistical approach, using tests like Kruskal–Wallis, Mann–Whitney U, and Cliff’s δ, provides insights into maintainers’ workload and the temporal dynamics of coinciding contributions.

The COINCIDE framework is a methodological and analytic apparatus for characterizing, quantifying, and reasoning about “coinciding contributions”—pull requests (PRs) and issues that are opened and closed within the window in which an npm-hosted library is actively mitigating a known vulnerability. COINCIDE formalizes methods for identifying and classifying these contributions, measuring their temporal and categorical overlap with vulnerability mitigation, and examining their relationship to maintainers’ workloads and the vulnerability-fix process itself (Rojpaisarnkit et al., 2024).

1. Formal Specification and Structure

COINCIDE is defined as the 5-tuple

$$\text{COINCIDE} = \bigl\langle \mathcal{V},\,\mathcal{C},\,\tau,\,\Phi,\,\Psi \bigr\rangle$$

where:

• $\mathcal{V}$: Set of vulnerability advisories $v_1,\ldots,v_K$, each with creation timestamp $t_k^{\mathrm{open}}$ and close timestamp $t_k^{\mathrm{close}}$.
• $\mathcal{C}$: Universe of contributions (PRs and Issues) to affected repositories.
• $\tau$: Taxonomy function $\tau:\mathcal{C}\to\{\textsc{Bug},\textsc{Feature},\textsc{Documentation},\textsc{Refactoring},\textsc{TestCase},\textsc{Other}\}$ assigning each $c_i$ to exactly one category, following the six-class scheme of Subramanian et al.
• $\Phi$: Set of metric functions $\{\phi_1, \phi_2, \ldots\}$ yielding real-valued measures of timing overlap, developer involvement, and workload.
• $\Psi$: Suite of statistical analyses, specifically Kruskal–Wallis (for multi-group), Mann–Whitney U (for two-group), and Cliff’s δ (for effect size).

2. Architectural Components and Pipeline

COINCIDE comprises six modular components supporting reproducible empirical analysis and practical workload assessment:

Module	Purpose / Action	Output
Data Ingestion & Vulnerability Matching	Query GitHub Advisory Database (2017–2023), filter advisories, match each $v_k$ to GitHub repositories	$\mathcal{V}$ mapped to repositories
Contribution Extraction	Mine all PRs and Issues from affected repositories	$\mathcal{C}$
Coincidence Detector	For each advisory $v_k$, define mitigation period $T_k$. Select contributions closed in $[t_k^{\mathrm{open}}, t_k^{\mathrm{close}}]$ as $\mathcal{C}_k$	Coinciding contributions per $v_k$
Taxonomy Classifier	Semi-automated keyword-based mapping (seeded by ≈5% manual sample) to assign $\tau$	Labeled $\mathcal{C}_k}$
Metric Engine	Compute timing, type distributions, maintainer involvement ($R_i$, $F_i$, $P^{(k)}_{t}$, $I_k$)	Metric values $\Phi$
Statistical Analyzer	Apply Kruskal–Wallis, Mann–Whitney U, Cliff’s δ to distributional questions	Hypothesis test results $\Psi$

3. Taxonomy and Classification of Contributions

All coinciding contributions $c_i$ are mapped by $\tau$ to one of six types according to keyword-based rules, with the following criteria:

• Bug: “fix”, “resolve”, “bug”, “issue”, “solve”
• Feature: “feat”, “add”, “integrate”, “support”, “improve”, “version”
• Documentation: “doc”, “readme”, “comment”, “documentation”
• Refactoring: “refactor”, “optimize”, “remove unused”
• TestCase: “test”, “unit test”, “CI”, “coverage”
• Other: No match to the above categories

A small manually labeled corpus (≈5%) is used for calibration and to verify classifier precision.

4. Quantitative Metrics and Statistical Analysis

Several metrics formalize the magnitude, timing, and relatedness of coinciding contributions with respect to the vulnerability mitigation window. For a vulnerability $v_k$ with $t_k^{\mathrm{open}}, t_k^{\mathrm{close}}$, and coinciding contributions $c_i \in \mathcal{C}_k$ (closed at $t_i^{\mathrm{close}}$):

• Mitigation Period:\ \ $T_k = t_k^{\mathrm{close}} - t_k^{\mathrm{open}}$
• Resolve-time:\ \ $R_i = t_k^{\mathrm{close}} - t_i^{\mathrm{close}}$
• Coincide-free Percentage:\ \ $$F_i = \frac{R_i}{T_k}\times 100\%,\ 0 \leq F_i \leq 100$$
• Type-Distribution per vulnerability:\ \ $$P^{(k)}_{t} = \frac{|\{c_i \in \mathcal{C}_k \mid \tau(c_i) = t \}|}{|\mathcal{C}_k|}\times 100\%$$, for each $t$
• Maintainer Involvement: Given $M_k$ (set of maintainers resolving the vulnerability), $$C^{(k)}_{\mathrm{byMant}} = \{ c_i \in \mathcal{C}_k \mid \mathrm{merged\_by}(c_i) \in M_k \}$$, then $$I_k = \frac{|C^{(k)}_{\mathrm{byMant}}|}{|\mathcal{C}_k|}\times 100\%$$

Summary metrics across $K$ vulnerabilities: $$\overline{T} = \frac{1}{K} \sum_{k=1}^K T_k,\qquad \overline{F} = \frac{1}{N} \sum_{i=1}^N F_i$$

Statistical routines include Kruskal–Wallis H (multi-group comparison), Mann–Whitney U (two-group), and Cliff’s δ (effect size), with $\lvert\delta\rvert$ interpreted as: negligible $<0.147$, small $[0.147, 0.33)$, medium $[0.33, 0.474)$, large $\geq 0.474$.

5. Methodological Pipeline

The COINCIDE methodology formalizes a stepwise pipeline for empirical studies:

1. Data Collection: Download all GitHub advisories (Oct 2017–Apr 2023), filter to 554 npm advisories, match to 348 repositories.
2. Contribution Mining: Extract all PRs (402,000) and Issues (823,000) for those repositories.
3. Coincidence Detection: For each $v_k$, compute $T_k$ and extract contributions closed within $[t_k^{\mathrm{open}}, t_k^{\mathrm{close}}]$: 2,159 PRs + 2,547 Issues.
4. Labeling and Taxonomy: Keyword-driven classifier (seeded by 30-item manual sample) assigns each $c_i$ to a category.
5. Metric Computation: Calculate $R_i$, $F_i$, $P^{(k)}_{t}$, $I_k$ for relevant sets.
6. Statistical Analysis: Apply Kruskal–Wallis, Mann–Whitney U, and Cliff’s δ to test research questions.
7. Manual Deep-Dive: Sample 326 PRs and 334 Issues for round-table coding of relatedness to the vulnerability (maintainer overlap, explicit security mentions).

6. Empirical Patterns and Key Findings

Analysis of 4,699 coinciding PRs and Issues reveals the following:

• Category Distributions: Among PRs, 30.97% are Bug, 33.50% Feature, 10% Documentation, 9% Refactoring, 7% TestCase, 9% Other. Issues show similar Bug/Feature dominance, but “Other” is more variable.
• Timing Overlap: Average coincide-free percent $\overline{F} \approx 45.89\%$, implying that maintainers spend ≈54% of the mitigation period working on non-vulnerability contributions. Some contributions resolve at window start ($F_i=100\%$), others at end ($F_i=0\%$). No significant difference in coincide-freeness between PRs and Issues (Mann–Whitney U $p<0.01$), unless stratified by disclosure timing.
• Relatedness to Vulnerability: Maintainer overlap ($I_k$) is $\approx 37.99\%$ for PRs and $20.18\%$ for Issues. Only 2.2% of sampled contributions mention security explicitly or update vulnerable dependencies. Thus, approximately 68% of coinciding contributions have no relation to the vulnerability other than temporal co-occurrence.
• Statistical Differences: Merge rates across categories are significantly different (Kruskal–Wallis $p<0.001$, large $\delta$ for Bug vs Feature).

7. Practical Guidance and Tooling Implications

COINCIDE motivates several recommendations and engineering artifacts for workflow improvement:

• Priority Dashboards: Display a “coinciding workload” indicator in GitHub Security Advisories, visualizing concurrent non-security PRs or Issues during mitigation.
• Adaptive Triage Support: Extend triage bots (e.g., Dependabot) to tag contributions by taxonomy and historical $F_i$-based impact projections.
• Notification Throttling: Recommend deferring low-priority PRs/Issues when >50% of window is occupied by coinciding work.
• Workload Forecasting: Surface $\overline{F}$ and $I_k$ metrics to support capacity planning (e.g., reviewer ramp-up).
• Security Mention Extraction: Apply static heuristics to prioritize PRs/Issues referencing affected dependencies or CVEs.

By integrating structured taxonomy, timeline alignment, overlap quantification, and statistical rigor, COINCIDE establishes a repeatable analytic lens for understanding non-security maintenance work during security patching windows. Its modular methodology enables both retrospective studies and practical workload management for open-source maintainers (Rojpaisarnkit et al., 2024).