VulCoCo: Vulnerable Code Clone Detection

• VulCoCo is a detection method for vulnerable code clones that leverages embedding-based retrieval and LLM validation to identify exploitable logic in reused code.
• It employs a structured pipeline that includes repository processing, vector-based candidate retrieval with cosine similarity, and JSON-formatted LLM reasoning for validation.
• Evaluations on both a synthetic benchmark and real-world data show VulCoCo doubling P@5 and achieving 100% recall, leading to effective automated pull requests and new CVEs.

VulCoCo is a method for detecting vulnerable code clones (VCCs), introduced in "A Simple Yet Effective Method for Detecting Vulnerable Code Clones" (Bui et al., 22 Jul 2025). It targets the problem that code reuse can propagate exploitable logic across repositories when developers copy vulnerable code with or without edits. The method combines embedding-based retrieval with LLM validation: starting from known vulnerable functions, it retrieves syntactically or semantically similar candidate functions from a large corpus and then assesses whether those candidates still retain the same vulnerability. The work also introduces a synthetic benchmark spanning Type-1 through Type-4 clones and reports both benchmark and real-world results, including automated pull requests and newly issued CVEs.

1. Conceptual scope and problem definition

A code clone is a fragment of code copied, with or without edits, from a source location into another codebase (Bui et al., 22 Jul 2025). A vulnerable code clone is defined as a pair of code snippets that both share one of the four standard clone types, Type-1 through Type-4, and still exhibit the same security vulnerability, such as a buffer overflow or SQL injection. This definition ties clone detection to vulnerability preservation rather than to similarity alone.

The problem is operationally important because a vulnerability that has been discovered and fixed in one project may remain exploitable in unseen clones elsewhere. In that setting, VCC detection is not merely duplicate-code discovery; it requires identifying cloned logic that continues to implement the vulnerable behavior after edits, rewrites, or refactorings.

The paper situates VulCoCo against several limitations in prior approaches. Syntactic-only matching, including hashes, AST diffs, and token patterns, fails on non-trivial edits, especially Type-3 and Type-4 clones. Coarse-grained vulnerability detectors can assign a vulnerable label without tying the alert to a known CVE or providing an explanation. The reported lack of explainability reduces developer trust and actionability, while ad-hoc and unreproducible benchmarks obstruct fair comparison.

2. Processing pipeline and retrieval stage

VulCoCo operates in three phases: repository processing, embedding-based retrieval, and LLM validation (Bui et al., 22 Jul 2025). In repository processing, the method collects top-starred, active GitHub repositories in C, C++, and Java, and uses tree-sitter to extract every function or method as a standalone unit. The granularity is therefore function-level.

In the retrieval phase, each function $c$ is encoded as a $d$-dimensional vector $f(c) \in \mathbb{R}^d$ using the pretrained code embedding model jina-embeddings-v4. Similarity between functions is then computed with cosine similarity:

$$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$

All embeddings are $L_2$-normalized and indexed with FAISS. For each known vulnerable function $v$, the system retrieves all target functions $t$ such that

$\mathrm{sim}(f(v),f(t)) \ge t_s,$

where the similarity threshold $t_s$ is $0.7$ in benchmarks and $d$0 in live runs. To exclude clones that are closer to the fixed implementation than to the vulnerable one, VulCoCo discards any candidate $d$1 for which

$d$2

This retrieval design is explicitly intended to cover both syntactic and semantic similarity. The benchmark and analysis sections in the paper associate this stage with high recall, particularly on clone classes where syntax-driven methods degrade.

3. LLM validation and explainability

After retrieval, each candidate function is validated by an LLM rather than being accepted on similarity alone (Bui et al., 22 Jul 2025). The candidate $d$3, the original vulnerable function, and its fixed version are provided to Claude Sonnet 4 through a structured prompt. The model is required to return a JSON-only response in the following schema:

$t_s$2

The actual response format permits "is_vulnerable": true / false and "confidence_level": 1–5, with "justification" containing detailed reasoning. The requirement of exact JSON output is part of the protocol, not an incidental implementation detail.

The stated rationale is that the LLM can move beyond surface similarity and reason about whether the same flaw persists in the candidate. In the context established by the paper, this addresses a central weakness of prior coarse-grained vulnerability predictors: they may emit a vulnerability label, but without grounding it in a known vulnerable function or supplying an explanation that developers can inspect. VulCoCo’s validation stage is therefore intended to improve both precision and explainability.

4. Synthetic benchmark construction

Because the paper identifies a lack of reproducible vulnerable code clone benchmarks, it constructs a synthetic benchmark named SyVC (Bui et al., 22 Jul 2025). The seed data consist of 100 real vulnerable–fixed function pairs drawn from CWEs in the 2024 CWE Top 25. Since each pair contributes both a vulnerable and a fixed function, clone generation is applied to 200 functions in total.

For each of those 200 functions, five clones are generated: one Type-1 clone based on whitespace or comments, one Type-2 clone based on renaming identifiers or literals, two Type-3 clones based on statement insertion, deletion, or reordering, and one Type-4 clone based on semantic rewrite. Clones inherit the label of their origin, yielding 500 positive examples from vulnerable functions and 500 negative examples from fixed functions.

The benchmark statistics are reported as follows. Type-1 produces 200 clones with average token difference 15.79; Type-2 produces 200 clones with average token difference 9.99; Type-3 produces 400 clones with average token difference 57.42; and Type-4 produces 200 clones with average token difference 64.68. The total benchmark therefore contains 1,000 clones, evenly split between positive and negative labels. The clone distribution makes the benchmark explicitly cover both near-duplicate and semantically transformed settings.

5. Evaluation methodology and empirical results

The synthetic benchmark uses Precision@k and mean average precision (MAP) as metrics (Bui et al., 22 Jul 2025). Precision@k is defined as

$d$4

and average precision for a query $d$5 is

$d$6

with

$d$7

On SyVC, the reported scores are: Hash-based, $d$8, $d$9, $f(c) \in \mathbb{R}^d$0, $f(c) \in \mathbb{R}^d$1; ReDeBug, $f(c) \in \mathbb{R}^d$2, $f(c) \in \mathbb{R}^d$3, $f(c) \in \mathbb{R}^d$4, $f(c) \in \mathbb{R}^d$5; MOVERY, $f(c) \in \mathbb{R}^d$6, $f(c) \in \mathbb{R}^d$7, $f(c) \in \mathbb{R}^d$8, $f(c) \in \mathbb{R}^d$9; FIRE, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$0, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$1, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$2, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$3; SrcVul, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$4, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$5, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$6, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$7; and VulCoCo, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$8, $$\mathrm{sim}(u,v)=\frac{u\cdot v}{\|u\|\cdot \|v\|}.$$9, $L_2$0, $L_2$1. The paper states that VulCoCo doubles the next best $L_2$2 from MOVERY’s 32.06 to 68.25 and achieves $L_2$3 MAP over MOVERY.

The real-world evaluation draws queries from 9,539 vulnerable–fixed pairs from CleanVul and PrimeVul, using threshold $L_2$4 to limit LLM calls. Ground truth is established through manual inspection by two experts. The reported confusion statistics and derived metrics are: Hash-based, $L_2$5, $L_2$6, $L_2$7, Precision $L_2$8, Recall $L_2$9, $v$0; ReDeBug, $v$1, $v$2, $v$3, Precision $v$4, Recall $v$5, $v$6; MOVERY, $v$7, $v$8, $v$9, Precision $t$0, Recall $t$1, $t$2; FIRE, $t$3, $t$4, $t$5, Precision $t$6, Recall $t$7, $t$8; SrcVul, $t$9, $\mathrm{sim}(f(v),f(t)) \ge t_s,$0, $\mathrm{sim}(f(v),f(t)) \ge t_s,$1, Precision $\mathrm{sim}(f(v),f(t)) \ge t_s,$2, Recall $\mathrm{sim}(f(v),f(t)) \ge t_s,$3, $\mathrm{sim}(f(v),f(t)) \ge t_s,$4; and VulCoCo, $\mathrm{sim}(f(v),f(t)) \ge t_s,$5, $\mathrm{sim}(f(v),f(t)) \ge t_s,$6, $\mathrm{sim}(f(v),f(t)) \ge t_s,$7, Precision $\mathrm{sim}(f(v),f(t)) \ge t_s,$8, Recall $\mathrm{sim}(f(v),f(t)) \ge t_s,$9, $t_s$0. The paper summarizes this result by stating that VulCoCo finds all 127 real VCCs and out-performs FIRE in F1 by $t_s$1.

In deployment-oriented evaluation, 400 automated pull requests were submitted to 284 open-source projects. Among them, 75 pull requests were merged, 15 resulted in newly issued CVEs, and 89 were closed, often because of unused code or maintainer preference. These outcomes position VulCoCo not only as a benchmarked detector but also as a workflow with demonstrated effects in open-source maintenance.

6. Error profile, comparative behavior, and projected extensions

The paper’s analysis attributes VulCoCo’s strongest behavior to Type-3 and Type-4 clones, where syntax-based tools fail (Bui et al., 22 Jul 2025). The explanation given is that embeddings capture semantic likeness while the LLM reasons about preserved vulnerability logic. On SyVC, VulCoCo is reported to find 248 unique true positives not caught by any other method.

Its failure modes are also characterized. False positives occur mainly when the vulnerable and fixed versions differ by only one or two lines, so that generated clones blur the distinction and the LLM mislabels them. Another issue is JSON parse errors caused by unescaped characters in the LLM response. Performance can also degrade on very large functions, defined here as 60 or more lines, where the true fix is buried and difficult for the LLM to localize; the paper notes that slicing-based methods can succeed in those cases.

The future directions proposed in the paper are correspondingly concrete. They include prompt engineering to handle escape characters and to ask the model to express uncertainty, fine-tuning or lightweight classifiers to reduce LLM costs and improve consistency, integrating inter-procedural and context-aware signals beyond single functions, and closing the loop with automated patch suggestion based on the LLM’s justification. A plausible implication is that the method’s current architecture is already modular enough to support substitution or augmentation of the validation stage without altering the retrieval core.

7. Position within vulnerable clone detection

VulCoCo is described as a lightweight and scalable approach that combines scalable embedding-based retrieval for high recall with an LLM validation step for high precision and explainability (Bui et al., 22 Jul 2025). Within the comparison set used by the paper—Hash-based, ReDeBug, MOVERY, FIRE, and SrcVul—its distinguishing feature is the explicit coupling of known vulnerable functions, candidate retrieval from large corpora, and structured LLM reasoning over the vulnerable and fixed versions of the same code.

The method therefore sits between traditional clone detection and vulnerability classification. It does not reduce the problem to raw syntactic matching, and it does not reduce it to a binary vulnerable/non-vulnerable label detached from a known vulnerable exemplar. Instead, its operational target is the preservation of a specific vulnerability across copied or transformed code fragments. That framing is central to the paper’s contribution, as are the reproducible SyVC benchmark and the reported real-world pull-request and CVE outcomes.