CodeGenLink: AI Code Provenance Tool

• CodeGenLink is a provenance-assistance tool that identifies web pages containing code similar to AI-generated snippets and suggests potential licenses.
• It integrates with Visual Studio Code via GitHub Copilot, employing clone detection and textual similarity analysis with adjustable thresholds to filter results.
• Preliminary evaluations on CodeSearchNet and CoderEval demonstrate high precision in linking and license retrieval, addressing provenance opacity in autogenerated code workflows.

CodeGenLink is a provenance-assistance tool for AI-generated code that helps developers find web pages containing code very similar to automatically generated code and, when possible, surface the likely license associated with that code. Implemented as a GitHub CoPilot extension for Visual Studio Code, it addresses a specific gap in autogenerated-code workflows: once code is produced in a Copilot or chat interaction, the usual web-based traceability path is missing, even though concerns about trustworthiness, attribution, copyright, and redistribution remain (Bifolco et al., 1 Oct 2025).

1. Provenance opacity in AI-generated code

CodeGenLink is motivated by the difference between direct web reuse and LLM-generated reuse. When developers copy code directly from the web, they can often inspect the source repository or page, infer ownership, and check licensing terms. With LLM-generated code, that provenance is obscured: the output appears detached from any visible source, even though prior work cited by the paper reports that models may memorize or reproduce training code, including copyleft code. The resulting problem is framed as one of provenance opacity in autogenerated code workflows, with concrete implications for copyright and licensing risk assessment (Bifolco et al., 1 Oct 2025).

The system is explicitly not presented as a mechanism for proving origin. Its purpose is narrower and more operational: help developers find links containing code very similar to a generated snippet, and, when possible, indicate the license of the likely origin. This positioning is consistent with earlier empirical evidence that LLM-based assistants provide a mixture of relevant and irrelevant links and still leave developers with substantial “provenance debt”; one such study reported that 66% of the links from Bing CoPilot and 28% from Google Gemini were relevant, while still concluding that LLM-based assistants suffer from serious provenance debt (Bifolco et al., 21 Jan 2025).

2. Integration model and usage modes

The integration context is central to the design. CodeGenLink is implemented as a Copilot extension using Visual Studio Code’s Chat Participants interface, and it targets developers working inside Visual Studio Code with GitHub Copilot. The tool exposes two usage modes.

In Mode 1, a developer invokes a dedicated @CodeGenLink chat participant. After the LLM generates code, CodeGenLink automatically uses that output to search for relevant links. The paper states that the code-generation prompt is inherited from the authors’ prior study, while the follow-up retrieval prompt is:

“Search the Web to find links where I can get more information about the generated code present in this response.”

In Mode 2, a developer selects code already present in the editor and invokes a @LinkSearcher participant to search for similar code sources. This mode can be used with code produced by Copilot completion, but the paper states that it can in principle operate on any selected snippet. In the selected-code workflow, the command named “Send Selected Code to CodeGenLink” triggers the same downstream analysis and visualization. The unifying interaction model is “generate or select code, then ask for likely source links and license hints without leaving the IDE” (Bifolco et al., 1 Oct 2025).

3. Retrieval, extraction, and similarity analysis

The end-to-end workflow is a pipeline. After the LLM responds, an LLM Response Processor converts the unstructured output—generated code and returned links—into structured data suitable for analysis. A Snippet Extractor then fetches the retrieved pages and extracts code snippets using domain-specific rules, because different sites expose code differently. The paper gives concrete examples: Stack Overflow code is extracted from nested <pre> and <code> tags, while GitHub code is accessed through raw URLs. The current implementation explicitly supports GitHub, Stack Overflow, and GeeksForGeeks, and is intended to be extensible by adding extraction rules for more domains (Bifolco et al., 1 Oct 2025).

Filtering occurs in the Analysis Coordinator, which pairs the generated or selected snippet with snippets extracted from each retrieved link and computes similarity. Candidate links returned by the search-capable LLM are described as noisy: some may be topically related to the task but not contain code close to the generated snippet. To reduce this noise, CodeGenLink applies either or both of two analyses: textual similarity and clone detection.

The clone detector wraps CCFinderSW. The textual analyzer tokenizes each snippet, converts it into a vector, and computes cosine similarity between vectors. The paper does not provide pseudocode, a formal scoring function, or exact tokenization and vectorization details beyond this description, and it does not specify a combined ranking formula. The reported decision logic is threshold-based: “only code pairs exceeding a given similarity threshold are retained.” In the evaluation, thresholds are described as “user-determined thresholds for the maximum cloning ratio and textual similarity,” explored in the range $[0.5-0.8]$. Similarity is computed at the snippet level; the paper does not mention file-level, AST-level, embedding-level, or line-level matching (Bifolco et al., 1 Oct 2025).

The paper repeatedly emphasizes that similarity is evidence rather than proof. In practice, it serves two roles: eliminating unrelated search results and identifying links that are “likely” origins because they contain closely matching code.

4. License inference and system architecture

License identification is the second major function. The License Identifier uses different strategies depending on the retrieved URL. For GitHub repositories, it first queries the GitHub API for repository metadata, including the license field. If that metadata is absent, it searches the repository for a LICENSE or similarly named file and analyzes it with Google’s License Classifier. For some domains, the tool relies on known site-wide licensing policy; the paper gives Stack Overflow as the example, whose code snippets are treated as licensed under CC BY-SA 4.0. To support that, CodeGenLink maintains a domain/license map. For other web pages, it scans the HTML for license keywords from a predefined SPDX-based list and uses any matches to infer a license (Bifolco et al., 1 Oct 2025).

The output is intentionally cautious. In the user interface, the tool reports a suggestion such as Possible license found: <<LICENSE>>. The paper notes that this may be inaccurate, especially when the detected license applies to the page content in general rather than to the embedded code itself. In many cases the tool can only infer repository-level or page-level metadata rather than exact licensing of the precise snippet.

Architecturally, CodeGenLink depends on an LLM with web search support; the implementation used OpenAI’s web-search-preview, though the paper states that other similar APIs could be substituted. It also depends on the GitHub API, CCFinderSW, and Google’s License Classifier. Installation requires Python dependencies and external tools, specifically CCFinderSW and Google License Identifier/Classifier. Results are displayed inside VS Code via a WebView that renders a table of surviving links and their associated license information, if any. The paper’s screenshots describe a Copilot chat request such as “extract zip file recursively” on the left and, on the right, an embedded HTML panel listing URLs judged likely to be the origin and the corresponding possible license (Bifolco et al., 1 Oct 2025).

5. Preliminary evaluation

The evaluation is explicitly preliminary and focuses on precision rather than recall. Recall was not evaluated because the full ground-truth set of true source links for generated code is typically unknown and no benchmark of generated snippets paired with exact source URLs exists. The dataset contains two parts: 101 coding tasks from CodeSearchNet, comprising 46 Java and 55 Python tasks, and 100 randomly selected coding tasks from CoderEval, with 50 per language. For each task, the authors first asked Copilot to generate code and then used that generated code to ask Copilot chat to retrieve related links. Two authors independently manually assessed each link and resolved disagreements through discussion; no inter-rater statistic is reported (Bifolco et al., 1 Oct 2025).

On CodeSearchNet, cloning-ratio precision was 0.88 at threshold 0.5 with 1 average returned link, 0.88 at 0.6 with 1, 0.94 at 0.7 with 1, and 1.00 at 0.8 with 1. Textual-similarity precision was 0.58 at 0.5 with 2 average links, 0.59 at 0.6 with 2, 0.70 at 0.7 with 2, and 0.81 at 0.8 with 1. On CoderEval, cloning-ratio precision was 0.90 at 0.5 with 1 link, 0.94 at 0.6 with 1, 0.94 at 0.7 with 1, and 1.00 at 0.8 with 1. Textual-similarity precision was 0.58 at 0.5 with 2 links, 0.63 at 0.6 with 2, 0.67 at 0.7 with 1, and 0.86 at 0.8 with 1. The reported pattern is consistent across both benchmarks: higher thresholds increase precision and reduce the number of shown links, while clone-based filtering appears substantially more precise than textual similarity alone.

The link pool was sparse in genuinely relevant provenance evidence. For CoderEval, the tool provided 263 links, but only 14, related to 11 different code snippets, were judged “highly relevant.” For CodeSearchNet, it provided 278 links, of which only 29, belonging to 26 snippets, were considered highly relevant. The paper interprets this as evidence that the initial search stage is noisy and that downstream similarity filtering is necessary.

Licensing results are also reported quantitatively. Among the 278 links from CodeSearchNet, CodeGenLink retrieved a license for 109 links, and only six of those were wrongly assigned. For CoderEval, annotators judged 120 of the 131 retrieved licenses as correct. The paper also states that the percentage of links for which a license could be associated is relatively low, about $\simeq 44\%$, largely because many retrieved links are blogs or tutorials where automatic license determination is difficult (Bifolco et al., 1 Oct 2025).

6. Scope, limitations, and place in the literature

CodeGenLink is presented as a support tool rather than a certifier. It cannot guarantee provenance, and it cannot currently determine “which one was generated by the AI,” meaning it does not establish temporal or causal origin between the web code and the generated snippet. Its effectiveness depends heavily on the quality of the search-capable LLM’s retrieved links, and to a lesser extent on the clone detector and licensing classifier. It may produce false negatives when search misses relevant sources or extraction fails, and false positives when similar but not original code is found. Scalability is not evaluated, and recall remains unknown. Most importantly, the authors do not claim that CodeGenLink proves copyright compliance or legal safety (Bifolco et al., 1 Oct 2025).

Within the broader research landscape, the tool sits at the intersection of code provenance, clone detection, code search, software licensing analysis, and legal concerns about LLM-generated code. The paper contrasts it with LiCoEval, stating that LiCoEval uses similarity analysis to assess whether generated code is part of an LLM’s training set, whereas CodeGenLink searches the web for matching snippets and identifies associated licenses, allowing it to work even when the LLM’s training data is inaccessible. The novelty claimed is therefore a practical IDE-integrated workflow that combines LLM-based web search, web-page code extraction, similarity filtering, and license inference to support provenance inspection for generated code.

A persistent misconception is that such a tool can certify authorship or legal status. The paper explicitly rejects that interpretation. CodeGenLink should instead be understood as a pragmatic provenance-assistance extension: it narrows down likely origins and surfaces possible license obligations, but it is not sufficient on its own to certify source attribution, legal compliance, or definitive copyright status.