Vul-RAG: Secure Vulnerability Detection
- Vul-RAG is a framework that augments software vulnerability detection by retrieving multi-dimensional security knowledge from historical CVEs, including functional semantics, causes, and fixes.
- It employs a three-phase pipeline—offline knowledge-base construction, BM25-based retrieval with Reciprocal Rank Fusion, and knowledge-guided detection—to distinguish vulnerable from patched code.
- The approach not only improves detection accuracy but also exposes new risks in RAG systems, prompting further research into adversarial resilience and privacy safeguards.
Searching arXiv for papers on "Vul-RAG" and closely related RAG security/vulnerability research. Vul-RAG denotes a cluster of retrieval-augmented generation methods and analyses at the intersection of software security and RAG research. In its canonical usage, the term names a knowledge-level RAG framework for source-code vulnerability detection that retrieves abstracted vulnerability knowledge—functional semantics, root causes, and fixing strategies—from historical CVEs and patches, rather than retrieving raw code alone (Du et al., 2024). In adjacent usage, the label also appears in vulnerability-preserving data augmentation, automated vulnerability repair, and in a separate line of work that uses “Vul-RAG” to describe the security and privacy vulnerabilities of RAG systems themselves (Arzanipour et al., 24 Sep 2025).
1. Terminological scope and lineage
The term is used in more than one technical sense across the literature. One line of work centers on software vulnerability analysis and remediation; another treats “Vul-RAG” as a structured view of the attack surface of RAG pipelines. The result is a polysemous term whose meaning depends on whether the subject is software-security application or RAG-system security.
| Work | Primary task | Defining characteristics |
|---|---|---|
| "Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG" (Du et al., 2024) | Vulnerability detection | Retrieves functional semantics, vulnerability causes, and fixing solutions |
| "VulScribeR: Exploring RAG-based Vulnerability Augmentation with LLMs" (Daneshvar et al., 2024) | Data augmentation for DLVD | Mutation, Injection, and Extension strategies |
| "RAVEN: Agentic RAG for Automated Vulnerability Repair" (Gadey et al., 21 Jun 2026) | Automated repair | Multi-faceted retrieval, Curator Agent, Generator–Reviewer loop |
| "RAG Security and Privacy: Formalizing the Threat Model and Attack Surface" (Arzanipour et al., 24 Sep 2025) | RAG security/privacy formalization | Corpus, retriever, generator, and orchestration threat model |
Within software-security research, the core motivation is that vulnerability detection, augmentation, and repair all benefit from contextualized security knowledge that is not reliably captured by standalone LLMs or conventional deep learning-based vulnerability detectors. Within RAG-security research, the same term marks the fact that retrieval creates new privacy, integrity, and compositional risks that do not reduce to ordinary LLM prompt attacks.
2. Knowledge-level Vul-RAG for vulnerability detection
The defining system in this literature reframes vulnerability detection as knowledge-guided reasoning. Its central claim is that vanilla LLMs and other learning-based detectors struggle to distinguish vulnerable code from near-identical patched code because they overfit superficial lexical cues and fail to recover the root cause of the vulnerability (Du et al., 2024). To make this concrete, the work introduces PairVul, a benchmark of 4,314 function-level vulnerable–patched pairs across 2,073 CVEs, with a train split of 1,462 pairs from 896 CVEs and a test split of 592 pairs from 373 CVEs. On this dataset, the reported overall accuracy of LineVul, DeepDFA, LLMAO, and Cppcheck is approximately $0.50$–$0.54$, while pairwise accuracy—requiring both the vulnerable and patched function in a pair to be labeled correctly—falls to $0.01$–$0.10$.
Vul-RAG addresses that failure by retrieving multi-dimensional vulnerability knowledge distilled from Linux kernel CVEs and their fixes. Each knowledge item contains three dimensions. The first is functional semantics, represented by an abstract purpose and a detailed behavior description. The second is vulnerability causes, represented by an abstract vulnerability description, a detailed vulnerability description, and a triggering action. The third is fixing solutions, represented by a summary of the mitigation strategy. The system stores these as textual abstractions rather than graphs, ASTs, or DFGs.
The pipeline has three phases. In offline knowledge-base construction, the system extracts vulnerable code, patched code, patch diffs, and CVE descriptions, then uses LLM prompting to produce the functional-semantics, cause, and fix fields. In online retrieval, it derives a query from the target function’s code, abstract purpose, and detailed behavior; runs BM25 retrieval over the indexed fields; and fuses the results with Reciprocal Rank Fusion to obtain a final top-10 candidate set. In knowledge-augmented detection, the model iterates over those candidates and asks two Boolean questions for each item: whether the target code exhibits the retrieved vulnerability cause, and whether it already implements the retrieved fixing solution. If a cause is present and the fix is absent, the function is labeled vulnerable.
The retrieval layer uses BM25 with and , while reranking is purely RRF-based rather than cross-encoder-based. The decision rule is non-probabilistic: there is no calibration stage or confidence thresholding. This design is significant because it moves the detection problem away from snippet similarity and toward explicit reasoning over root causes and repair logic.
3. Empirical performance, reproducibility, and known limits
On PairVul, the original Vul-RAG system reports overall and , outperforming LLMAO at and , while LineVul, DeepDFA, and Cppcheck remain near $0.54$0 and $0.54$1–$0.54$2 (Du et al., 2024). The same work reports balanced precision and recall at approximately $0.54$3, and a user study in which manual detection accuracy improves from $0.54$4 without the retrieved knowledge to $0.54$5 with it. On a curated cross-CVE set of 60 functions across 30 CVEs where relevant knowledge exists, recall is $0.54$6 and precision is $0.54$7.
A later reproducibility study reimplemented Vul-RAG locally with open-weight models and kept the original knowledge base fixed to isolate model effects (Kaniewski et al., 3 Jun 2026). The replicated results are close but not identical: Qwen2.5-Coder-32B-Instruct reaches pairwise accuracy $0.54$8, balanced recall $0.54$9, and balanced precision $0.01$0, while DeepSeek-Coder-V2-Instruct falls to pairwise accuracy $0.01$1 on 565 of 586 pairs because of CUDA OOM failures on long inputs. Newer models do not break the apparent ceiling. Qwen3-Coder-30B-A3B-Instruct reaches $0.01$2 pairwise accuracy; DeepSeek-R1-Distill-Qwen-32B and QwQ-32B both reach $0.01$3; and DeepSeek-R1-Distill-Llama-8B also reaches $0.01$4. The study therefore reports a plateau at approximately $0.01$5 pairwise accuracy and argues that model scale alone is not the limiting factor.
The stated reasons for this ceiling are structural rather than merely parametric. The reproduced analysis identifies knowledge-base quality, semantically mismatched retrieval, mismatched fixing solutions, extremely long functions, and brittle prompt-format compliance as principal bottlenecks. It also notes that reasoning models help modestly but produce at least $0.01$6 more tokens on median. Another important clarification concerns a recurring secondary claim: the detailed synthesis explicitly states that the paper text does not report discovery of previously unknown Linux kernel bugs or assignment of new CVEs, so those claims are not substantiated there (Du et al., 2024).
4. Extensions: augmentation, repair, and comparative RAG pipelines
Vul-RAG principles have been extended beyond binary detection. "VulScribeR: Exploring RAG-based Vulnerability Augmentation with LLMs" formulates RAG as a vulnerability-preserving data-augmentation pipeline for deep learning-based vulnerability detection (Daneshvar et al., 2024). Its corpus combines Devign clean functions with a BigVul training subset of 6,610 vulnerable functions carrying vulnerable-line metadata. Vulnerable functions are embedded with CodeBERT $0.01$7, clustered with KMeans into $0.01$8 clusters, indexed with Lucene BM25 per cluster, and retrieved at function granularity. The system defines three augmentation strategies: Mutation, Injection, and Extension. Injection treats a clean function as a query and retrieves top-1 similar vulnerable functions per cluster; Extension does the reverse; Mutation uses prompt-level transformation without retrieval. With 5K generated vulnerable samples, Injection improves average F1 by $0.01$9 over NoAug, $0.10$0 over VulGen, $0.10$1 over VGX, and $0.10$2 over ROS. With 15K samples, the corresponding gains are $0.10$3, $0.10$4, $0.10$5, and $0.10$6. The reported cost is as low as US\$ 1.88 per 1K generated samples.
"RAVEN: Agentic RAG for Automated Vulnerability Repair" applies the same general retrieval logic to patch synthesis rather than detection (Gadey et al., 21 Jun 2026). It integrates a Semantic Retriever, a Selector Agent, a Context Retriever with a Curator Agent, a Patch Generator, a Patch Reviewer, and a Patch Validator in a controlled iterative loop. Historical fixes are retrieved from CVEFixes using multi-faceted indexing, while the Curator Agent traverses the repository to recover caller/callee relationships, helper functions, policy checks, and other cross-file dependencies that are not visible in the local vulnerable snippet. On 160 real-world CVEs across Java and C, the framework reports an overall repair success rate of $0.10$7, including $0.10$8 over 83 Java vulnerabilities, $0.10$9 over 29 C vulnerabilities, 0 on 23 out-of-distribution Vul4J cases, and 1 on 16 unseen 2024 Zero-Day cases.
A separate comparative study of LLM-based code vulnerability detection, although not using the canonical Vul-RAG name for a specific framework, reports that a knowledge-grounded RAG pipeline built from the MITRE CWE database outperforms both supervised fine-tuning and a Dual-Agent audit architecture (Saju et al., 1 Jan 2026). In that setup, CWE documents are parsed with PyMuPDF, chunked at 2 tokens with 3 token overlap, embedded, stored in Chroma, and retrieved with top-4. Using LLaMA 3.2-3B as the base model, the study reports overall accuracy 5 and F1 6 for RAG, compared with 7 for SFT, 8 for Dual-Agent, and 9 for the base model.
5. Vul-RAG as a formal security and privacy problem in RAG systems
A distinct research line uses “Vul-RAG” to mean the structured vulnerability landscape of RAG systems themselves. "RAG Security and Privacy: Formalizing the Threat Model and Attack Surface" gives a component-level model of a retrieval-RAG pipeline with corpus 0, encoder 1, index 2, retriever 3, orchestrator 4, and generator 5, plus optional rerankers, filters, moderation, and verification modules (Arzanipour et al., 24 Sep 2025). The pipeline is formalized as query submission, encoding and indexing, retrieval 6, context assembly 7, generation 8, and post-processing. Trust boundaries are placed around user inputs, embeddings and vector stores, generation outputs, and orchestration across services.
The same work formalizes adversaries by both access level and operational role. The access taxonomy distinguishes black-box adversaries observing only 9, gray-box adversaries with limited internal visibility, and white-box adversaries with privileged access to retriever weights, stored embeddings, generator parameters, or orchestration code. The role taxonomy includes external users, data providers, index maintainers, model hosts or administrators, passive network observers, and colluding combinations of those roles.
Its most important contribution is a formal decomposition of attack surfaces by pipeline stage. Query submission admits prompt injection and deanonymization. Encoding and embedding admit semantic leakage and inversion attacks. Indexing and storage admit cross-tenant leakage, linkability, unauthorized read/write, and tampering. Retrieval admits adversarial reranking, index poisoning, and trigger-based hijacking. Context assembly admits contamination by malicious instructions in retrieved documents. Generation admits verbatim or near-verbatim leakage of retrieved content and in-context jailbreaks. Post-processing admits inadequate redaction and citation of sensitive excerpts.
The work also gives formal definitions for document-level membership inference, passage-level membership inference, data poisoning, and prompt/content injection. For document-level membership inference, the adversary’s advantage is
0
For retriever-level differential privacy, neighboring datasets 1 and 2 satisfy
3
and by post-processing the generator inherits the same privacy bound, implying that membership-inference advantage can be upper-bounded by 4, up to 5-accounting. This is one of the few explicit end-to-end privacy claims in the RAG literature. The same framework identifies prioritized mitigations: corpus hygiene, provenance verification, adversarial training of encoders, diversity-aware or group retrieval, instruction isolation, score perturbation for DP retrieval, per-tenant isolation, secure enclaves, strict IAM, and post-generation redaction and verification.
6. Attack literature, misconceptions, and open problems
The formal threat model is matched by a rapidly growing attack literature. "BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of LLMs" shows that poisoning only 10 adversarial passages, approximately 6 of the corpus, can produce 7, 8, and 9 for Contriever on NQ, while increasing GPT-4’s refusal ratio from 0 to 1 and increasing the negative-response rate on Donald Trump queries from 2 to 3 (Xue et al., 2024). The attack is semantic-group-triggered rather than fixed-string-triggered, and it couples retrieval backdoors with generation-phase denial-of-service and stance steering.
"Spa-VLM: Stealthy Poisoning Attacks on RAG-based VLM" extends the problem to multi-modal RAG (Yu et al., 28 May 2025). In its setting, single-modal poisoning attacks have a 4 failure rate because text-only poison cannot pass the visual gate and image-only poison lacks effective text-side steering. Spa-VLM instead injects five malicious image–text entries and reports 5, precision 6 on E-VQA and 7, precision 8 on InfoSeek with a reranker present. This indicates that cross-modal fusion introduces a distinct gate-and-rerank attack surface rather than eliminating poisoning risk.
"EmoRAG: Evaluating RAG Robustness to Symbolic Perturbations" identifies a different failure mode: near-imperceptible emoticon tokens (Zhou et al., 1 Dec 2025). With 9 injected documents among millions, a single emoticon inserted at the beginning of the query can drive attack success rates to 0–1 and F1 to 2–3 across NQ, MS-MARCO, and code retrieval settings. The work attributes this to rare-token effects, positional sensitivity, and high-dimensional amplification, and reports that larger embedding models around 7B parameters are often more vulnerable than approximately 110M-scale retrievers.
"RAG-Pull: Imperceptible Attacks on RAG Systems for Code Generation" demonstrates a black-box attack based on invisible Unicode code points such as U+200B, U+200C, U+200D, U+FEFF, U+00AD, U+2060, U+200E, U+200F, and U+00A0 (Stambolic et al., 13 Oct 2025). Using a vetted set of 382 non-rendering characters, the method perturbs queries, target code snippets, or both. Combined query-and-target perturbations achieve 4 retrieval success on Python Alpaca and Java VFD at 5, and the paper reports up to 6 end-to-end success. The attack is notable because it requires no gradients, no tokenizer internals, and no parameter access.
"The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systems" shows that the transparency of citation-grounded RAG can itself be an attack vector (Choi et al., 28 Feb 2025). Because systems disclose the hosts they trust, attackers can crawl those hosts, synthesize poisoned documents with preserved lexical hooks, and republish them on the same platforms. In online experiments against ChatGPT and Perplexity, poisoned documents are retrieved in approximately 7 of answers for fictional individuals, driving accuracy from 8 to 9 for ChatGPT and from 0 to 1 for Perplexity. The same work reports smaller but still measurable drops for rare-species questions.
Taken together, these studies correct a common misconception: retrieval is not merely an auxiliary grounding layer that passively improves factuality. It is a high-leverage control surface whose failures can dominate end-to-end behavior even when the generator is safety-aligned or instruction-following. The current literature therefore converges on a shared open-problem set: compositional guarantees across retrieval and generation, encrypted or oblivious retrieval, privacy-preserving indexing, robust retrievers against adaptive attackers, standardized certification and audits, and community-agreed benchmarks for membership inference, poisoning, prompt injection, and leakage (Arzanipour et al., 24 Sep 2025). In that sense, Vul-RAG now names both a family of security-oriented RAG applications and a research agenda about the fragility, controllability, and formal assurance of retrieval-augmented systems themselves.