VulnGuard Prompt: Securing LLMs & Code

• VulnGuard Prompt is a security paradigm using shallow ensemble classifiers and calibrated thresholds to robustly defend against prompt injection and jailbreak attacks.
• It structures LLM inputs with context extraction, explicit reasoning scaffolds, and API call sequencing to improve software vulnerability detection accuracy.
• The design emphasizes contamination-aware benchmarking and deployment-oriented evaluation to balance efficiency with precise, adversarial resilience.

VulnGuard Prompt is a label used in several recent arXiv papers for prompt-centric or guardrail-centric mechanisms that address two closely related problems: defending LLM systems against prompt injection and jailbreak attacks, and structuring LLM inputs for software vulnerability detection and security analysis. In the prompt-injection setting, it denotes a lightweight guardrail system built from shallow classifiers, threshold calibration, and contamination-aware benchmarking; in the software-security setting, it denotes structured prompts that emphasize context extraction, explicit reasoning scaffolds, and verification-oriented output formats (Neves et al., 4 Jun 2026, Zibaeirad et al., 22 Mar 2025, Li et al., 14 Nov 2025, Zhang et al., 2023). Across these usages, the recurring design elements are discriminative detection, prompt structuring, context management, calibration, and deployment-oriented evaluation.

1. Terminological scope and research setting

In "GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection" (Neves et al., 4 Jun 2026), VulnGuard Prompt is described as a guardrail system against prompt-injection and jailbreak attacks built around three core pillars: a shallow-model ensemble architecture, rigorous threshold calibration, and careful benchmark construction to avoid contamination. The same label is also used in several software-security papers as a prompt-engineering guide or template for vulnerability detection, especially in C/C++ code analysis and context-aware repository analysis (Zibaeirad et al., 22 Mar 2025, Li et al., 14 Nov 2025, Zhang et al., 2023).

This dual usage is central to the term. In one branch, VulnGuard Prompt is a classifier-facing security layer that consumes prompts or tool outputs and returns a binary decision about maliciousness or safety (Neves et al., 4 Jun 2026). In the other branch, it is a prompt template or prompting workflow that shapes the reasoning process of an LLM performing vulnerability detection, often by adding API-call sequences, data-flow information, repository-level context, or verification steps (Zhang et al., 2023, Zibaeirad et al., 22 Mar 2025, Li et al., 14 Nov 2025).

A plausible implication is that the phrase functions less as a single standardized artifact than as a family of security-oriented prompting and guardrail designs. The shared objective is not merely classification accuracy, but robust behavior under adversarial or context-rich conditions.

2. Guardrail formulation for prompt injection and jailbreak detection

The most explicit guardrail formulation appears in GuardNet (Neves et al., 4 Jun 2026). There, VulnGuard Prompt employs an ensemble of three independent BiLSTM-based classifiers, each containing roughly 15 million parameters, for a total of about 47 million parameters. Each head is purely discriminative and has no language-modeling or decoding layers. The input pipeline is:

• Input $x$ to a shared tokenizer, specifically bert-base-multilingual-cased
• A two-layer BiLSTM per head, $z_i = \mathrm{BiLSTM}_i(x)$
• A final dense layer with sigmoid activation, $p_i = \sigma(W_o \cdot z_i + b_o)$

The three probabilities are combined by a simple arithmetic mean:

$$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$

A global binary decision is then made by thresholding:

$$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$

The reported operating point is $\tau = 0.65$ on a held-out validation set, chosen to balance precision and recall across attack and benign classes (Neves et al., 4 Jun 2026).

The paper states the underlying hypothesis as follows: robustness in adversarial scenarios depends more on the diversity of example coverage and threshold calibration than on model scale. The reported results support only part of that claim. GuardNet achieves competitive performance compared with lightweight detectors and high efficiency at low latency, but larger LLMs such as Mistral-7B and Llama-3.1-8B still achieve superior performance in terms of F1 score and AUROC on the blind JBB-Behaviors benchmark (Neves et al., 4 Jun 2026).

This places VulnGuard Prompt in a distinctive part of the design space: it is not framed as a generative or agentic defense, but as an inexpensive, embedded, discriminative guardrail suitable for production environments with cost and infrastructure constraints (Neves et al., 4 Jun 2026).

3. Threshold calibration, contamination control, and benchmark design

A defining methodological feature of the GuardNet formulation is explicit threshold calibration. Rather than selecting $\tau$ heuristically, the paper formalizes threshold selection as optimization over a chosen metric, typically the F1-score (Neves et al., 4 Jun 2026). With

$$\mathrm{Precision}(\tau)=\frac{TP(\tau)}{TP(\tau)+FP(\tau)}, \qquad \mathrm{Recall}(\tau)=\frac{TP(\tau)}{TP(\tau)+FN(\tau)}$$

the threshold objective is

$$F1(\tau)=\frac{2\cdot \mathrm{Precision}(\tau)\cdot \mathrm{Recall}(\tau)}{\mathrm{Precision}(\tau)+\mathrm{Recall}(\tau)}, \qquad \tau^* = \arg\max_{\tau\in[0,1]} F1(\tau)$$

The paper also gives an alternative based on Youden’s index,

$$J(\tau)=TPR(\tau)-FPR(\tau), \qquad \tau^*=\arg\max J(\tau)$$

and monitors AUROC to decouple performance from any particular threshold choice (Neves et al., 4 Jun 2026).

The benchmark design is equally emphasized. VulnGuard’s training and evaluation sets are described as being carefully constructed to avoid optimistic leakage. Two datasets are central:

Benchmark	Size	Role
awall-test	$z_i = \mathrm{BiLSTM}_i(x)$0	development and threshold calibration
JBB-Behaviors	$z_i = \mathrm{BiLSTM}_i(x)$1	blind out-of-distribution test

The proprietary benchmark awall-test is balanced across benign, prompt-injection, and jailbreak examples and is generated synthetically by an LLM under strict licensing and with controlled coverage of known attack taxonomies. The paper explicitly states that partial, declared leakage may occur because public attack patterns can be reproduced by the generator, and therefore the set is used only for development and threshold calibration (Neves et al., 4 Jun 2026). By contrast, JBB-Behaviors is described as a blind benchmark of real-world, previously unseen examples with no overlap with training data (Neves et al., 4 Jun 2026).

To mitigate contamination and preserve adversarial diversity during training, the reported pipeline applies a cap of 8 000 samples per source, minimum quotas for key attack axes such as persona hijacks, JSON exfiltration, and multilingual obfuscation, inclusion of hard negatives, and a WeightedRandomSampler with weights $z_i = \mathrm{BiLSTM}_i(x)$2 across sources (Neves et al., 4 Jun 2026).

This emphasis on contamination-aware evaluation aligns with PromptShield, which likewise treats realistic deployment traffic, out-of-distribution splits, and low-FPR calibration as first-order methodological concerns (Jacob et al., 25 Jan 2025). PromptShield reports that practical deployable detectors must operate at $z_i = \mathrm{BiLSTM}_i(x)$3, and evaluates detectors at target false positive rates of $z_i = \mathrm{BiLSTM}_i(x)$4, $z_i = \mathrm{BiLSTM}_i(x)$5, $z_i = \mathrm{BiLSTM}_i(x)$6, and $z_i = \mathrm{BiLSTM}_i(x)$7 (Jacob et al., 25 Jan 2025). The overlap in emphasis suggests that VulnGuard Prompt belongs to a broader line of work in which benchmark curation and threshold selection are treated as part of the defense itself rather than as post hoc evaluation choices.

4. Prompt formulations for software vulnerability detection

In the software-security literature, VulnGuard Prompt denotes structured prompting strategies for vulnerability analysis rather than a standalone classifier. The clearest early formulation appears in "Prompt-Enhanced Software Vulnerability Detection Using ChatGPT" (Zhang et al., 2023). That work organizes prompt design into improved basic prompts, integration of auxiliary information, and multi-round dialogue.

The basic role-enhanced template is:

$\tau = 0.65$5

Auxiliary information is then injected through API-call sequences and data-flow graph descriptions. The best-performing order reported in that paper is API before code and DFG after code (Zhang et al., 2023). A consolidated template is:

$\tau = 0.65$6

The same paper also introduces a multi-round design in which the model first summarizes code intent and then performs the vulnerability judgment (Zhang et al., 2023).

VulnSage extends this line by defining four zero-shot prompt strategies—Baseline, Chain-of-Thought, Think, and Think & Verify—and by tying prompt structure to empirical ambiguity reduction (Zibaeirad et al., 22 Mar 2025). In the Think & Verify prompt, the model is instructed to produce an initial analysis, list findings, assign a confidence score, reanalyze if confidence is below $z_i = \mathrm{BiLSTM}_i(x)$8, perform verification, and then deliver a final assessment. The paper reports that Think & Verify reduces ambiguous responses from $z_i = \mathrm{BiLSTM}_i(x)$9 to $p_i = \sigma(W_o \cdot z_i + b_o)$0 while increasing accuracy (Zibaeirad et al., 22 Mar 2025).

VULPO pushes the prompt formulation toward context-aware repository reasoning (Li et al., 14 Nov 2025). Its example prompt template is explicitly named "You are VulnGuard, an expert vulnerability detector" and requires the model to:

1. Analyze code and decide whether it contains the target vulnerability, outputting HAS_VUL or NO_VUL
2. Indicate the root cause, exact location, and suggested fix
3. Format the output as: $\tau = 0.65$7

The same template also includes reward guidelines in the prompt, namely $p_i = \sigma(W_o \cdot z_i + b_o)$1 for correctness versus the CVE description, $p_i = \sigma(W_o \cdot z_i + b_o)$2 for accurate line localization, $p_i = \sigma(W_o \cdot z_i + b_o)$3 for semantic alignment, and $p_i = \sigma(W_o \cdot z_i + b_o)$4 as a format penalty (Li et al., 14 Nov 2025).

This progression shows a shift from direct binary questioning toward scaffolded reasoning, confidence assessment, verification, and constrained answer formats. The papers do not claim that one template is universally optimal across all security contexts. VulnSage explicitly states that no single approach universally excels across all security contexts (Zibaeirad et al., 22 Mar 2025).

5. Context extraction and context-aware reasoning

A major development in the VulnGuard Prompt lineage is the move from function-level prompting to context-aware analysis. VULPO introduces ContextVul, which augments function-level samples with a lightweight slice of repository-level context (Li et al., 14 Nov 2025). For a sample $p_i = \sigma(W_o \cdot z_i + b_o)$5 with repository $p_i = \sigma(W_o \cdot z_i + b_o)$6 and entry function $p_i = \sigma(W_o \cdot z_i + b_o)$7, the extracted context is defined as

$p_i = \sigma(W_o \cdot z_i + b_o)$8

where the call chain is typically traversed to depth $p_i = \sigma(W_o \cdot z_i + b_o)$9 (Li et al., 14 Nov 2025). The extracted material includes direct callees, imported headers, macros, global variables, typedefs, and function bodies relevant to the call chain.

VULPO reports that ablating context and using only the function causes F1 to drop by $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$0, and that $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$1 yields the best trade-off while $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$2 adds noise and slows RL (Li et al., 14 Nov 2025). The paper’s case study attributes a correct NO_VUL decision on a CWE-119 example to the presence of external check logic in helper.h, which the baseline misses (Li et al., 14 Nov 2025).

VulnSage independently argues for multi-granular analysis across function, file, and inter-function levels and recommends always including full context, including multi-function scraps and file paths (Zibaeirad et al., 22 Mar 2025). For large G3 samples, it recommends breaking the material into per-file segments and aggregating findings (Zibaeirad et al., 22 Mar 2025).

A related context-oriented development appears outside software vulnerability detection proper, in WebSentinel and ARGUS. WebSentinel uses a two-step design in which candidate segments are first extracted and then analyzed against the broader webpage context (Wang et al., 3 Feb 2026). ARGUS generalizes this into provenance-aware auditing for agent decisions by segmenting runtime observations into benign and anomalous spans, grounding action arguments to source spans, and checking user-derived invariants before execution (Weng et al., 5 May 2026). This suggests that context management in VulnGuard Prompt is not limited to source code repositories; it extends to agentic environments in which runtime observations can influence downstream actions.

6. Empirical results, deployment properties, and limitations

The reported empirical properties of VulnGuard Prompt vary by formulation. In the GuardNet setting, the key quantitative results are benchmark-specific. On awall-test, the paper reports $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$3 at $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$4 and $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$5. On JBB-Behaviors, it reports $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$6 at $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$7 and $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$8 (Neves et al., 4 Jun 2026). Latency on CPU is approximately $$p_{\mathrm{ensemble}}(x)=\frac{1}{3}\sum_{i=1}^{3} p_i(x)$$9 ms per example on an Intel i7-13700F, and the system is described as roughly $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$0 faster than typical 7–8B-parameter LLMs when used as classifiers (Neves et al., 4 Jun 2026).

In zero-shot vulnerability detection, the reported effects concern prompt strategy rather than classifier throughput. VulnSage reports the following vulnerability-detection accuracies: Baseline $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$1, CoT $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$2, Think $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$3, and Think & Verify $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$4 on vulnerable code, alongside a reduction in ambiguity from $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$5 for CoT to $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$6 for Think & Verify (Zibaeirad et al., 22 Mar 2025). The earlier ChatGPT-based study reports that chain-of-thought prompting can improve C/C++ performance, with $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$7 reaching F1 $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$8 and accuracy $$\hat y = 1 \quad \text{if} \quad p_{\mathrm{ensemble}}(x)\ge \tau,\qquad 0 \text{ otherwise}$$9 on that dataset (Zhang et al., 2023).

VULPO reports that prompt structure combined with on-policy optimization yields much larger gains in context-aware vulnerability detection: Qwen3-4B baseline F1 is approximately $\tau = 0.65$0, Qwen3-4B-SFT reaches approximately $\tau = 0.65$1, and Qwen3-4B-VULPO reaches approximately $\tau = 0.65$2, improving F1 by $\tau = 0.65$3 over Qwen3-4B and achieving performance similar to DeepSeek-V3.1 despite 150× fewer parameters (Li et al., 14 Nov 2025). The paper’s ablations attribute large performance drops to removing label scaling, sample scaling, or multi-dimensional reward shaping (Li et al., 14 Nov 2025).

Deployment guidance is correspondingly heterogeneous. GuardNet emphasizes embedding all models in the host process via PyTorch and HuggingFace Transformers, avoiding RPC overhead and extra microservice complexity (Neves et al., 4 Jun 2026). Recommended practices include lazy-loading weights, synchronous inference on the user’s request hot path when latency is critical, and logging raw probabilities and decisions for monitoring of FPR and FNR in production (Neves et al., 4 Jun 2026).

The software-vulnerability prompt papers emphasize prompt staging and pipeline integration instead. VulnSage recommends a three-stage workflow: Baseline for quick triage, Think for flagged cases, and Think & Verify for high-risk code (Zibaeirad et al., 22 Mar 2025). VULPO recommends explicit context up to a configurable depth $\tau = 0.65$4, structured output with separate reasoning and answer fields, and dynamic adjustment of the context window based on call-graph breadth (Li et al., 14 Nov 2025).

The limitations are also explicit. GuardNet identifies sensitivity to threshold tuning, a generalization gap between blind and partially leaked data, and incomplete taxonomy coverage for new attack vectors (Neves et al., 4 Jun 2026). VulnSage notes language-specific performance differences, prompt bias such as the “buggy” versus “correct” wording effect, partial code comprehension, and context-length constraints (Zhang et al., 2023, Zibaeirad et al., 22 Mar 2025). VULPO notes that broader context can add noise, and that reward design must address asymmetric difficulty and reward hacking (Li et al., 14 Nov 2025).

Taken together, these results define VulnGuard Prompt as a security-oriented prompt and guardrail paradigm rather than a single algorithm. In prompt-injection defense, it is characterized by lightweight discriminative ensembles, threshold optimization, and contamination-aware evaluation (Neves et al., 4 Jun 2026). In vulnerability detection, it is characterized by structured reasoning templates, auxiliary program context, verification steps, and increasingly repository-aware or provenance-aware analysis (Zhang et al., 2023, Zibaeirad et al., 22 Mar 2025, Li et al., 14 Nov 2025, Weng et al., 5 May 2026).