VeriGrey: Greybox Security Validation

• VeriGrey is a grey-box security validation framework that uses runtime behavioral signals to guide test generation for LLM agents and web applications.
• It instruments tool invocations and database communications to reveal vulnerabilities that traditional black-box methods often overlook.
• Empirical evaluations demonstrate significant gains in injection success rates through coverage-guided search and mutational prompt injection.

Searching arXiv for the cited VeriGrey paper and closely related graybox security-testing work. {"^^^^^^^^2query2 OR ti:\2^^^^^^^^"VeriGrey: Greybox Agent Validation\"","max_results":5,"sort_by":"submittedDate","sort_order":"descending"} {"^^^^^^^^2query2 OR ti:\2query2^^^^} {"^^^^^^^^2query2 OR ti:\2query2^^^^} VeriGrey denotes a grey-box security-validation paradigm in which test generation is driven by runtime behavioral signals rather than by purely black-box input/output observation. In current literature, the term most directly identifies the framework presented in "VeriGrey: Greybox Agent Validation," which targets LLM agents that autonomously plan and invoke external tools (&&&^^^^2query2^^^^&&&). The term has also been used for a graybox web-security analyzer implemented as a ZAP plugin cooperating with a modified MariaDB, which upgrades a blackbox scanner with database traffic interception, input injection and tracking, and context-aware output analysis to detect stored and context-sensitive XSS (Steinhauser et al., 2020). Across both usages, the common principle is to expose internal-but-lightweight execution structure—tool-invocation sequences in agent systems, or database fetch and rendering contexts in web applications—to improve the discovery of vulnerabilities that conventional black-box methods systematically miss.

^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^. Scope, nomenclature, and unifying idea

The two principal uses of the name apply to distinct security domains but share a common methodological commitment to gray-box feedback. In the agent setting, VeriGrey instruments tool invocations and treats newly observed tools, transitions, and full sequences as coverage. In the web-security setting, the analyzer instruments the database protocol boundary and reasons about the path from stored values to browser contexts.

Usage	Domain	Feedback signal
VeriGrey: Greybox Agent Validation	LLM agents with external tools	Tool names, arguments, transitions, full sequences
Graybox XSS analyzer	Web applications with SQL back ends	Database fetch metadata and rendering context

This shared structure is significant because both target systems exhibit behaviors that are difficult to expose through surface-level prompt mutation or HTTP replay alone. In both cases, the gray-box signal is not a full white-box model of internals; it is a lightweight execution signature that improves exploration without requiring model internals in the agent case or web-application source instrumentation in the XSS case.

2. Agentic-AI validation problem and security model

In its primary usage, VeriGrey is a grey-box validation framework for LLM agents that autonomously plan and invoke external tools (&&&^^^^2query2^^^^&&&). The motivating observation is that LLM agents differ from single-shot LLMs in that they act over time, reason, and call tools such as shell, web, calendar, email, and MCP servers. This autonomy introduces several security risks: indirect prompt injection, malicious skill discovery, unsafe tool usage, and unexpected emergent behaviors.

Indirect prompt injection arises when adversarial instructions embedded in external resources such as web pages, emails, MCP tools, or skill marketplaces are ingested and acted on. Malicious skill discovery concerns marketplace skills whose instructions or side effects can cause autonomous installation and execution. Unsafe tool usage includes invocation of dangerous tools such as unrestricted shell or web_fetch for exfiltration. Unexpected emergent behaviors arise when complex tool/LLM interactions produce rare sequences that bypass guardrails.

The framework is motivated by the insufficiency of black-box red-teaming for agents. The stated reasons are that feedback is sparse, agent behavior is mediated by tool choices and their sequencing rather than surface text alone, and many dangerous behaviors are infrequent and require exploring rare tool sequences and transitions. VeriGrey therefore uses the actual sequence of tool invocations and their arguments as grey-box feedback, improving exploration and focusing mutation effort on inputs that lead the agent into previously unseen or risky behaviors.

A central design choice is mutational prompt injection with context linking. Rather than generating generic adversarial text, VeriGrey mutates seeds so that the injection task is framed as a necessary step of completing the agent functionality. This suggests that the framework is not merely searching for prompt fragments that override the policy; it is attempting to preserve plausibility with respect to the user task, thereby increasing the chance that an agent planner will route execution into dangerous tools.

3. Formalization and coverage-guided search

The agent is formalized as a tuple

PRESERVED_PLACEHOLDER_^^^^2query2^^^^

where PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^ is the state space, $T = \{\tau_1,\dots,\tau_m\}$ is the set of tools, $\pi$ is the LLM-driven planner/executor, and $E$ is the environment in which tools produce outputs (&&&^^^^2query2^^^^&&&). Given a user task $u$ and an injected prompt $p$, an execution yields a tool invocation sequence

$s = (z_1, z_2, \dots, z_n),$

with

$$z_i = (\mathrm{name}(\tau), \mathrm{args}(\tau), \mathrm{meta}).$$

VeriGrey maintains coverage over tool-level behaviors at three granularities: tool coverage, transition coverage, and sequence coverage. It defines

$T_s = \{\mathrm{name}(z_i) : z_i \in s\},$

and

PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^

Let the previously seen sets be PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^, PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\22^^^^, and PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^3. The feedback indicators are

PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^4

PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^5

PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^6

The mutation energy is

PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^7

and seeds are considered interesting if

PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^8

The paper also gives a generic fitness form,

PRESERVED_PLACEHOLDER_^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^9

but uses integer increments per satisfied metric with $T = \{\tau_1,\dots,\tau_m\}$^^^^2query2^^^^. The vulnerability predicate is defined over injection tasks $T = \{\tau_1,\dots,\tau_m\}$^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^ and their oracles $T = \{\tau_1,\dots,\tau_m\}$2:

$T = \{\tau_1,\dots,\tau_m\}$3

The mutation operator is an LLM-guided operator

$T = \{\tau_1,\dots,\tau_m\}$4

subject to a linking constraint

$T = \{\tau_1,\dots,\tau_m\}$5

Operationally, the mutator is instructed to infer the user intent from $T = \{\tau_1,\dots,\tau_m\}$6, craft a scenario where performing $T = \{\tau_1,\dots,\tau_m\}$7 is presented as a prerequisite or diagnostic step needed to complete $T = \{\tau_1,\dots,\tau_m\}$8, and incorporate tool hints gleaned from $T = \{\tau_1,\dots,\tau_m\}$9, such as using read_file before web_fetch. The search loop is evolutionary and coverage-guided: instrument tool calls, construct initial prompts, execute with prompt injection, update coverage and vulnerability databases, mutate with context linking, and select seeds that yielded new coverage. Convergence is discussed informally rather than as a proof; the paper states that grey-box guidance accelerates discovery of rare sequences and that removing feedback substantially reduces performance, especially on hard tasks requiring longer sequences.

4. Evaluation, ablations, defenses, and case studies

On the AgentDojo benchmark, the evaluation uses 97 user tasks, an average of 8.7 injection tasks per environment, a ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2query2^^^^-execution budget per testing campaign, and two methods across three LLMs, yielding 582 campaigns (&&&^^^^2query2^^^^&&&). The principal metric is Injection Task Success Rate (ITSR), defined as the number of injection tasks completed at least once divided by the total injection tasks.

Model	Baseline ITSR	VeriGrey ITSR
GPT-4.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^	37.7%	7^^^^2query2^^^^.7%
Gemini-2.5-Flash	36.8%	47.4%
Qwen-3 235B	67.6%	8^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.7%

For GPT-4.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^, the paper reports a gain of +33.^^^^2query2^^^^ percentage points. By suite, GPT-4.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^ improves from ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^6.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^% to 48.2% on Workspace, from 4^^^^2query2^^^^.7% to 87.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^% on Travel, from 34.7% to 68.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^% on Banking, and from 78.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^% to ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2query2^^^^% on Slack. By difficulty, the same backend improves from 55.2% to 88.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^% on Easy, from 7.6% to 3^^^^2query2^^^^.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^% on Medium, and from ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.3% to 43.3% on Hard.

The ablation study isolates the two core mechanisms. Full VeriGrey reaches 7^^^^2query2^^^^.7% ITSR. Removing feedback yields 59.6%, with the most pronounced drop on hard tasks, where performance falls from 43.3% to 2^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.6%. Removing context bridging yields 44.^^^^2query2^^^^%, with large drops across all difficulties, including Easy from 88.^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^% to 6^^^^2query2^^^^.2%. The empirical implication is direct: both tool-sequence feedback and context-bridging mutation are critical to the reported efficacy.

The defense study evaluates ITSR alongside User Task Success Rate (UTSR) on a 3^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^-user-task subset. Without defense, UTSR is 85.^^^^2query2^^^^ and VeriGrey ITSR is 69.7 versus 4^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.2 for the baseline. Under prompt sandwiching, UTSR is 86.5 and VeriGrey ITSR is 65.2 versus ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^7.4. Under data delimiter defenses, UTSR is 8^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.^^^^2query2^^^^ and VeriGrey ITSR is 67.6 versus 36.4. Prompt injection detection with BERT reduces UTSR to 5^^^^2query2^^^^.4 and lowers VeriGrey ITSR to 2^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.6 versus 5.2, which the paper characterizes as significant usability degradation with high false positives. Tool filtering with whitelisted tools yields UTSR 8^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.7 and VeriGrey ITSR ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^7.4 versus ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\22^^^^.8, and is identified as the most effective overall defense among those studied with modest usability impact.

The real-world case studies extend the benchmark findings. On Gemini CLI, using ten injection tasks in a single-use Docker container and a user task adapted from SWE-bench (django-^^^^2id:([2603.17639](/papers/2603.17639)) OR ti:\2^^^^45^^^^2query2query2^^^^) with "run security scan" added to trigger an MCP tool, VeriGrey reaches 9^^^^2query2^^^^% ITSR (9/^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^) versus 6^^^^2query2^^^^% (6/^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^) for the baseline. It succeeds on hard tasks involving sending bash history, extracting SSH keys, and adding a malicious alias to ~/.bashrc. On Task 6, a cron-based payload, neither method achieves full success, although VeriGrey induces partial malicious behavior by downloading and running code without achieving persistent cron installation. Under a tool-filter defense on Gemini CLI, VeriGrey falls to 5^^^^2query2^^^^% ITSR and the baseline to 3^^^^2query2^^^^%, while bug-patch generation remains acceptable.

On OpenClaw, the study examines supply-chain skill vulnerabilities using ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^ selected malicious or suspect skills from a larger set flagged by KOI, Snyk, and Cisco. With a 2^^^^2query2^^^^-run budget per skill, original skills succeed at ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^/^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^ across all three backends, while VeriGrey-mutated skills reach ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^/^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^ on Kimi-K2.5, 9/^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^ on Opus-4.6, and 8/^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^ on GPT-5.2. The derived tactics include framing a malicious install as a "natural step" in SKILL.md, emphasizing autonomy with instructions such as "No manual intervention needed," and fabricating realistic usage examples that make malicious packages appear integral to the skill’s function.

5. Threat model, limitations, and role in agent assurance

The agent-validation framework addresses single-session attacks via indirect prompt injection from external resources, tool misuse leading to data exfiltration or unsafe code execution, privacy leakage, and supply-chain skill attacks based on malicious SKILL.md instructions or scripts (&&&^^^^2query2^^^^&&&). It is explicitly out of scope for persistent multi-session memory poisoning and long-term agent state attacks, although related work is noted. It also does not provide formal guarantees or proofs; VeriGrey is presented as a dynamic validation tool.

The operational assumptions are concrete. Agent frameworks must expose or be instrumentable to wrap tool invocations through decorators or hooks. The verifier must be able to inject prompts via known endpoints such as tool outputs or environment resources. Oracles must exist to judge injection-task success, logging must be permitted, and the agent must have permission to call dangerous tools unless tool filters block them.

The limitations are similarly explicit. Efficacy depends on LLM backend characteristics and varies by model, although the reported results remain consistently better than baseline. Not all vulnerability classes are addressed, including complex multi-agent collusion and covert channels. Developer effort is required to instrument tool-call sites, though the paper describes this effort as typically minimal in common frameworks and MCP environments. Overhead and time-to-discovery are not quantified, and cost depends on agent tool frequency and LLM usage.

Within an assurance workflow, VeriGrey is positioned as a recurring validator. Recommended integration includes CI/CD execution against agent updates and tool or skill changes, maintenance of a corpus of discovered adversarial prompts and sequences, prioritization of tests that extend tool and transition coverage, collection of reproducible evidence in the form of prompts, sequences, and tool arguments, and iterative derivation of defense updates such as tool filters and policy rules. A common misconception is that grey-box validation is equivalent to white-box verification; the paper instead places VeriGrey between black-box red-teaming and white-box analysis, using runtime feedback to capture emergent behavior without requiring model internals or heavy instrumentation.

6. Database–scanner cooperative analysis for stored and context-sensitive XSS

A second usage of the term denotes a graybox XSS analyzer whose prototype is "a ZAP plugin cooperating with a modified MariaDB"; the original paper itself describes the system as "our graybox XSS analyzer" rather than assigning a product name (Steinhauser et al., 2020). Its objective is to close the gap left by conventional blackbox scanners, which systematically test only reflected, context-insensitive XSS and often miss reflected context-sensitive, stored context-insensitive, and stored context-sensitive flaws.

The motivating distinctions are reflected versus stored XSS and context-sensitive versus context-insensitive rendering. Stored XSS involves longer and more complex dataflow because attacker input is saved, typically in a database, and later rendered in some other response. Context-sensitive XSS arises because sanitization must match the browser context of rendering: HTML text, JavaScript strings, CSS, URIs, and HTML attributes each require different handling. The paper’s exemplary mismatch uses htmlentities before placing a value inside a single-quoted JavaScript string embedded in a double-quoted HTML attribute; after browser decoding, quotes reappear inside the JavaScript string and break out of the intended context.

The analyzer’s architecture is centered on database traffic interception at the DB–webapp boundary. In request recording mode, the database works normally but reports fetched strings as table, column, and value triples. In response injection mode, it replaces selected fetched string values with scanner-provided payloads. The interception point is the MariaDB COM_QUERY path where string types such as MYSQL_TYPE_VARCHAR, VAR_STRING, and STRING are inserted into ResultSet packets. From the application’s perspective, this is nearly blackbox, because the web application itself is not instrumented.

Payload tracking is designed to survive transformations. The canonical payload template is abcdef<gh"ij'kl&mn:op\qr/stuv, with randomized alphanumeric segments around context-switching characters. Regex identification converts letters into case-insensitive alternations and replaces special characters with .{^^^^2query2^^^^,2^^^^2query2^^^^}, allowing matches after HTML entity encoding, URL encoding, JSON escaping, JavaScript string encoding, CSS value encoding, and related transformations. After a mutated response is obtained, the analyzer scans the HTTP body for regex matches, replaces them with unique lowercase placeholders, and recursively parses the output to determine browser contexts without allowing automatic decoders to reinterpret the payload during parsing.

The context model is formalized as a syntax-node sequence

$\pi$^^^^2query2^^^^

where $\pi$^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^ is an HTML node and later nodes represent embedded languages such as URI, JavaScript, or CSS. For each node type $\pi$2, the system defines a decoder $\pi$3 and an escape predicate $\pi$4. Examples include HTML double-quoted attributes, HTML single-quoted attributes, HTML text, URI positions, JavaScript single-quoted and double-quoted strings, and CSS strings. Let $\pi$5 be the sanitizer and $\pi$6 the encoder applied by the application. With

$\pi$7

and

$\pi$8

a vulnerability exists if any escape predicate holds along the decoding chain:

$\pi$9

A stronger execution-oriented condition is

$E$^^^^2query2^^^^

where

$E$^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^

The parser stack emulates browser behavior through Jsoup for HTML, Rhino for JavaScript, SteadyState CSS for CSS, and a custom URI lexer, with extensions for quote and URI distinctions. A pre-check optimization avoids unnecessary replays by confirming that the first 2^^^^2query2^^^^ characters of the original value appear in the output before injection. Injection granularity can be per fetch, per table and column, per table, or global "inject everything," trading recall against the number of HTTP replays.

7. Evaluation and positioning of the XSS analyzer

The XSS analyzer was evaluated on eight mature and technologically diverse web applications: Joomla, OrchardCMS, SuiteCRM, Fat Free CRM, OpenEMR, Jeesite, PrestaShop, and Mezzanine (Steinhauser et al., 2020). Across these applications, it recorded 2,762 correct sanitizations and 2^^^^2query2^^^^7 incorrect ones, and the paper states that it never misclassified a correct sanitization as incorrect.

Among the 2^^^^2query2^^^^7 incorrect sanitizations, 7^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^ cases (34%) were exploitable via the web UI to arbitrary JavaScript execution, ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^5 cases (7%) were exploitable to arbitrary JavaScript only with direct DB writes, ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2query2^^^^5 cases (5^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^%) did not permit arbitrary JavaScript execution, and ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^6 cases (8%) were flagged as possibly exploitable where a manual exploit was not found. The vulnerability pattern analysis reports that stored context-sensitive XSS was present in all eight applications and was exclusively present in Joomla, Fat Free CRM, and Mezzanine.

Performance is slower than baseline OWASP ZAP XSS plugins because the analyzer is single-threaded and relies on DB-global interception. The reported analysis time is typically ^^^^2id:(Zhang et al., 18 Mar 2026) OR ti:\2^^^^.6× to 7× baseline depending on injection granularity. The best time/recall trade-off was achieved by grouping all fetches from the same table and column, which produced nearly the same discoveries as the finest granularity with lower overhead.

Comparative evaluation against Acunetix, Burp Suite Professional, and OWASP ZAP’s standard XSS plugins shows that those scanners primarily detected reflected context-insensitive bugs and a subset of reflected context-sensitive issues. None found the stored context-sensitive flaws in Joomla, OrchardCMS, Fat Free CRM, and Mezzanine that the graybox analyzer reported. The broader significance is methodological: by moving the database out of the blackbox and coordinating injection at the DB–webapp boundary, the system combines near-blackbox portability with context-sensitive checks that resemble whitebox reasoning. A plausible implication is that VeriGrey, across both of its principal usages, marks a recurring pattern in security validation: lightweight runtime structure can be sufficient to expose vulnerability classes that remain opaque to purely external probing.