Agent Security Bench Overview

• Agent Security Bench is a suite of formal frameworks that rigorously defines an LLM agent’s components—such as system prompts, memory, and toolsets—to systematically evaluate threats like prompt injection and memory poisoning.
• It organizes attack and defense methodologies into comprehensive taxonomies, employing measures such as input delimiters, plan shuffling, and runtime output validation to mitigate a spectrum of security risks.
• Benchmarks and protocols are applied across diverse real-world scenarios using metrics like attack success rate (ASR) and Net Resilient Performance (NRP), highlighting the trade-offs between operational utility and agent vulnerability.

Agent Security Bench (ASB) encompasses a suite of formal frameworks, benchmarks, and evaluation protocols designed to systematically assess, compare, and improve the robustness of LLM-based agents against a broad spectrum of security threats. ASB and its successors have driven the emergence of reproducible, cross-domain, and multi-stage evaluation methodologies supporting both static and dynamic agent security analysis across realistic application settings.

1. Formalization of Security in LLM-Based Agents

Agent Security Bench formalizes the agent as a composite system comprising a backbone LLM, a concealed system prompt $p_{\mathrm{sys}}$, user queries $q$, accumulated observations $\mathcal{O}$, a toolset $\mathcal{T}$, and a memory database $\mathcal{D}$ (Zhang et al., 2024). The agent’s core policy iterates over plan generation and tool or memory invocations: $$\mathrm{Agent}\bigl(\mathrm{LLM}(p_{\mathrm{sys}}, q, \mathcal{O}, \mathcal{T}, \mathcal{E}_K(q\oplus\mathcal{T}, \mathcal{D}))\bigr) = a$$ The security threat model assumes adversaries may inject at multiple points: system/user prompts (Direct Prompt Injection, Plan-of-Thought backdoors), observation streams (Observation Prompt Injection), memory contexts (Memory Poisoning), and via registration of attacker-crafted tools. The main adversarial goal is to elicit unpermitted agent actions $a_m$ (such as data exfiltration, unsafe code execution, or privilege escalation) with high probability.

ASB provides explicit mathematical definitions and noninterference criteria for key classes of risk:

• Instruction-integrity: Ensuring prompt/observation contamination cannot redirect agent actions.
• Retrieval-confidentiality: Only authorized data should influence agent behavior.
• Capability-integrity: Tool use and output must comply with declared policy projections (Alpay et al., 25 May 2026).

2. Attack and Defense Methodologies

ASB organizes attacks and mitigations into an extensive taxonomy (Zhang et al., 2024, Zhang et al., 30 Mar 2026, Zhang et al., 14 Oct 2025, Chen et al., 26 Sep 2025, Wang et al., 9 Feb 2026, Hossain et al., 14 Jun 2026):

Attack Classes

• Direct and Indirect Prompt Injection (DPI, OPI): Overwriting instruction channels by textual inserts, tool return poisoning, or evasion via obfuscation.
• Memory Poisoning: Corrupting short- or long-term context to trigger malicious future planning.
• Plan-of-Thought (PoT) Backdoors: Planting covert triggers in system prompt or demonstration exemplars.
• Mixed/Composite Attacks: Orchestrating simultaneous compromise of multiple agent pipeline components.
• Tool Misuse and Malicious Tool Registration: Exploiting or introducing detector-evasive and privilege-escalating tool schemas, parameters, and responses (notably in Model Context Protocol environments (Zhang et al., 14 Oct 2025)).

Defenses

• Input Delimiters, Paraphrasing, Instructional Prevention: Prompt-level boundaries that attempt to confine model attention or neutralize overt attack vectors.
• Plan Shuffling: Disrupting demonstration-based triggers in PoT backdoor scenarios.
• Detection-Based Barriers: Perplexity or LLM-based classifiers to flag anomalous or high-entropy observations.
• Privilege Enforcement, Provenance Gating, Capability Allow-listing: Pre-generation projection onto authorized observations/actions, reliable for closing unauthorized channels (Alpay et al., 25 May 2026).
• Runtime Output Validation: Post-generation checkers to block or audit agent outputs and actions before execution.

A key insight is that prompt annotation alone is insufficient: only defenses that enforce input or output projections (removal of unauthorized information before generation or execution) provide strong noninterference guarantees (Alpay et al., 25 May 2026).

3. Benchmark Design and Evaluation Protocols

Agent Security Bench frameworks instantiate large-scale, multi-domain, and multi-agent evaluations across real-world–inspired scenarios (Zhang et al., 2024, Hofman et al., 21 May 2025, Cheng et al., 1 Jun 2026, Tian et al., 16 Jun 2026, Ying et al., 11 Oct 2025):

• Domains covered include IT, finance, medicine, e-commerce, autonomous vehicles, web automation, software reasoning, and more.
• Benchmarks enumerate hundreds to thousands of attacks, leveraging both synthetic and sourced prompt/memory/tool adversarial tasks.
• Each task is mapped to a canonical expected action sequence; deviation detected according to policy violation or forbidden outcome predicates.

Metrics are rigorously defined, with standard forms including: $$\begin{aligned} &\mathrm{ASR} = \frac{\#\text{successful attack cases}}{\#\text{total attempts}} \ &\mathrm{Refusal\ Rate\ (RR)} = \frac{\#\text{correct safety refusals}}{\#\text{total attacks}} \ &\mathrm{PNA} = \frac{\#\text{benign tasks completed}}{\#\text{total normal tasks}} \ &\mathrm{UST} = \mathrm{PNA}_d - \mathrm{ASR}_d\ (\text{utility--security trade-off}) \end{aligned}$$ Composite scores such as Net Resilient Performance (NRP) (Zhang et al., 14 Oct 2025) and $F_{\mathrm{attack}}$ (Cheng et al., 1 Jun 2026) directly encode operational utility versus residual attack surface.

Benchmarks such as SecureAgentBench (Chen et al., 26 Sep 2025), SafeClawBench (Tian et al., 16 Jun 2026), SeClaw (Cheng et al., 1 Jun 2026), and SkillVetBench (Hossain et al., 14 Jun 2026, Hossain et al., 30 May 2026) extend the methodologies to large-scale code, web automation, agent skill vetting, and spec-driven risk environments, often using Docker-based or containerized harnesses to evaluate full-repository or cross-skill adversarial flows.

4. Empirical Results and Findings

ASB and contemporary benchmarks consistently expose the fragility of current LLM agents and defenses:

• Average attack success rates in mixed or direct-injection settings frequently exceed 70–84% (e.g., ASB Mixed Attack ASR = 84.30%; SecureAgentBench SecureCorrectRate ≈ 9.2–15.2%; GrantBox ASR ≈ 84.8%; MCP Security Bench OP attack ≈ 74%) (Zhang et al., 2024, Chen et al., 26 Sep 2025, Hofman et al., 21 May 2025, Zhang et al., 30 Mar 2026, Zhang et al., 14 Oct 2025).
• Defenses such as delimiters, paraphrasing, and sandwiching typically reduce ASR only marginally (≤3–20% relative), and prompt-level security instructions are insufficient for meaningful mitigation.
• Path-level vulnerabilities frequently manifest: ASB, SCR-Bench (Xie et al., 13 Jun 2026), and SkillVetBench reveal that skills or toolchains benign in isolation can become hazardous in composition, e.g., through capability flow (CapFlow ASR = 33.6%), trust transfer (TrustLift ASR = 83.9%), or authorization confusion (AuthBlur +71% relative increase in risky approval).
• Multilingual security benchmarking (MAPS) shows attack success rates sharply worsen in non-English settings (average ASR increase +0.07, with individual languages up to +0.17), driven by translation artifacts and incomplete language-specific alignment (Hofman et al., 21 May 2025).
• In SecureAgentBench and SEC-bench Pro, LLM code agents exhibit severe underperformance on complex secure software tasks; PoC generation/patching success tops out below 40%, and correctness does not imply security (Chen et al., 26 Sep 2025, Lee et al., 26 May 2026).

5. Real-World, Path-Aware, and Runtime Security Analysis

Recent benchmarks emphasize path-aware, multi-stage, and runtime evaluation protocols:

• SCR-Bench (Xie et al., 13 Jun 2026) formally models skill composition as a directed dependency graph $G_h$, evaluating risk per path activation and downstream state changes instead of isolated node vetting.
• SafeClawArena and SeClaw (Niu et al., 29 Jun 2026, Cheng et al., 1 Jun 2026) adopt system-level and spec-driven adversarial task synthesis, integrating persistent state, cross-boundary data flow, and trajectory-aware sandboxing.
• SkillVetBench and SkillVetBench SARS (Hossain et al., 14 Jun 2026, Hossain et al., 30 May 2026) combine LLM-judge semantic analysis with sandboxed runtime auditing, capturing multi-modal and instruction-layer attacks.
• AgentTrust and TrustBench (Yang, 6 May 2026, Sharma et al., 10 Mar 2026) implement real-time action interception, explainable verdict scores, and plugin-based policy modules for runtime LLM agent safety with median sub-2ms intervention latency.

Path-level and runtime methods, along with compositional and trajectory-audited metrics, consistently demonstrate that evaluation of semantic compliance alone is insufficient. Observable harms can and do occur outside semantic-judge acceptance, highlighting the critical necessity for enforceable boundaries, runtime validation, and defense-in-depth.

6. Limitations, Challenges, and Future Directions

Despite major advances, current agent security benchmarks face intrinsic challenges:

• Static or text-only analysis misses runtime and emergent compositional hazards.
• Privilege escalation via tool misuse and Model Context Protocol integration remain intractable for prompt-only alignment.
• Coverage limitations persist, particularly for languages, agent paradigms (e.g., multi-agent workflows), and real-world regression or continuous-release settings.
• Certified robustness at the intent-to-execution boundary, especially under permitted leakage and privilege transformation, is an open research problem (Alpay et al., 25 May 2026).
• The divergence between operational utility and attack resilience, as quantified by NRP/UST and $q$0, suggests trade-offs yet unresolved—stronger models often exhibit both higher task completion and higher vulnerability (Zhang et al., 14 Oct 2025, Zhang et al., 30 Mar 2026).

Recommended research and deployment directions encompass:

• Automated, scalable spec-driven task synthesis for bespoke adversarial scenarios (Cheng et al., 1 Jun 2026).
• Path-aware composition risk modeling and sandboxed trajectory validation in skill ecosystems (Xie et al., 13 Jun 2026).
• Continuous evaluation pipelines and audit trail archiving in enterprise environments.
• Hybrid static–dynamic defenses, privilege-aware planning, and agent-side output validation.
• Hardening cross-ecosystem benchmarks to reflect realistic agent deployments, supply chain security, and multilingual/global contexts.

ASB and its successors remain the canonical reference point for reproducible and rigorous agent security evaluation in LLM-based systems, establishing a foundation for progress on both bench-level risk quantification and practical defense assessment (Zhang et al., 2024, Chen et al., 26 Sep 2025, Tian et al., 16 Jun 2026, Xie et al., 13 Jun 2026, Alpay et al., 25 May 2026, Zhang et al., 14 Oct 2025, Hofman et al., 21 May 2025, Cheng et al., 1 Jun 2026, Zhang et al., 30 Mar 2026).