Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming

Abstract: The fast growth of open-source AI infrastructure, from model serving engines and agent platforms to the Model Context Protocol (MCP) ecosystem and the LLMs themselves, has outpaced the security tooling available to defend it. We present AI-Infra-Guard, an open-source framework that organizes AI red teaming around a single observation: the attack surface of an AI agent is stratified across layers (infrastructure, protocol/tool, agent behavior, and model), and no single detection paradigm fits all of them. The framework therefore matches a paradigm to each layer, from deterministic rule matching over 75+ AI components and 1{,}400+ vulnerability rules, through LLM-driven agentic auditing of MCP servers and agent-skill packages and multi-turn black-box agent red teaming, to a jailbreak harness with 26+ attack operators over sixteen datasets. To our knowledge it is the only open-source framework to span all of these, including supply-chain auditing of the agent skills that increasingly extend AI agents. We release AI-Infra-Guard as open source so that \emph{layer-paradigm matching} can serve as a practical foundation for agent security and a shared base for the community to build on.

• The paper introduces AI-Infra-Guard, an open-source framework that aligns detection techniques with four distinct security layers.
• It employs deterministic rule matching, LLM-driven semantic auditing, multi-turn adversarial red teaming, and statistical evaluation.
• Empirical results demonstrate high F1 scores and recall, effectively exposing vulnerabilities across infrastructure, protocol, behavior, and model layers.

Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming

Introduction and Motivation

The rapid proliferation of open-source AI infrastructure—from model-serving engines and agent orchestration platforms to the Model Context Protocol (MCP) and the underlying LLMs—has resulted in a significantly expanded and stratified attack surface. Traditional security assessment tools, built around established catalogues and regular versioning conventions, fail to account for the irregularities and novel risks inherent to modern AI systems. The landscape is further complicated by the co-existence of four distinct security layers: infrastructure, protocol/tool, agent behavior, and model alignment. Each requires a fundamentally different detection paradigm. "Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming" (2606.31227) introduces AI-Infra-Guard, an open-source framework that operationalizes the principle of layer-paradigm matching. The system integrates deterministic rule matching, LLM-driven agentic auditing, multi-turn adversarial red teaming, and statistical alignment robustness evaluation, effectively spanning all critical strata of the AI agent attack surface.

Layer-Paradigm Matching: A Theoretical Foundation

The central theoretical claim advanced in this work is that effective security assessment in AI systems demands matching each attack surface layer with a detection technique that yields sufficiently strong evidence for the relevant class of vulnerabilities. The framework distinguishes four classes of evidence—signature, semantic, behavioral, and statistical—and aligns each with a corresponding detection paradigm:

• Infrastructure: Deterministic rule matching for observable signatures and known vulnerabilities
• Protocol/Tool: LLM-driven semantic analysis for reasoning about code, tool metadata, and MCP-specific attack vectors
• Agent Behavior: Multi-turn, adversarial black-box red teaming to expose runtime-only vulnerabilities
• Model Alignment: Large-scale statistical attack enumeration with LLM-based judgment to evaluate alignment robustness

This assignment is grounded in two formal principles:

1. Security Heterogeneity Principle: Classes of AI security risks arise from fundamentally different system properties, precluding evaluation by a single universal methodology.
2. Evidence Sufficiency Principle: A security assessment procedure is valid only if it produces evidence sufficient to establish the security property at each layer.

Infrastructure Scanning: Deterministic Rule-Based Layer

Infrastructure scanning targets the identification of deployed AI components, their versions, and known exposures. The module implements a declarative, boolean-expression-based matching language for both component fingerprinting and vulnerability detection, decoupled from conventional regular-expression-based engines. The fingerprint corpus covers 75+ AI components with over 1,400 vulnerability rules, supporting non-semantic version normalization to address the spectrum of versioning inconsistencies observed in practice.

Upon component identification, the matching engine applies version predicates or infers findings when version data is unavailable. It explicitly classifies findings into three confidence tiers: verified (evidence-based), version-matched (advisory-driven), and inferred (identity-only). The scan pipeline is highly parallelized and disk-backed, enabling coverage of large, potentially internet-scale target sets without sacrificing resource efficiency.

Figure 1: The infrastructure-scanning pipeline, combining fingerprint analysis, vulnerability matching, and agentic visual/risk assessment, culminating in a structured and actionable security report.

Protocol/Tool Layer: LLM-Driven MCP and Skill Auditing

The MCP auditing module addresses the inadequacy of signature-based detection for protocol-level vulnerabilities, such as command injection, credential theft, tool poisoning, and tool shadowing. AI-Infra-Guard deploys an agentic harness, leveraging LLM-as-auditor within bounded, multi-stage pipelines. The harness is equipped with specialized tools (filesystem, shell/grep, MCP interaction agents), dynamic context management, and role-based model routing. The detection corpus is encoded in natural-language Prompt-as-Rule specifications, with explicit inclusion and exclusion criteria designed to minimize both false positives and over-reporting, reflecting best practices from the OWASP MCP Top 10.

The system supports both static (white-box) and dynamic (black-box) MCP auditing. Static analysis enables direct reasoning over source; dynamic analysis confines itself to tool metadata and runtime interaction, emphasizing output-centric evidence validation, especially for tool poisoning and indirect prompt injection.

Figure 2: The MCP-auditing pipeline, illustrating the sequential reasoning, interaction, and validation carried out across static and dynamic assessment modes.

Skill auditing extends this paradigm to the AI agent supply chain. Skills—modular capability packages—introduce new supply-chain risks, including manifest tampering and privilege escalation. The agentic pipeline decomposes skill artifacts, applies lightweight static risk retrieval, and employs LLM-based contextual analysis within a controlled environment. The integration of external threat intelligence and iterative badcase-driven improvement loops further enhances the precision and adaptability of detection.

Agent Behavior: Multi-Turn Black-Box Red Teaming

Assessment of agent behavior necessitates adversarial multi-turn interaction. The framework abstracts attacks as skill manifests (advanced Prompt-as-Rule specifications), each operationalized as an attack playbook targeting specific risk families: data leakage, tool abuse, indirect injection, and authorization bypass. These manifest-constrained agents reason over target capabilities, adapt attack strategies dynamically, and enforce structured escalation ladders combined with cost-aware stop rules.

A salient design feature is the systematic anchoring of subjective LLM judgments with deterministic evidence. For canary token-based SSRF or indirect injection via marker tokens, confirmation moves beyond LLM opinion to verifiable output, greatly increasing report actionability.

Model-Layer Jailbreak Evaluation: Large-Scale Statistical Red Teaming

The model layer is assessed through comprehensive jailbreak evaluation, framed as a statistical robustness question. AI-Infra-Guard integrates a composable harness unifying a wide spectrum of attack operators (single-turn encoding, role-play, context poisoning, multi-turn crescendo/tree-search strategies) and red-teaming datasets ($\sim$7,000 harmful prompts across 16 curated corpora). Modular plugin support and customizable LLM-as-judge metrics expand the system's range and adaptability. The framework generates directly comparable safety scores and attack success rates across models and deployments.

Figure 3: The jailbreak-evaluation harness, demonstrating the synergy between systematic attack operator application and rigorous model outcome judgment.

System Architecture and Integration

The detection modules are orchestrated through a distributed server-agent architecture, built for scalability and uniform dispatch of heterogeneous, resource-heterogeneous tasks. Workers (in Go and Python) execute scans and audits, streaming structured events and intermediate results to clients via WebSocket/SSE channels. The centralized knowledge base (fingerprints, CVEs, benchmarks) is used by all modules; rule bases and plugins are remotely updatable. Modular delivery via CLI, web UI, or agent skill ensures broad ecosystem integration.

Figure 4: The distributed server-agent architecture connecting user interfaces, core detection engines, agent modules, and centralized knowledge resources.

Empirical Results and Claims

• The skill supply chain module, evaluated on SkillTrustBench (5,520 cases across 9 threat categories), achieves a loose F1 score above 0.98 with leading models (e.g., Claude Opus 4.6, GLM 5.1), and recall near 1.0, confirming assessment effectiveness is model-dependent and that the detection harness exposes base-model limitations rather than adding bottlenecks.
• The framework is, to the authors' knowledge, the only open-source system that comprehensively spans infrastructure, protocol/tool, agent behavior, and model layers, including agent skill supply chain auditing, and matches each to its theoretically justified detection paradigm.
• Prompt-as-Rule, combined with explicit exclusion criteria and objective confirmation (canaries, markers), consistently improves both precision and recall, particularly when used in LLM-driven domains susceptible to over-reporting or prompt injection.

Broader Implications and Future Directions

This work establishes a formal and operational link between the spectrum of AI agent vulnerabilities and the evidentiary and methodological requirements for their assessment. Practically, the system models how heterogeneous, evolving agent ecosystems can be defended through extensible, community-driven knowledge bases and audit modules. Theoretical contributions include formalizing the evidence classes underlying security findings and setting a precedent for layer-adaptive security assessment.

The modular, compositional approach to attack operator and metric plugin management in the model-layer component enables rapid incorporation of emerging jailbreak techniques and benchmarks. Future work may focus on large-scale measurement of real-world agent exposures, cross-layer detection integration (feeding infrastructure/model findings into behavioral probes), and continuous expansion of rule and benchmark corpora.

Conclusion

"Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming" delivers a rigorously structured, technically diverse framework for agent security assessment. Through systematic application of layer-paradigm matching, it aligns detection procedures to the inherent nature of vulnerabilities at each layer, ensuring both pragmatic effectiveness and theoretical soundness. The adoption of Prompt-as-Rule, objective confirmation, and scanner self-defense patterns serves as a blueprint for future security tooling in complex AI ecosystems. This work sets a new standard for comprehensive, evidence-driven AI agent security auditing and defines an extensible architecture for the evolving threat landscape.