Exception Taxonomy in Agentic Artifacts

• Exception taxonomy in agentic artifacts is a structured framework categorizing failures in autonomous, LLM-powered workflows across cognitive and operational phases.
• Taxonomies like TRAIL, Aegis, and SHIELDA offer granular classifications that enhance error diagnosis, benchmarking, and system recovery.
• Empirical evaluations show these frameworks improve debugging and reliability by systematically mapping errors to distinct agent workflow stages.

Exception taxonomy across agentic artifacts refers to structured frameworks for categorizing, formalizing, and diagnosing failures—sometimes termed “exceptions”—that arise during the execution of workflows by autonomous, agentic systems (typically LLM-based). Such taxonomies deliver a principled basis for error diagnosis, benchmarking, recovery, and system design. Research in this domain has produced complementary but distinct taxonomies, varying in granularity, abstraction, and operational focus, but unified in their attention to the multifaceted breakdowns that occur in modern agentic artifacts (Deshpande et al., 13 May 2025, Song et al., 27 Aug 2025, Zhou et al., 11 Aug 2025).

1. Definitions and Rationale

Exception taxonomies in agentic artifacts are formal structures that assign a unique type or class to each observed or anticipated failure during agent workflow execution. These exceptions range from language-model reasoning errors to operational breakdowns in tool invocation, resource management, and inter-agent communication. The aim is comprehensive coverage of error provenance—tracing exceptions across cognitive (reasoning/planning) and operational (execution/environment) phases.

Motivations for taxonomic structuring include: (1) scalable error analysis over complex traces, (2) reproducible evaluation of agent reliability, (3) systematic support for exception handling, and (4) comparability of systems and benchmarks at a fine granularity (Deshpande et al., 13 May 2025, Song et al., 27 Aug 2025, Zhou et al., 11 Aug 2025).

2. Leading Exception Taxonomies across Agentic Systems

Several taxonomies have gained prominence, each emphasizing different dimensions of agentic error:

2.1 TRAIL Taxonomy

The TRAIL hierarchy (Deshpande et al., 13 May 2025) organizes all turn-level failures in agentic workflows into a three-layer, disjoint partition:

• Top level:
  • Reasoning Errors ($R$)
  • System Execution Errors ($E$)
  • Planning & Coordination Errors ($P$)
• Second level: Each top-level category decomposes as follows:

| Category | Subcategories | |---------------------------------|---------------------------------------------------------------------| | Reasoning Errors ($R$) | Hallucinations ($H$), Information Processing ($I$), Decision Making ($D$), Output Generation ($O$) | | System Execution Errors ($E$) | Configuration Issues ($C$), API & System Issues ($A$), Resource Management ($M$) | | Planning & Coordination Errors ($P$) | Context Management ($X$), Task Management ($T$) |

Each leaf class is defined over execution spans $s\in S$, with precise set-theoretic or predicate-style definitions. For instance, $$H_{lang} = \{ s \in S \mid s\textrm{’s content contradicts ground truth or is fabricated} \}$$. This hierarchy is validated on data from software engineering (SWE-Bench), multi-agent open-world IR (GAIA), and is claimed to generalize across single- and multi-agent artifacts.

2.2 Aegis Exception Taxonomy

Aegis (Song et al., 27 Aug 2025) proposes a concise, six-type taxonomy for agent-environment interaction failures, clustered into three families:

Group	Failure Modes
Exploration Failures	State-space Navigation Failure, State Awareness Failure
Exploitation Failures	Tool Output Processing Failure, Domain Rule Violation, User Instruction Following Failure
Resource Exhaustion	Resource Exhaustion Failure

Each mode is characterized both formally (e.g., violation of a correctness predicate $C$ over the observed trace prefix) and by its manifestation in representative benchmarks: State-space Navigation (failure to explore a state space or collect necessary context), State Awareness (misalignment between agent model and environment true state), Exploitation failures (errors in processing outputs, following domain rules, adhering to user natural language), and resource exhaustion (turn or token budget overruns).

2.3 SHIELDA’s Artifact-Oriented Taxonomy

SHIELDA (Zhou et al., 11 Aug 2025) offers the most granular, artifact-centered taxonomy: 36 exceptions mapped to 12 agentic artifacts/components, spanning Reasoning/Planning (RP), Execution (E), or both. Artifacts include Goal, Context, Reasoning, Planning, Memory, KnowledgeBase, Model, Tool, Interface, TaskFlow, OtherAgent, ExternalSystem. Each exception $e_i$ is assigned to a unique artifact via $f(e_i)$ and its typical phase via $\varphi(e)$.

For example: AmbiguousGoal (Goal, RP), ToolInvocationException (Tool, E), MemoryPoisoning (Memory, RP/E), and ProtocolMismatch (ExternalSystem, E).

3. Formal Characterization and Structure

Core formalisms employed across these taxonomies include:

• Taxonomic partitions: Spans or events $S$ are partitioned into mutually exclusive sets (e.g., $R \cup E \cup P = S$; $R \cap E = \emptyset$, etc., in TRAIL).
• Artifact and phase mapping: Each exception is mapped to one artifact ($f: E \to A$) and, if applicable, a workflow phase ($$\varphi: E \to \{\mathrm{RP}, \mathrm{E}, \mathrm{RP/E}\}$$).
• Predicate-based definitions: Many exceptions are defined by predicates on span content, environmental state, or failed compliance with external specifications (e.g., $R(a_t, s_t) = \mathrm{False}$ for domain rule violation in Aegis).

The SHIELDA taxonomy, for example, is represented as $E = \bigcup_{a} E_a$, where $E_a = \{ e\in E \mid f(e)=a \}$, and each $e$ also labeled by $\varphi(e)$ (Zhou et al., 11 Aug 2025).

4. Manifestation and Prevalence Across Agentic Artifacts

Empirical analysis across multiple domains demonstrates how exception types cluster by artifact and workflow archetype:

• Single-agent software pipelines (SWE-Bench): Output Generation and System Execution errors dominate; configuration and schema issues are pervasive.
• Multi-agent open-world IR (GAIA): Planning & Coordination and Information Processing errors increase in prevalence, especially context mismanagement and retrieval misinterpretation.
• Tool-heavy environments: Tool invocation, output malformation, and unavailability emerge as recurrent sources of failure across both interactive and batch agentic tasks.
• Memory-centric artifacts: MemoryPoisoning and OutdatedMemory have high impact in agents with persistent or shared memory integration.

Key quantitative findings from the TRAIL dataset (Deshpande et al., 13 May 2025):

Error Category	Percent of All Errors
Reasoning Errors	60%
System Execution Errors	15%
Planning & Coordination	25%

Within reasoning, Output Generation constitutes 42% of errors. Resource exhaustion can account for >80% of failures in data-intensive settings (CRM, Retail) (Song et al., 27 Aug 2025).

5. Illustrative Exception Types and Examples

Exception categories are concretely instantiated in system traces. Selected examples:

• Tool-Output Misinterpretation (TRAIL I_mis): Agent misreads an empty tool output list as substantive evidence (Deshpande et al., 13 May 2025).
• State Awareness Failure (Aegis): Agent deletes a file in an unintended directory due to internal-environment state desynchronization (Song et al., 27 Aug 2025).
• FaultyTaskStructuring (SHIELDA): Planner agent schedules a hotel for fewer nights than required, violating global constraints (Zhou et al., 11 Aug 2025).
• UnavailableTool (SHIELDA): Third-party API downtime halts agent progression despite correct agent reasoning (Zhou et al., 11 Aug 2025).
• MemoryPoisoning (SHIELDA): Agent retrieves externally corrupted demonstration data, resulting in dangerous driving plans (Zhou et al., 11 Aug 2025).

6. Impact on Diagnostics, Recovery, and Benchmarking

The adoption of formal exception taxonomies impacts agentic system design and evaluation across several dimensions:

• Error localization: Automated or human-in-the-loop systems can classify errors by type, supporting targeted debugging and recovery strategies (Deshpande et al., 13 May 2025).
• Structured handling and recovery: SHIELDA introduces modular handler patterns specialized for each exception type, comprising local handling, flow control, and state recovery modules (Zhou et al., 11 Aug 2025).
• Performance evaluation: Fine-grained error metrics enable per-category reporting, facilitating cross-system comparisons at the resolution of root-cause classes (Deshpande et al., 13 May 2025).
• Benchmarking: Taxonomies such as TRAIL and Aegis yield standardized evaluation protocols; e.g., measuring LLM judge accuracy in localizing exception spans (with current state-of-the-art joint accuracy at 11% across domains) (Deshpande et al., 13 May 2025).
• System-for-agent improvements: Environment-side interventions (as in Aegis) targeting specific failure modes can improve agent success rates by 6.7–12.5%, rivaling model improvements (Song et al., 27 Aug 2025).

7. Comparative Perspectives and Generalization

Despite differences in granularity and abstraction, the leading taxonomies reveal a convergent structure: all exceptions in agentic artifacts are grounded in agent cognition, system interface, execution environment, or inter-agent communication. The Aegis taxonomy asserts near-completeness for single-agent, tool-enabled environments, while the SHIELDA taxonomy is designed for cross-phase, cross-artifact composability (36 types across 12 artifacts). TRAIL demonstrates ecological validity across benchmark classes.

A plausible implication is that future work will further unify these frameworks, extending them to dynamic, open-ended agent workflows, reasoning chains, multi-modal systems, and compositional recovery architectures.

• TRAIL: "TRAIL: Trace Reasoning and Agentic Issue Localization" (Deshpande et al., 13 May 2025)
• Aegis: "Aegis: Taxonomy and Optimizations for Overcoming Agent-Environment Failures in LLM Agents" (Song et al., 27 Aug 2025)
• SHIELDA: "SHIELDA: Structured Handling of Exceptions in LLM-Driven Agentic Workflows" (Zhou et al., 11 Aug 2025)