AgentDebug: Debugging LLM Agent Systems

• AgentDebug is a suite of methodologies that defines structured observability and deterministic workflows to diagnose and correct failures in LLM-based agent systems.
• It employs techniques such as context-bound delegation, interactive debugging, and constraint-based root-cause localization to improve error resolution.
• Empirical evaluations demonstrate significant gains in bug localization accuracy and recovery rates, reducing developer frustration and enhancing overall system reliability.

AgentDebug is a technical paradigm and suite of methodologies for precisely diagnosing, attributing, and correcting failures in LLM-based agentic AI systems, emphasizing systematic observability, intervention-driven root cause localization, and evidence-grounded recovery. The term and reference implementations span research on delegation observability, trajectory localization, constraint-driven diagnostics, interactive debugging, and recovery-layer architecture in both single- and multi-agent workflows (Mishra et al., 8 Jun 2026, Hutter et al., 6 Feb 2026, Ma et al., 7 Dec 2025, Garg et al., 20 Feb 2026, Liu et al., 23 Mar 2026, Fu et al., 13 Oct 2025, Mulian et al., 18 Feb 2026, Wang et al., 26 May 2026, Zhu et al., 29 Sep 2025, Zhao et al., 9 May 2026, Epperson et al., 3 Mar 2025). AgentDebug research seeks to move post-hoc error reconstruction from unstructured, heuristic log analysis to deterministic, context-aware workflows that support blameless incident attribution, blast radius analysis, high-fidelity step localization, and iterative correction.

1. The Observability Substrate and Delegation Context

Traditional tracing and audit mechanisms are structurally incapable of unambiguously linking tool actions in agentic systems to the precise scope of delegated authority under which they execute. This leads to ambiguity: the same execution trace may be consistent with multiple, incompatible delegation assignments. AgentDebug systems, as formalized in “Observability for Delegated Execution in Agentic AI Systems” (Mishra et al., 8 Jun 2026), resolve this by enforcing at-capture context binding via:

• Gateway placement: Lightweight gateways intercept and wrap every agent–tool boundary, minting or validating delegation identifiers. Each outbound interaction is tagged at the point of invocation with an immutable delegation_id (with lineage via parent_id upon subdelegation) and reduced to a small set of normalized action/resource vocabularies (e.g., {read, write, share, invoke} plus resource type and sensitivity).
• Common Information Model (CIM): The CIM encodes both the execution graph (events and causal trace/span links) and the authority graph (delegation–lineage DAG). This dual representation preserves separation between the chain of causality (trace_id, span_id) and the exact locus of delegated authority (delegation_id).
• Semantic invariants: The system maintains invariants such as stable, non-reusable delegation IDs, strict separation between authority and causality, lineage closure (delegation graph forms a DAG), cross-system action/resource normalization, and principal integrity enforcement.
• Direct queries: With CIM events, reconstructions such as “all actions executed under delegation d” are reduced to direct lineage closure and filter queries, irrespective of temporal fragmentation or cross-tool fanout. Forensic queries no longer depend on heuristic windowing or cross-service correlation.

2. Debugging Primitives and Interactive Control in Agentic Workflows

AgentDebug methodologies extend traditional debugging paradigms to the agent level, subsuming trajectories of LLM queries, tool calls, and multi-agent plan–execute loops into structured, interactively controllable sessions:

• Breakpoint adaptation: AgentStepper and related frameworks instrument agents to support breakpointing not only at code steps, but at semantically meaningful agent events: before/after LLM prompt generation, before/after tool invocation, and manual intervention points (Hutter et al., 6 Feb 2026).
• Stepwise and live editing: Execution can be paused, modified at any interaction cycle, and resumed. This enables precise counterfactual exploration—what would have happened if a plan, parameter, or tool invocation had been different?
• Trajectory models: Agent execution is formally modeled as finite event sequences—tuples of (prompt, response, tool call, outcome, code commit), represented as conversation graphs and repository diffs. These structures enable both graphical visualization and algorithmic breakpoint insertion.
• Low integration overhead: Adding full interactive agent debugging (e.g., AgentStepper) requires only modest code changes—typically 5–7 added API calls and ≈40 lines per agent scaffold.
• Quantitative benefit: User studies on AgentStepper demonstrate significant improvements in bug localization success (60% vs. 17%) and stark reductions in developer frustration (2.4 vs. 5.4 on a 7-point scale) (Hutter et al., 6 Feb 2026).

3. Root Cause Attribution and Constraint-Based Localization

AgentDebug frameworks implement principled, taxonomy-driven root-cause localization in complex agent trajectories via:

• Failure taxonomies: Modular hierarchies (e.g., AgentErrorTaxonomy in (Zhu et al., 29 Sep 2025), nine-category grounded-theory in (Barke et al., 2 Feb 2026)) systematically categorize errors by module—memory, planning, reflection, action, system—and by failure type (plan adherence, invalid invocation, invention, misinterpretation, etc.).
• Constraint synthesis: Automated or semi-automated constraint generators derive both global invariants (tool schema, policies) and dynamic, step-conditioned assertions (relational/temporal invariants over evolving agent state). At each event step, these constraints are evaluated for satisfaction or violation.
• Validation log and LLM adjudication: All constraint violations (with localized evidence) are logged stepwise. LLM-based “judge” modules map violation patterns (with checklist-style taxonomies) to (step, category) pairs, identifying the earliest “irrecoverable” failure step.
• Evaluation benchmarks: AgentRx and AgentDebug are tested on large trajectory datasets (e.g., RootSE (Wang et al., 26 May 2026), AgentErrorBench (Zhu et al., 29 Sep 2025)), achieving 24–25 pp gains in localization accuracy and up to 26% relative increases in downstream task success over prior baselines (Barke et al., 2 Feb 2026, Wang et al., 26 May 2026, Zhu et al., 29 Sep 2025).

4. Intervention-Driven Debugging and Automated Structured Recovery

Resolution in strongly agentic workflows demands not only diagnosis, but validated intervention and actionable recovery:

• Do-then-Verify: Frameworks such as DoVer (Ma et al., 7 Dec 2025) segment complex runs into trials, generate failure hypotheses, design minimal interventions (plan/instruction edits), replay from checkpoints, and directly measure recovery (full success or milestone gain). This outcome-oriented loop rigorously tests whether a candidate intervention resolves the task or makes measurable progress.
• Recovery Guidance Gates: PROBE (Zhao et al., 9 May 2026) formalizes the translation from structured evidence (multi-signal telemetry, diagnosis) to intervention hints via a Guidance Gate. This gate enforces grounding (all recovery targets trace to evidence), actionability (the fix is executable under constraints), and domain-appropriate scope (no speculative infra fixes).
• Key metrics: Recovery protocols are evaluated for diagnosis accuracy and end-to-end recovery rate, with PROBE yielding a 65% top-1 diagnosis and 21% recovery on previously failing SWE-bench and production workflow cases (Zhao et al., 9 May 2026).
• Counterfactual rerolling: AgentDebug architectures support iterative rollouts from the critical failure step, injecting feedback and tracking if correction resolves the primary failure while preserving compute efficiency (Zhu et al., 29 Sep 2025).

5. Real-World Integration: Multi-Agent, Code, and Production Cases

AgentDebug is instantiated across disparate agentic systems, including collaborative multi-agent teams (e.g., AGDebugger (Epperson et al., 3 Mar 2025)), coding agents (e.g., Debug2Fix (Garg et al., 20 Feb 2026), DAIRA (Liu et al., 23 Mar 2026)), education agents (DebugTA (Fu et al., 13 Oct 2025)), and real-world production orchestrators (AgentFixer (Mulian et al., 18 Feb 2026)):

• Structured session management: Systems like AGDebugger (Epperson et al., 3 Mar 2025) and AgentStepper (Hutter et al., 6 Feb 2026) support message-level checkpointing, agent state reversion, inline edit-and-replay, and configuration forking, yielding rapid counterfactual “what-if” exploration.
• Tool and dynamic analysis integration: Debug2Fix and DAIRA (Garg et al., 20 Feb 2026, Liu et al., 23 Mar 2026) embed program-level debugging (JDB, PDB, test tracing) into coding agent workflows. This allows runtime variable inspection, stack trace analysis, and step-level semantic report parsing to be surfaced as first-class evidence in the LLM’s decision loop, substantially improving bug resolution.
• Pluggable interface and side-channel modes: Recovery layers such as PROBE are attached as non-intrusive side channels requiring no agent policy/tool/core change—side-channel hints or artifacts feed diagnosis back into the agent loop or incident management pipelines.
• Empirical impact: Across tasks (WebArena, AppWorld, SWE-bench) and system scales (Llama-4, GPT-4o, Mistral), integrating AgentDebug layers yields improvements in pass-rate (up to 12–22% over baseline) and enables mid-size models to close the performance gap with frontier LLMs (Mulian et al., 18 Feb 2026, Garg et al., 20 Feb 2026, Liu et al., 23 Mar 2026).

6. Limitations, Open Challenges, and Future Directions

AgentDebug frameworks present several tradeoffs and continuing research challenges:

• Constraint and taxonomy coverage: Performance is sensitive to the fidelity of error taxonomies and quality of constraint synthesis, particularly in novel or underspecified domains. Human-in-the-loop refinement and domain adaptation remain open areas (Barke et al., 2 Feb 2026, Zhu et al., 29 Sep 2025).
• Intervention scope: Most implementations (DoVer, PROBE) limit interventions to orchestrator-level edits or bounded guidance; agent code/tool augmentation and multi-round, cross-service orchestrations are future targets.
• LLM subjectivity: LLM-based judges can introduce biases in step or category localization; techniques for interpretability, auditors, or ensemble consensus are increasingly important.
• Handling irrevocable side effects: For agents triggering IRL real-world effects (e.g., emails, database changes), safe-action modeling, dry-run simulation, and audit replay are recognized needs (Epperson et al., 3 Mar 2025).
• Scalability and efficiency: Systems such as TrajAudit (Wang et al., 26 May 2026) address long-context and high-noise trajectories by semantic folding and on-demand evidence retrieval, achieving token savings (≥ 18%) without degradation in localization accuracy.
• Self-improving loops: Feeding aggregated diagnostic outputs into LLM-based self-reflection (AgentFixer-style interactive debug prompts) opens the way to automated, dialog-driven agent improvement cycles (Mulian et al., 18 Feb 2026).

AgentDebug has rapidly evolved from conceptual taxonomies and manual root-cause analysis to concrete, production-grade pipelines for fine-grained, interpretable, and actionable agent debugging. Its integration of delegation-aware observability, trajectory-localization, structured evidence fusion, and interactive recovery loop is reshaping operational best practices for maintaining reliability in agentic AI systems.