Extending raggy’s debugging primitives to agent-based architectures

Determine whether the raggy debugging primitives—specifically, the Query, Retriever, LLM, and Answer components—can be generalized to more complex agent-based architectures that include additional tools and interactions (such as document creation and database querying), and specify how to expand the set of debugging primitives to adequately support these architectures.

Background

The paper introduces raggy, a tool that provides four core primitives (Query, Retriever, LLM, Answer) and an interactive interface for debugging retrieval-augmented generation (RAG) pipelines. In the discussion, the authors note that RAG pipelines increasingly operate as components within broader agent-based systems involving multiple tools and complex workflows.

Given this broader context, the authors explicitly raise the unresolved question of whether raggy’s current primitives are sufficient for multi-agent scenarios and what additional primitives or extensions would be required. Addressing this problem would help bridge the gap between RAG-specific debugging and comprehensive agent-based system debugging.