Index Configuration in the RAG-IR Abstraction

Determine whether the RAG-IR workload abstraction should include explicit database index configurations—such as index type and tuning parameters—as part of the algorithm input, or instead expose only retrieval quality requirements (e.g., recall thresholds) to system designers; ascertain the impact of these choices on end-to-end retrieval-augmented generation quality when different index configurations meeting the same recall may return different document sets.

Background

RAG-IR is proposed as a per-request, dataflow-based workload abstraction that cleanly separates algorithm-level quality concerns from system-level performance considerations. A design decision in RAG-IR is whether to require the algorithm side to specify the index type and parameters or to leave index selection to the system side subject to a quality target (e.g., recall).

Specifying the index configuration guarantees reproducibility of the retrieved results used to evaluate generation quality, but it may limit system-side performance optimization. Conversely, exposing only a quality requirement enables system designers to choose optimized indexes, but different indexes satisfying the same recall can return different document sets, potentially altering end-to-end RAG quality.