Origin of Errors in RAG: Context Utilization vs. Context Sufficiency

Determine whether errors in Retrieval-Augmented Generation (RAG) systems arise because large language models fail to utilize the retrieved context or because the retrieved context is insufficient to answer the query.

Background

The paper studies Retrieval-Augmented Generation (RAG) systems, where LLMs receive external context at inference time to improve factuality and performance on open-domain question answering. A central uncertainty motivating the work is whether observed errors stem from the model’s inability to leverage the provided context or from the retrieval failing to supply information that is sufficient to answer the query.

To investigate this, the authors introduce the notion of “sufficient context” and develop an autorater to label question–context pairs as sufficient or insufficient without requiring ground-truth answers. This framing enables stratified analyses of model behavior, but the fundamental question of whether errors primarily reflect failures of context utilization versus context insufficiency is explicitly identified as open.