Trade-off Between Coverage and Source Quality in AI Knowledge Systems

Determine principled methods to trade off topical coverage against source quality when constructing AI-powered knowledge systems, so that the breadth of addressable topics is balanced with the verifiability and reliability of sources.

Background

Within the Discussion section, the authors highlight a fundamental design tension for AI-powered knowledge systems: increasing the breadth of topics a system can address often conflicts with maintaining strict source quality and verifiability. In AVA’s deployment, an initial narrow, high-quality corpus produced high abstention rates, while expanding the curated corpus reduced abstention without sacrificing evidence standards.

The paper frames this as an unresolved, general problem beyond their specific system, suggesting that future work should identify principled strategies for balancing corpus coverage and source quality, potentially informing how to set verification thresholds, monitor abstention rates, and decide when to expand a vetted corpus.