Set-Wise Passage Selection Approach

• Set-wise passage selection is a retrieval method that selects cohesive groups of passages to fulfill diverse query requirements rather than ranking them independently.
• It uses chain-of-thought reasoning to decompose complex queries and map subgoals to candidate passages, ensuring full evidence coverage.
• This approach enhances multi-hop QA and RAG by reducing redundancy and noise, leading to higher accuracy with fewer passages compared to traditional methods.

A set-wise passage selection approach is a family of retrieval and reranking methodologies that, rather than assessing passages independently, considers groups of candidate passages as a collection and selects the subset that best satisfies the complex, often multi-faceted, information needs of a query. This paradigm is motivated by observation that for many tasks, especially multi-hop question answering and retrieval-augmented generation (RAG), correct answers require assembling complementary evidence spread across multiple passages. Unlike pointwise or pairwise ranking that prioritizes relevance of individual passages, set-wise selection mechanisms explicitly aim for holistic coverage, diversity, and minimal redundancy in the selected set.

1. Motivation and Conceptual Shift

Traditional retrieval and RAG systems typically focus on ranking individual passages by their independent relevance to a query, using methods such as pointwise scoring, pairwise ranking, or listwise aggregate functions. However, multi-hop question answering and other complex information-seeking tasks expose the limitations of such approaches: a high-ranked list of individually relevant passages may still fail to collectively provide all the disparate evidence needed for reasoning chains or compositional answers (Lee et al., 9 Jul 2025).

The set-wise passage selection approach redefines the objective. Rather than determining the best ordering of passages, it seeks an optimal subset—a set (typically of fixed or dynamic size k)—that together comprehensively address all aspects of a query. This shift addresses key issues of diversity, coverage, and noise reduction, and is particularly valuable in scenarios where query intent or answer derivation depends on synthesizing multiple pieces of information distributed across the corpus.

2. Chain-of-Thought Reasoning and Information Requirement Identification

A central innovation in advanced set-wise selection methods is the use of chain-of-thought (CoT) reasoning to explicitly identify the subgoals, attributes, or "information requirements" embedded in a complex query. SETR (Set-wise passage selection for Retrieval-Augmented Generation) exemplifies this by prompting a LLM to decompose the query and produce an explicit list of requirements necessary for answer construction (Lee et al., 9 Jul 2025).

The CoT process operates as a precursor to passage selection:

• The model performs a stepwise decomposition of the question, listing required facts, relations, entities, or subtasks.
• Each requirement then serves as an anchor or mapping target for passages.
• This disciplined identification of information needs enables the subsequent set selection mechanism to ensure all elements receive coverage in the final selected set of passages.

By tracing which requirements are covered by which candidate passages, the approach enables not only effective relevance scoring, but also a principled means to enforce diversity and avoid redundant information within the final context set for RAG.

3. Set-Wise Selection Mechanism

The core set-wise selection procedure follows the information requirement mapping:

• Given a candidate pool (retrieved by BM25, dense models, or neural retrievers), the system compares each passage to the list of information requirements derived by CoT.
• Passages are evaluated and selected such that, together, they maximize collective satisfaction of the requirements with minimal overlap.
• Selection is guided by explicit rules or a LLM under a CoT-style prompt, which might, for example, ask to output "### Final Selection: [2] [5]" corresponding to the indices of the best covering set (Lee et al., 9 Jul 2025).

This contrasts with ranking-based selection, which typically outputs the top-k individually relevant passages without accounting for setwise complementarities or redundancy.

Key criteria considered in the selection include:

• Relevance: how well does a passage address a specified requirement?
• Diversity: does the passage offer distinct or complementary information compared to others already selected?
• Coverage: does the aggregate set span all listed requirements, minimizing gaps in evidence?

4. Differences from Traditional Ranking Methods

Set-wise passage selection introduces a number of methodological departures from prior frameworks:

• Target: It shifts from producing a globally optimal ranking to outputting a set of passages with maximal collective utility.
• Efficiency: In experiments, set selection approaches such as SETR achieve comparable or superior retrieval and QA accuracy using significantly fewer passages as input to the RAG generator (reduction of 40–50% in required set size) (Lee et al., 9 Jul 2025).
• Evaluation: Classical metrics like MRR, NDCG, and Hit@k remain useful, but information coverage metrics (e.g., IC@k, see below) are promoted to specifically measure set completeness versus gold-standard evidence.
• Reduction of Noise: Set selection actively reduces redundancy, ensuring that selected passages are not only individually strong but also non-overlapping and collectively exhaustive for the query's subgoals.

Whereas pointwise and pairwise models assume passage utility is independent, set-wise frameworks explicitly reason over the interdependencies and complementarities between passages.

5. Quantitative Evaluation and Information Coverage Metric

A quantitative haLLMark of set-wise passage selection is the use of set-coverage metrics to objectively measure how well the selected set covers the necessary gold evidence. A representative metric defined in SETR is:

$$\mathrm{IC}@k = \frac{\left| \bigcup_{i=1}^k \{ e \in p_i \} \cap \{ e \in E_\text{gold} \} \right|}{|E_\text{gold}|}$$

Where $p_i$ is the $i$-th selected passage, $e$ is an evidence span, and $E_\text{gold}$ is the set of all gold-standard evidence spans for the query (Lee et al., 9 Jul 2025).

Experiments demonstrate that set-selection methods:

• Increase average information coverage significantly compared to reranking-only baselines (e.g., from ~19% to over 36%).
• Achieve higher QA metrics (Exact Match, F1, accuracy) with smaller sets.
• Outperform both proprietary LLM-based rerankers (e.g., RankGPT) and open-source rerankers (e.g., RankLlama, RankVicuna) on multi-hop benchmarks such as HotpotQA, 2WikiMultiHopQA, MuSiQue, and MultiHopRAG (Lee et al., 9 Jul 2025).

6. Practical Applications and Integration

Set-wise passage selection provides value in settings where comprehensive context selection is essential:

• Multi-hop QA: The approach ensures that all reasoning steps are grounded in the evidence, which is crucial for legal, medical, or academic queries where the answer must be well-justified.
• Conversational Agents: In dialogue systems, it helps select context passages that collectively enrich the agent's response and maintain topic consistency across turns.
• Retrieval-Augmented Generation Pipelines: By integrating set-wise selectors in place of standard rerankers, systems reduce hallucination risk, handle input token budget more efficiently, and improve factual completeness.

Integration is feasible as a post-processing step following first-stage retrieval, leveraging the same candidate pool as traditional rerankers yet optimizing the final context set by maximizing coverage and diversity.

7. Technical Details and Future Directions

Technical implementation in SETR, and similar frameworks, is characterized by:

• Use of LLMs to explicitly decompose the query and map requirements to candidates via prompt engineering.
• Chain-of-thought prompts guiding the selection in a stepwise, transparent format.
• Output parsers employed to extract lists of selected passage indices from LLM-generated text.
• Adoption of new metrics (e.g., IC@k) in addition to classic IR metrics.

An open-sourced reference implementation is provided by the authors (Lee et al., 9 Jul 2025).

Potential future directions include:

• Scaling mechanisms to larger candidate pools or adapting to very long/hierarchical queries.
• Integrating learned scoring or optimization layers to handle more fine-grained selection objectives.
• Extending the approach to multimodal or cross-lingual settings where information requirements may be spread across sources of different types.

Set-wise passage selection thus establishes a principled and empirically validated alternative to ranking-based retrieval for complex, evidence-intensive language tasks, with demonstrated benefits in both end-to-end question answering performance and evidence coverage.