CoveR: Dense Retrieval with Coverage Awareness

• The paper introduces CoveR, a dense retrieval method that redefines ranking by optimizing for nugget coverage rather than just document relevance.
• It employs a bi-encoder architecture based on ModernBERT with coverage contrastive training and self-distillation guided by sub-question answerability.
• Evaluations indicate CoveR significantly boosts coverage metrics while preserving overall relevance, enhancing retrieval-augmented generation tasks.

CoveR is a dense retrieval method for coverage-aware retrieval in long-form retrieval-augmented generation (RAG). It is designed to retrieve document sets that jointly maximize nugget coverage—the fraction of distinct facts or sub-information needs addressed—while retaining strong document-level relevance. Architecturally, CoveR is a bi-encoder based on a single Transformer encoder, and methodologically it is distinguished by coverage-based contrastive training, coverage self-distillation, and supervision derived from sub-question answerability judgments in the SCOPE dataset (Ju et al., 27 May 2026).

1. Problem formulation and retrieval objective

CoveR is motivated by the claim that long-form RAG poses a coverage-based ranking problem rather than a purely relevance-based one. In this setting, ranking methods must ensure the inclusion of comprehensive relevant nuggets that can later be synthesized into a comprehensive output. Standard dense retrieval objectives generally optimize document-level relevance, whereas CoveR explicitly targets cross-document informational breadth (Ju et al., 27 May 2026).

The paper defines coverage in terms of sub-questions. For a query $q$ with sub-questions $SQ=\{sq_j\}$, a document’s coverage score is

$$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$

where $J(d,sq_j)$ is an answerability judgment on a $0$–$5$ scale, and grades $\ge 4$ are treated as “answerable” (Ju et al., 27 May 2026). This operationalizes coverage as the proportion of the query’s decomposed information needs that are addressed by a candidate passage.

A central distinction in the paper is that standard contrastive training treats any relevant document as “positive” and all else as “negative,” thereby encouraging a tight cluster. CoveR instead samples positives and negatives according to coverage level. This suggests that the method is intended not merely to rank individually relevant passages, but to learn an embedding geometry in which high-coverage documents are preferentially retrieved for queries whose downstream task requires synthesis across multiple nuggets (Ju et al., 27 May 2026).

2. Encoder architecture and inference pipeline

CoveR uses a single Transformer encoder $\mathrm{Enc}_\theta$—ModernBERT-base, with 149 M parameters and 8 K context—for both queries and passages. Queries and documents are encoded independently using task prefixes:

$$E_q=\mathrm{Enc}_\theta(\text{"search\_query: \{q\}"})$$

and

$$E_d=\mathrm{Enc}_\theta(\text{"search\_document: \{d\}"}).$$

The resulting token representations are mean-pooled and $SQ=\{sq_j\}$0-normalized,

$SQ=\{sq_j\}$1

and retrieval scores are computed by cosine similarity,

$SQ=\{sq_j\}$2

Because query and document encodings are independent, the model supports fast approximate nearest-neighbor retrieval with FAISS (Ju et al., 27 May 2026).

This design places CoveR within the dense bi-encoder family rather than cross-encoder reranking. The significance of that choice is practical: the method aims to preserve the indexing and latency properties expected of production-scale retrieval systems while altering the supervision signal so that the embedding space reflects coverage-aware preferences. In the paper’s framing, the novelty lies in the learning objective and training data rather than in introducing a more complex retrieval architecture (Ju et al., 27 May 2026).

3. Coverage contrastive learning and self-distillation

The first training component is Coverage Contrastive, or “CovCon.” Given a candidate pool $SQ=\{sq_j\}$3, CoveR partitions documents by coverage score into high-coverage and low-coverage subsets:

$SQ=\{sq_j\}$4

$SQ=\{sq_j\}$5

Typical settings are $SQ=\{sq_j\}$6 and $SQ=\{sq_j\}$7. A positive document $SQ=\{sq_j\}$8 is sampled from $SQ=\{sq_j\}$9 and negatives $$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$0 from $$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$1. With temperature $$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$2, the loss is

$$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$3

This loss pushes high-coverage documents closer to the query and low-coverage documents further away (Ju et al., 27 May 2026).

The second component is Coverage Self-Distillation, or “CovDistil.” Here the query’s sub-questions act as finer teachers of coverage. Each sub-question is scored against documents in the mini-batch, and a teacher distribution is formed as

$$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$4

The student distribution $$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$5, induced by the query-to-document scores, is aligned to this teacher through Kullback–Leibler divergence:

$$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$6

The final training loss is

$$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$7

In the reported system, $$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$8 and $$\mathrm{Cov}(q,d)=\frac{|\{sq_j\in SQ \mid J(d,sq_j)\ge 4\}|}{|SQ|},$$9 (Ju et al., 27 May 2026).

Methodologically, CoveR thereby differs from heuristic diversification methods such as MMR or MultiQ+RRF/SimSum/RR, which appear only as baselines in evaluation. The paper’s position is that coverage should be encoded during representation learning itself, rather than introduced only as a post-retrieval re-ranking heuristic (Ju et al., 27 May 2026).

4. SCOPE dataset and supervision pipeline

CoveR is trained on SCOPE, a dataset created because no large-scale coverage-labeled data existed. The paper describes SCOPE as approximately 90 K queries and approximately 3 M candidate documents; Table 2 summarizes it as 81 K queries, 12 M sub-questions, 3 M documents, an average of 14.3 nuggets per query, 5.1 positives, and 18.4 negatives (Ju et al., 27 May 2026).

SCOPE is built from Researchy Questions, described as a 90 K-query Bing log collection in which each “researchy” query has a GPT-4 decomposition into sub-questions. Because the original query sometimes misaligns lexically with its sub-questions, the pipeline rewrites each query into a “report-style” prompt. Candidate passages are then obtained by retrieving the top 100 passages from ClueWeb-22 B with BM25. These are reranked with an instruction-tuned Qwen3.0-Reranker, and the top 20 are selected as pseudo-relevant, with any original clicked URLs also included (Ju et al., 27 May 2026).

Answerability judgments are generated with Llama 3 70B using rubric-based prompting. The pipeline produces 24 M $J(d,sq_j)$0 judgments over 1.2 M sub-questions, after which grades $J(d,sq_j)$1 are treated as answerable. The resulting coverage score $J(d,sq_j)$2 supplies the supervision signal for both CovCon sampling and CovDistil teacher construction (Ju et al., 27 May 2026).

The data construction strategy is significant because it converts a long-form generation requirement into a dense retrieval training signal. A plausible implication is that CoveR depends critically on the quality of sub-question decomposition and answerability assessment, since both the positive/negative sampling scheme and the teacher distribution are derived from them.

5. Training procedure and optimization regime

Training proceeds in two stages. Stage 1 is a relevance pre-finetune on MSMARCO passage ranking, using 491 K queries and 8.8 M passages with one positive each and hard negatives from BM25 and CoCondenser. The loss is standard contrastive cross-entropy. The reported configuration is 3 epochs, batch size $J(d,sq_j)$3 queries $J(d,sq_j)$4 positive $J(d,sq_j)$5 negatives$J(d,sq_j)$6 documents in-batch, learning rate $J(d,sq_j)$7, temperature $J(d,sq_j)$8, maximum lengths $J(d,sq_j)$9 and $0$0, and FlashAttention (Ju et al., 27 May 2026).

Stage 2 is coverage training on SCOPE. Each of the 81 K queries is paired with approximately 5 high-coverage positives and 16 low-coverage negatives. Training again runs for 3 epochs with the same batch and optimizer settings, but now uses the combined objective

$0$1

The initialization is the Nomic-AI ModernBERT unsupervised checkpoint (Ju et al., 27 May 2026).

This two-stage regime encodes an explicit separation between generic passage relevance and coverage-aware adaptation. The paper’s ablations indicate that pre-finetuning on MSMARCO preserves relevance on BEIR and improves $0$2-nDCG on CRUX, whereas pre-finetuning on SCOPE-flatten—where each sub-question is treated as an independent query—improves coverage slightly but lags relevance (Ju et al., 27 May 2026). That comparison clarifies that CoveR is not trained to replace conventional relevance supervision, but to augment it with coverage-aware structure.

6. Evaluation, empirical performance, and interpretation

CoveR is evaluated on both relevance and coverage metrics. Relevance is measured with Precision@10 and nDCG@10 against human-annotated document-level relevance, and with average nDCG@10 on the 13-dataset BEIR zero-shot benchmark. Coverage is measured with $0$3-nDCG@10, which novelty-weights discounted gain and penalizes redundant nuggets, and Cov@10, defined as the fraction of unique nuggets covered in the top 10. The nugget benchmarks are NeuCLIR’24 ReportGen (19 queries, 7,049 human nuggets), CRUX-MDS DUC04 (50 queries, 15 nuggets), and CRUX-MDS Multi-News (100 queries, 10 nuggets). Baselines include BM25, SPLADE-v3, Nomic-Embed, Qwen3-Embed 0.6B and 8B, internal “Unsup.” and “Rel.” variants, and heuristic diversity methods such as MMR and MultiQ+RRF/SimSum/RR (Ju et al., 27 May 2026).

On the nugget benchmarks, CoveR improves coverage substantially. Without pre-finetuning, it raises NeuCLIR $0$4-nDCG@10 from 45.8 to 58.4 and Cov@10 from 55.4 to 67.5 relative to “Rel.”, while nDCG@10 rises from 72.3 to 86.4. With MSMARCO pre-finetuning, “CoveR (pFT on MS)” reports NeuCLIR nDCG@10 of 84.0, $0$5-nDCG@10 of 57.7, and Cov@10 of 66.9. On CRUX–DUC04, $0$6-nDCG improves from 51.9 to 57.6 and Cov from 58.4 to 62.7. On Multi-News, $0$7-nDCG improves from 49.5 to 58.4 and Cov from 51.8 to 59.0 (Ju et al., 27 May 2026).

Figure 1 is summarized as showing that CoveR moves the relevance-coverage frontier upward and to the right, with approximately a 10% coverage gain and minimal relevance drop. On BEIR, “CoveR(pFT on MS)” slightly exceeds the “Rel.” baseline in average nDCG@10, at 50.2 versus 50.1, which the paper uses to argue that coverage training does not harm out-of-domain relevance (Ju et al., 27 May 2026).

The ablation studies support the same interpretation. Positives sampled from $0$8 and negatives from $0$9 yield the best trade-off. Setting $5$0 optimizes both BEIR nDCG and CRUX-DUC04 $5$1-nDCG and Cov. Using the reconstructed “report” query rather than the raw Bing query gives a +2% gain on $5$2-nDCG@10. Oracle-multiQ bounds using true nugget sub-questions raise NeuCLIR $5$3-nDCG to as high as 72%, indicating additional headroom for explicit multi-step or set-selection methods (Ju et al., 27 May 2026).

A qualitative example in the paper concerns the query “Why are gas prices spiking?” Standard relevance-trained retrieval tends to return multiple articles centered on “OPEC cuts,” yielding Cov@3 of approximately 0.3. CoveR instead retrieves an overview of global oil markets, one document on U.S. supply chain constraints, and one on regulatory or tax factors, producing Cov@3 of approximately 0.8 (Ju et al., 27 May 2026). The paper presents this as evidence that coverage-aware retrieval changes not just the score ordering of relevant documents, but the semantic spread of the retrieved set.

7. Position within long-form RAG research

CoveR occupies a specific position in long-form RAG: it is a dense retriever optimized for set-level informational coverage while maintaining the operational advantages of bi-encoder retrieval. Its central contribution is to turn coverage from a downstream evaluation criterion into a training objective, using synthetic but structured supervision based on sub-question answerability (Ju et al., 27 May 2026).

Within that framing, the method addresses a common misconception in long-form retrieval: that coverage can be handled adequately by post-hoc diversification alone. CoveR’s results argue for a different view, namely that coverage-aware behavior can be induced directly in the embedding space through coverage-based contrastive sampling and self-distillation. The reported BEIR results further constrain a second concern—that optimizing for coverage necessarily harms conventional relevance—by showing parity or slight improvement in average nDCG@10 after MSMARCO pre-finetuning and SCOPE coverage training (Ju et al., 27 May 2026).

At the same time, the oracle-multiQ results imply that CoveR does not saturate the problem. The paper explicitly identifies headroom for explicit multi-step or set-selection methods. This suggests that CoveR is best understood as a representation-learning foundation for coverage-aware retrieval rather than a final solution to coverage-based ranking in long-form RAG (Ju et al., 27 May 2026).