PassageRank: A Passage Ranking Framework

• PassageRank is a framework that utilizes passage-level ranking functions and feature integration to accurately assess document relevance in information retrieval tasks.
• It employs a two-stage process where a learned passage-ranking model first scores text segments, and the top passage signals are injected into the overall document ranking process.
• Modern implementations integrate LLMs and dense embeddings (as in PE-Rank) to achieve improved efficiency and effectiveness while reducing computational overhead.

PassageRank refers to a set of techniques and frameworks that leverage the identification and ranking of passages within documents to improve information retrieval effectiveness, particularly in contexts such as ad-hoc document ranking and open-domain question answering. PassageRank systems typically operate in two stages: (1) learning and applying a passage-level ranking function to identify the most relevant segments of text for a given query, and (2) injecting these passage-level signals into the downstream document (or higher-level unit) ranking function. This paradigm addresses the shortcomings of whole-document relevance assignment, especially in domains—such as TREC’s assessment regime—where the presence of a brief yet highly relevant passage can determine overall document relevance (Sheetrit et al., 2019).

1. Core Methodologies in PassageRank

The PassageRank family centers on first constructing a passage–ranking function $f_p(q, p)$ that scores candidate passages $p$ for a query $q$ based on a learned combination of features. Typical feature sets ($\phi_p(q,p) \in \mathbb{R}^m$) include LLM (LM) similarities, semantic similarities (e.g., word embeddings, ESA), lexical overlap, positional attributes, entropy measures, and stopword statistics.

$f_p(q,p) = w_p^\top \phi_p(q,p)$

For ranking model training, both pairwise SVM-rank and LambdaMART listwise approaches are used, with objectives tailored to maximize NDCG@10 or minimize pairwise ranking loss over labeled passage pairs (Sheetrit et al., 2019).

The document-level ranking function $f_d(q,D)$ is then constructed by aggregating passage-level information—most effectively, by incorporating the features $\phi_p(q, g_{\max}(D))$ of the single top-ranked passage $g_{\max}(D)$ into the overall document feature vector $\psi_d(q,D)$. The final ranking score is computed as:

$f_d(q,D) = w_d^\top \psi_d(q,D)$

where $$\psi_d(q,D) = \phi_d(q,D) \oplus \phi_p(q,g_{\max}(D))$$, and $\oplus$ denotes concatenation (Sheetrit et al., 2019). Empirically, this “max-passage” approach outperforms document-only or passage-statistics-driven variants.

2. Feature Engineering and Representation Learning

Passage-based ranking models rely on a rich, multi-granular feature set for $\phi_p(q,p)$, including but not limited to:

• Surface-level signals: query–passage unigrams and bigrams, passage length, stopword fractions, and match overlap.
• Semantic features: ESA similarities, Word2Vec centroid cosine, entity overlap via TagMe, SynonymsOverlap.
• Contextual statistics: relative position within document, similarity to adjacent passages, entropy, and LM similarity normalization.

In modern neural ranking architectures, representation learning evolves using two main approaches:

• Contextualized encoder extraction: Extracting fine-grained, token or sentence-level embeddings from pretrained contextual models (e.g., BERT), with further aggregation (mean, attention, dynamic memory) to derive passage representations (Leonhardt et al., 2021).
• Dense retrieval and passage embedding: Compressing a passage into a single vector via a bi-encoder or specialized sequence encoder, which is then mapped into the downstream ranking model (notably in PE-Rank, discussed below) (Liu et al., 2024).

3. Integration with Modern LLM and Embedding Architectures

Recent developments exploit LLMs for passage re-ranking, either by direct prompt-based ranking or via integration of dense passage embeddings as compressed context:

Listwise LLM-Based Reranking and PE-Rank

PE-Rank introduces an architecture that leverages bi-encoder-derived passage embeddings. Each passage embedding $e_{p_i}$ is mapped via a two-layer MLP $E_m$ into the LLM’s token-embedding space. These mapped embeddings are treated as special tokens $\langle p_i \rangle$, thus compressing the reranking window from $n \cdot L_p$ to $n$ passages per query, resulting in significant input length and latency reductions (Liu et al., 2024). Sequence generation is controlled via dynamic-constrained decoding, where at each step the LLM generates only among the remaining candidate passage tokens, guaranteeing well-formed listwise ranking outputs.

The loss function used is a sequential listwise ListMLE loss, with auxiliary objectives including content-aware loss (full-text grounding) and KL distillation to match predictions from full-text versus embedding-only inputs:

$$\mathcal{L} = \mathcal{L}_{rank} + \mathcal{L}_{content} + \alpha \cdot \mathcal{L}_{KL}$$

with $\alpha=0.2$ (Liu et al., 2024).

PassageRank and Feature Augmentation

Traditional PassageRank models inject passage-derived features directly into document ranking rather than relying on brittle interpolations or statistical aggregations. This approach avoids metric-divergence and allows the document ranker to exploit the same information that makes a passage salient to the query (Sheetrit et al., 2019).

4. Empirical Effectiveness and Efficiency

Across a range of corpora and retrieval tasks, PassageRank frameworks demonstrate large, statistically significant improvements over strong baselines. For example:

Dataset	Baseline MAP	PassageRank MAP	P@10 Baseline	P@10 PassageRank
ROBUST	0.254	0.290	0.433	0.480
WT10G	0.195	0.235	0.290	0.381
GOV2	0.292	0.350	0.534	0.656
ClueWeb09B	0.187	0.246	0.339	0.452

All differences are statistically significant ($p < .05$) (Sheetrit et al., 2019). Furthermore, sentence-level and embedding-based compressions permit PassageRank to be applied at scales and with candidate windows (e.g., $n=100$ passages) previously infeasible for LLMs (Liu et al., 2024).

PE-Rank achieves over $4\times$ speedup and requires only $\sim 15\%$ of the prefill tokens and $20\%$ of the generated tokens compared to uncompressed LLM rerankers, losing less than $2$ nDCG@10 points in effectiveness (Liu et al., 2024).

5. Connections to Related Models and Research Directions

PassageRank operationalizes the observation that TREC and similar evaluation regimes award document relevance for even short, highly relevant passages. This motivates direct comparison with:

• Global document LTR: PassageRank’s max-passage approach consistently outperforms both LM-only and document-level LTR by focusing on segments most aligned with query intent.
• Neural memory models: Sentence-level reasoning with dynamic memory networks (DMNs) captures evidence spread across multiple sentences, outperforming global [CLS]-based strategies and allowing for lighter-weight adaptation (e.g., DMN-only fine-tuning) (Leonhardt et al., 2021).
• LLM listwise reranking: Embedding-based context compression, as in PE-Rank, enables high-efficiency listwise LLM reranking while maintaining competitive effectiveness. Dynamic constrained decoding ensures output validity and reduces inference cost (Liu et al., 2024).

Notably, methods integrating passage-based signals (including PaRaDe’s demonstration selection and TWOLAR’s LLM distillation (Drozdov et al., 2023, Baldelli et al., 2024)) extend the PassageRank philosophy to few-shot, LLM-augmented, or teacher-student settings.

6. Limitations and Open Challenges

While empirically effective, PassageRank methodologies exhibit limitations:

• Architectural complexity can rise, as in PE-Rank, requiring separate embedding mappers, frozen retrievers, and LLM fine-tuning (Liu et al., 2024).
• Current methods are not fully plug-and-play; swapping the passage encoder or embedding model requires retraining or realignment.
• Performance can be sensitive to initial candidate ordering and feature selection. Methods incorporating only a single top passage (as in JPDₛ) are most effective, but handling multi-passage evidence may require more sophisticated aggregation.

A plausible implication is that future research may focus on universal embedding-to-token alignment, smarter initial retrieval cascades, or hierarchical/multimodal passage representations.

7. Historical Context and Impact

The PassageRank paradigm established that leveraging passage-level signals—whether engineered features, contextualized embeddings, or LLM-derived representations—supports materially better document ranking in real-world IR evaluations (TREC, ad hoc retrieval, open-domain QA). From mid-2010s feature-based models (Sheetrit et al., 2019) to recent LLM and dense embedding approaches (Liu et al., 2024), PassageRank remains foundational to neural IR research, informing the architecture of state-of-the-art systems across supervised, zero-shot, and resource-constrained retrieval settings.