Mixture-of-Retrievers (MoR)

• Mixture-of-Retrievers (MoR) is a framework that integrates diverse retriever types (sparse, dense, human-in-the-loop) using adaptive, zero-shot weighting.
• It leverages both pre-retrieval signals (geometric diagnostics) and post-retrieval metrics (e.g., Moran’s I) to dynamically balance retriever contributions per query.
• Empirical results demonstrate MoR’s superior performance in NDCG and MRR across scientific and graph-based benchmarks compared to standalone models.

A Mixture-of-Retrievers (MoR) is a retrieval architecture designed to leverage the complementary strengths of heterogeneous retrieval models—such as sparse, dense, and human-in-the-loop retrievers—by dynamically integrating their outputs to optimize retrieval quality on a per-query basis. MoR methodologies aim to address the intrinsic diversity of information needs and the uneven performance of any single retriever type across query distributions by using zero-shot, adaptive weighting or score-level fusion without requiring query-label supervision or retriever fine-tuning for each new domain (Kalra et al., 18 Jun 2025, Cai et al., 2023, Lei et al., 27 Feb 2025).

1. Formal Foundations and Variants

The core principle underlying MoR is to maintain a fixed pool of $N$ retrievers $\mathcal{L}_R=\{R_1, \ldots, R_N\}$, each producing a relevance score $s_i(q,d) \in \mathbb{R}$ for candidate document $d$ in response to query $q$. MoR assigns a query-specific, nonnegative weight $w_i(q)$ to each retriever and computes an aggregate score: $\tilde{s}(q, d) = \sum_{i=1}^N w_i(q) s_i(q, d)$ Documents are ranked by descending $\tilde{s}$.

A critical innovation is the zero-shot weighting of component retrievers. Two main signal classes are used in contemporary MoR designs (Kalra et al., 18 Jun 2025):

• Pre-retrieval signal ($V_{\mathrm{pre}}$): Measures the geometric “familiarity” of a query’s embedding with respect to the topology of a retriever’s document embeddings, defined via clustering over embedding centroids.
• Post-retrieval signals: Include metrics such as Moran’s I (spatial autocorrelation over top-$k$ retrieved results) and average $\mathcal{L}_R=\{R_1, \ldots, R_N\}$0 computed over these retrieved documents.

Two canonical weighting variants are defined:

• MoR-pre: $\mathcal{L}_R=\{R_1, \ldots, R_N\}$1
• MoR-post: Combines all trustworthiness signals as $\mathcal{L}_R=\{R_1, \ldots, R_N\}$2, with $\mathcal{L}_R=\{R_1, \ldots, R_N\}$3 tunable, then normalized via $\mathcal{L}_R=\{R_1, \ldots, R_N\}$4 projection.

In graph-structured retrieval, MoR has been extended to pair textual and structural retrievers through multi-stage planning, traversal, and trajectory-aware reranking (Lei et al., 27 Feb 2025), denoted as $\mathcal{L}_R=\{R_1, \ldots, R_N\}$5 formed from a sum of planning graphs, mixture-of-retriever outputs, and path-embedded reranking.

2. Components and System Design

MoR repositories typically draw upon the following retriever families (Kalra et al., 18 Jun 2025, Cai et al., 2023):

Retriever Type	Example Models	Distinctive Capability
Sparse (Lexical)	BM25, SPLADE	Exact term overlap
Dense (Semantic)	SimCSE, Contriever, DPR, GTR, MPNet, ANCE, TAS-B	Semantic/syntactic similarity
Local (Token-level)	ColBERT	Fine-grained local matches
Human-in-the-loop	Simulated domain experts	Domain-specialist and non-oracle
Structural (in KGs)	Structural traversal	Graph path-based inductive cues

BM25 offers zero-shot lexical matching without trainable parameters, while dense models exploit pre-trained transformer encodings from various objectives. Human retrievers are operationalized as expert agents delivering oracle or partial rankings in specific domains, their outputs incorporated with learned or trust-derived weights.

Deep fusion techniques further extend each retriever across multiple granularity levels (e.g., query-doc, sub-query-prop), amplifying coverage by indexing at multiple resolutions.

3. Training, Weighting, and Inference Mechanisms

MoR approaches achieve adaptive integration by learning to assign or compute weights for each retriever per-query, frequently without access to explicit query–label pairs:

• Zero-shot MoR: All trustworthiness signals and geometric diagnostics are computed from the retriever’s corpus and query alone. No task or domain fine-tuning is required (Kalra et al., 18 Jun 2025).
• Competitive learning (e.g., CAME): Employs a two-stage training algorithm: standardized learning where all experts are trained on the same data, and specialized learning where each expert is updated proportionally to its relative success on each instance, enforcing expert specialization (Cai et al., 2023).
• Expert selection and fusion: At inference, per-query retriever scores are normalized (e.g., min-max scaling to $\mathcal{L}_R=\{R_1, \ldots, R_N\}$6), with top candidates merged from all retrievers and re-ranked by the MoR composite score. For production deployment, early rejection or pruning heuristics leverage small $\mathcal{L}_R=\{R_1, \ldots, R_N\}$7 weights to ignore marginal retrievers on a given query.

In knowledge-graph settings, plans generated by LLMs lead to joint traversal of structural and textual modalities, with evidence fused at each step and final results reranked via an MLP that embeds the traversal’s structural, textual, and routing fingerprints (Lei et al., 27 Feb 2025).

4. Empirical Performance and Comparative Results

Quantitative evaluations demonstrate that MoR models consistently outperform all standalone retrievers and even larger single-retriever LLMs on diverse domains without retriever-specific fine-tuning (Kalra et al., 18 Jun 2025, Cai et al., 2023, Lei et al., 27 Feb 2025).

• On scientific and medical benchmarks (NFCorpus, SciDocs, SciFact, SciQ), MoR-post achieves average NDCG@20 scores of 58.7, exceeding both the best unsupervised (MPNet: 52.3) and best supervised (TAS-B: 51.3) dense retrievers, as well as 7B-parameter LLM retrievers (GritLM-7B: 56.5).
• In RAG exact match (EM@1), MoR-post maintains a +5 to +6 point margin over sparse and dense baselines.
• For simulated human expert mixtures, post-signal MoR produces an average NDCG@20 of 91.7 compared to 57.7 with equal human-only weights, a +58.9% relative lift (Kalra et al., 18 Jun 2025).
• In text-rich graph knowledge bases, MoR achieves the highest MRR (58.77), Recall@20 (66.14), and top-2 Hit@1 scores on the STaRK benchmark, outperforming pure textual, pure structural, and prior hybrid baselines by significant margins (Lei et al., 27 Feb 2025).
• Competitive learning MoRs (like CAME) yield state-of-the-art MRR@10 on MS MARCO, TREC DL, and Natural Questions, surpassing both strong single retrievers and prior ensembling approaches (Cai et al., 2023).

Ablation studies validate that each trust signal and fusion layer contributes materially; deep fusion and trajectory-aware reranking yield measurable improvements. In fast-inference regimes, thresholding retriever weights enables MoR to retain >95% of its full performance using only a minority of the retrievers.

5. Interpretability, Modality Integration, and Human Collaboration

MoR architectures provide natural interpretability by exposing per-retriever weighting and by explicitly modeling modality-wise and granularity-wise contributions. In the context of graph knowledge bases, joint planning-driven traversal of textual and structural experts, with path-wise fingerprinting, allows the system to explain both which paths (semantically or topologically) underlie a result and the mode of retrieval (textual vs. structural) at each inference step (Lei et al., 27 Feb 2025).

MoR supports the seamless inclusion of nonparametric retrievers, including human and domain-expert agents, by extending the trustworthiness weighting and geometric signal mechanism. The framework robustly identifies the relevant domain expert for in-domain queries and can blend oracle, heuristic, or random advice with machine-based retrievers to improve both retrieval recall and downstream generative quality (Kalra et al., 18 Jun 2025). This property is particularly salient for collaborative retrieval in domains with incomplete or rapid-evolving inductive knowledge.

6. Applications, Limitations, and Future Directions

MoR is directly applicable in retrieval-augmented generation pipelines, open-domain and domain-specialist search, knowledge-graph completion, and settings requiring collaboration between machine and human experts. It enables plug-and-play composition with LLM readers and can be pruned or thresholded for efficient production deployment with minimal loss of quality.

Limitations of current approaches include the computational overhead of multi-retriever fusion, dependence on well-chosen geometric or trust signals, and the difficulty of automatically identifying highly complementary component retrievers ex ante. Further, existing MoR variants typically assume independence among retriever outputs or fixed architectures for the weighting function; future developments may include full attention-based or graph neural fusion across retrievers, adaptive retriever pool selection, or the synthesis of trust signals from generative and retrieval pathways.

A plausible implication is that, as retrieval-augmented generation becomes ubiquitous in LLM systems, MoR methods may become an essential backbone for robust, general-purpose, and domain-adaptive search pipelines across modalities and user populations. Continued research will likely explore more general trust signal classes, reinforcement learning for fusion, and broader integration with human agents.

7. Related Work and Distinctions

Early ensemble retrieval approaches simply merged retrieval results via rank aggregation or summed scores but did not adapt per-query or enforce specialization among retriever types. MoR distinguishes itself from static ensembles through its query-adaptive, trustworthiness-weighted combinations (Kalra et al., 18 Jun 2025). Competitive learning MoR models (e.g., CAME) differ by using end-to-end learning to route signal and specialization dynamically among expert retrievers (Cai et al., 2023).

In knowledge-graph retrieval, MoR stands apart from hybrid or direct aggregation models by structuring the interaction as a planned, multi-stage mixture, with explicit organizing (reranking) using structural, textual, and routing fingerprints, yielding superior performance for complex, multi-hop queries (Lei et al., 27 Feb 2025).

These developments collectively establish Mixture-of-Retrievers as a central paradigm for adaptive, high-recall, and interpretable retrieval in both text and multi-modal knowledge-rich domains.