Poly-PRAG: Latent Routing for Parametric RAG

• Poly-PRAG is a framework that integrates latent routing of a pool of LoRA adapters to enable efficient, parametric retrieval-augmented generation.
• It employs a latent routing mechanism and sparse expert selection to reduce inference overhead and mitigate overfitting in multi-hop, knowledge-intensive tasks.
• Empirical results demonstrate state-of-the-art performance with significant improvements over traditional RAG and prior PRAG models, ensuring scalable knowledge injection.

Poly-PRAG designates the "Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapters" framework, an advanced paradigm for integrating structured and parametric knowledge into LLMs. Poly-PRAG builds on and addresses core limitations of earlier retrieval-augmented generation (RAG) and parametric RAG (PRAG) frameworks by employing latent routing mechanisms, sparse expert selection, and multi-task learning to achieve both high parametric recall and efficient inference. It has demonstrated state-of-the-art performance on multi-hop, knowledge-intensive tasks and enables scalable, resource-efficient knowledge injection for neural LLMs (Su et al., 21 Nov 2025).

1. Motivation and Conceptual Background

Traditional RAG systems concatenate $k$ retrieved passages as context to the LLM prompt. This approach leads to substantial input length increases—if each passage contains $|d|$ tokens and the query has $|q|$, the prompt is $|q| + k|d|$ tokens—escalating inference costs quadratically with sequence length. Furthermore, context-bloating degrades answer quality as LLMs dilute attention across both relevant and irrelevant evidence.

Parametric RAG (PRAG) attempts to incorporate passages directly into the model's parameters, commonly through attaching a dedicated LoRA (Low-Rank Adaptation) adapter for each document. While this increases parametric recall and shortens input lengths, one-to-one PRAG has severe scalability and overfitting drawbacks:

• Each adapter sees limited passage-specific data, causing overfitting and poor generalization.
• For $|T|$ documents, $|T|$ adapters must be stored and dynamically loaded/unloaded at inference—an $\mathcal{O}(|T|p)$ storage and $\mathcal{O}(|T|)$ loading operation per query, with $p$ the LoRA parameter count.

Poly-PRAG introduces a shared, sparsely-activated latent expert mechanism: a set of $m \ll |T|$ LoRA adapters, where document encoding and retrieval are mediated by a routing network. This both pools document supervision across adapters (mitigating overfitting) and amortizes inference overhead (allowing a fixed parameter set to serve all downstream queries).

2. Model Structure: Latent Routing and Mixture-of-Experts Parameterization

Poly-PRAG parameterizes each passage via a mixture over a latent pool of LoRA experts:

1. Let $|d|$0 denote a frozen base-model weight matrix (e.g., from a transformer layer).
2. $|d|$1 LoRA experts are defined as low-rank residuals: $|d|$2 for $|d|$3, $|d|$4.
3. For passage $|d|$5, an encoder outputs representation $|d|$6.
4. A routing network produces logits $|d|$7, which define a sparse vector $|d|$8:
   • During training: Gumbel-Softmax sampling is used to yield sparse ($|d|$9-constrained, $|q|$0-hot) weights.
   • At inference: Top-$|q|$1 logits are softmaxed to produce $|q|$2 over experts.
5. The adapted parameter for $|q|$3 is $|q|$4.

A one-hot “task ID” for each passage can be concatenated to routing MLP inputs, facilitating passage-specific specialization.

3. Multi-Task Training and Offline Encoding

Poly-PRAG treats passage encoding as a multi-task objective:

• Each document $|q|$5 is a “task”; its training set is $|q|$6, with $|q|$7 paraphrases and $|q|$8 QA pairs.
• The network is optimized via cross-entropy over next-token prediction:

$|q|$9

where $|q| + k|d|$0 denotes the combined effect of the routed LoRA weights.

Model updates affect the LoRA parameters $|q| + k|d|$1, the router parameters $|q| + k|d|$2, and optionally the passage encoder, while keeping the base LLM fixed.

4. Online Retrieval and Inference Procedure

Inference in Poly-PRAG involves the following sequence:

1. Retrieve top-$|q| + k|d|$3 relevant passages $|q| + k|d|$4 for a given query $|q| + k|d|$5.
2. For each retrieved passage $|q| + k|d|$6, compute routing logits $|q| + k|d|$7 and construct $|q| + k|d|$8—a sparse weighting over $|q| + k|d|$9 experts.
3. Aggregate sparse gate activations across the $|T|$0 passages:
   • For each expert $|T|$1, sum $|T|$2 over retrieved documents.
4. Merged adapters are injected into the LLM by updating the relevant layers to $|T|$3.
5. The adapted LLM generates $|T|$4.

The entire Poly-PRAG run requires loading the $|T|$5 experts to GPU memory once, with only lightweight routing logic executed per query. For comparison, standard RAG requires sequence length scaling as $|T|$6, and PRAG/DyPRAG require $|T|$7 adapter loads per query, whereas Poly-PRAG incurs $|T|$8 fixed adds and routing.

Key pseudocode fragment:

$|T|$8 (Su et al., 21 Nov 2025)

5. Empirical Performance and Ablation

Poly-PRAG achieves state-of-the-art (SOTA) results across widely used multi-hop and knowledge-intensive benchmarks:

Base LLM	Method	Avg F1 (%)
LLaMA3-2.1B	Vanilla	22.82
	Standard RAG	27.45
	PRAG	26.99
	DyPRAG	28.80
	Poly-PRAG	32.68†
Qwen2.5-1.5B	Vanilla	23.76
	Standard RAG	23.33
	PRAG	28.18
	DyPRAG	26.19
	Poly-PRAG	30.26†
LLaMA3-8B	Vanilla	33.02
	Standard RAG	31.23
	PRAG	41.59
	DyPRAG	37.16
	Poly-PRAG	42.68†

†: Statistically significant with $|T|$9 (Su et al., 21 Nov 2025).

Ablations further demonstrate Poly-PRAG’s efficiency:

• LoRA rank: For HotpotQA, Poly-PRAG achieves competitive F1 (15.18%) at $|T|$0 with only 84MB storage, versus PRAG’s 12.04% F1 and 8.1GB storage.
• Number of experts ($|T|$1): Optimal $|T|$2 is domain-specific—typically $|T|$3 for complex QA, with diminishing returns beyond $|T|$4 in simpler domains.
• Injection layer: Adapters injected into both FFN and self-attention (“Cross”) yield the best results, indicating that both representational and relational aspects benefit from low-rank parametric adaptation.

6. Extensions, Domain Adaptations, and Related Poly-PRAG Variants

The core Poly-PRAG principle—routing queries and passages to a small, fixed set of latent experts—generalizes beyond text QA to structured scientific domains and poly-perspective retrieval.

In polymer science, structured Poly-PRAG pipelines utilize both dense semantic retrievers (VectorRAG) and knowledge-graph-based multi-hop retrieval (GraphRAG). The resulting hybrid system supports both broad semantic context (via paragraph embeddings; e.g., Qwen3-Embedding-4B, 3584d) and precise, citation-grounded reasoning (via entity–relation graphs with canonicalization and multi-hop traversal). Evaluation on polymer knowledge reveals that:

• GraphRAG outperforms semantic vector RAG in precision, recall (Recall@8=0.938), and interpretability by structuring evidence as entity graphs, while maintaining citation reliability and low latency.
• VectorRAG delivers superior narrative context but reduced discriminative accuracy (Gupta et al., 18 Feb 2026).

For medical domains, PolyRAG extends Poly-PRAG with “polyview” retrieval: candidate passages are each evaluated along multiple axes (relevance, utility, supplement, authoritativeness, timeliness, and composability). This multi-objective scoring is then used to construct a top-$|T|$5 supporting set that maximizes both topical coverage and answer veracity. On the PolyEVAL benchmark, incorporating these perspectives yields substantial improvements in result accuracy (e.g., a +10–15% boost in correct answer ratio over best single-view RAG baselines, with correct rate improved to 71.6% for CARE domain and Hit@3/NDCG@3 also leading all tested baselines) (Gan et al., 21 Apr 2025).

7. Limitations and Future Research Directions

Poly-PRAG compresses the parameter count by an order of $|T|$6, thereby reducing storage and overfitting and facilitating rapid, resource-lean online inference. Sparse expert activation ($|T|$7) ensures that per-query computational and memory overhead is negligible.

Emerging challenges include:

• Zero-shot routing: Enabling the routing function and expert allocation to generalize to unseen documents or out-of-distribution passages without retraining adapters.
• Dynamic expert pool: Adapting to evolving corpora by growing or pruning adapters on demand as new knowledge arrives.
• Semantic specialization: Encouraging latent experts to specialize topically or by semantic clusters using entropy or KL-regularized constraints on routing distributions.
• Cross-domain fusion: Integrating structured (graph-based) retrieval and dense retrieval into a unified Poly-PRAG pipeline for optimal factual grounding and context richness.

A plausible implication is that future Poly-PRAG variants will blend dense, graph, and multi-perspective retrieval architectures, leveraging the full spectrum of both learned and symbolic representations for domain-adaptive RAG.

• "Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapters" (Su et al., 21 Nov 2025)
• "Retrieval Augmented Generation of Literature-derived Polymer Knowledge: The Example of a Biodegradable Polymer Expert System" (Gupta et al., 18 Feb 2026)
• "POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applications" (Gan et al., 21 Apr 2025)