ExpRAG: Experience-Based RAG Framework

• ExpRAG is a retrieval-augmented generation framework that uses structured experiences to ground LLM inference in domain-specific contexts.
• It employs a multi-stage retrieval process—coarse selection followed by fine-grained retrieval—to extract highly relevant case-based data from large record systems like EHRs.
• Empirical evaluations demonstrate that ExpRAG significantly improves accuracy in clinical QA and adaptive learning tasks compared to text-only retrieval methods.

ExpRAG (“Experience Retrieval-Augmentation”) designates a class of retrieval-augmented generation (RAG) frameworks that leverage stored “experiences”—whether in the form of structured electronic health records (EHRs), task/solution/feedback histories, or general external knowledge—to enhance LLMs’ (LLMs) performance on specialized, knowledge-intensive, and adaptive tasks. ExpRAG approaches share the common principle of augmenting LLM inference with highly relevant case- or experience-based context, retrieved via efficient and often multi-stage mechanisms, and have been instantiated in multiple domains, including clinical QA and agentic test-time learning (Ou et al., 23 Mar 2025, Wei et al., 25 Nov 2025). The term ExpRAG also appears in the literature as a synonym for “ExpertRAG” (Gumaan, 23 Mar 2025), denoting RAG systems with selective mixture-of-experts routing, although this article treats these as conceptually distinct.

1. Architectural Foundations

ExpRAG frameworks instantiate retrieval-augmented generation with an explicit focus on retrieving prior experiences—structured or unstructured records of prior cases, actions, or solutions—as the main source of non-parametric grounding for LLM reasoning.

• In EHR-based clinical QA (Ou et al., 23 Mar 2025), ExpRAG operates as a two-stage coarse-to-fine pipeline. Given a patient $p$, a query $q$, and a large patient corpus $\mathcal{D}$ of discharge summaries, ExpRAG:

  1. Uses structured EHR codes (diagnosis, medication, procedure) to rank and select a shortlist $\mathcal{D}'$ of clinically similar cases. Pairwise similarity $\tau(p,p')$ between patient $p$ and $p'$ is computed as a weighted sum of Jaccard similarities over code sets:

     $$\tau(p, p') = \lambda_1 \tau_{\text{Diag}}(p, p') + \lambda_2 \tau_{\text{Med}}(p, p') + \lambda_3 \tau_{\text{Proc}}(p, p')$$

  with $\lambda_1 + \lambda_2 + \lambda_3 = 1$. 2. Applies fine-grained retrievers (e.g., BM25+, FLARE, Contriever) to extract paragraphs $d_*$ from $\mathcal{D}'$ maximizing relevance to $q$. 3. Constructs a composite input of [background of $p$] + [$d_*$] + [$q$] for LLM downstream processing.

• In agentic and test-time learning settings (Wei et al., 25 Nov 2025), ExpRAG (here, “Experience Retrieval and Aggregation”) is formalized as a baseline module for retrieving and integrating prior (input, output, feedback) tuples. Each memory entry $m_i$ is encoded as $S(x_i, \hat{y}_i, f_i)$. Embedding-based retrieval (cosine similarity in encoder space) surfaces the top-$k$ prior experiences relevant to the new task $x_t$, yielding a prompt $C_t = [x_t; R_t]$ for LLM generation.

2. Retrieval and Memory Mechanisms

The canonical ExpRAG retrieval process comprises:

• Coarse Selection: Filters the large experience/memory base by structured, task-specific indices (e.g., ICD/NDC codes for EHRs; keywords, embeddings for general experience).
• Fine Retrieval: Within the candidate set, applies textual/semantic retrievers:
  • BM25 and variants for lexical matching.
  • Hybrid approaches (embedding+keyword).
  • Dense retrievers such as Contriever (requires clinical/domain fine-tuning for best results).
  • Context-aware methods (FLARE, auto-merging).
• Integration and Update:
  • Retrieved experiences are concatenated with the current query or context, enabling in-context learning in the LLM.
  • In agentic frameworks (Wei et al., 25 Nov 2025), memory is typically append-only: after generating $\hat{y}_t$ and observing $f_t$, the tuple $m_t = S(x_t, \hat{y}_t, f_t)$ is appended to $M_{t+1} = M_t \cup \{ m_t \}$.

3. Evaluation Domains and Datasets

Clinical QA: DischargeQA (Ou et al., 23 Mar 2025)

• DischargeQA consists of 1,280 questions over diagnosis, medication, and instruction tasks, constructed from MIMIC-IV discharge summaries.
  • Query types: Diagnosis inference (436 multi-select), Medication inference (444 multi-select), Instruction inference (400 single-select).
  • Distractor options are generated for non-triviality and to ensure realistic ambiguity.
• ExpRAG provides significant gains relative to text-only retrieval. For example, using GPT-4o, EHR-based ExpRAG achieves Instruction Accuracy 91.3%, Diagnosis Accuracy/F1 21.3/0.530, and Medication Accuracy/F1 9.68/0.638 versus lower scores for text-based ranker baselines.

Streaming Memory and LLM Agents: Evo-Memory (Wei et al., 25 Nov 2025)

• Evo-Memory benchmarks incremental self-evolving memory across ten multi-turn and single-turn reasoning datasets.
• ExpRAG serves as a baseline, demonstrating substantial improvements:
  • GPQA-Diamond: exact match 0.55 → ExpRAG 0.70
  • ToolBench API accuracy: 0.76/0.62 → ExpRAG 0.88/0.72
  • AlfWorld (multi-turn): Success 0.28 → ExpRAG 0.59, Progress 0.60 → ExpRAG 0.79

4. Comparative Performance and Ablations

ExpRAG consistently outperforms direct-ask (LLM-only) and text-only retrieval pipelines by leveraging structured experience signals.

• In clinical QA, average relative accuracy improvement over text-based rankers is 5.2% (Ou et al., 23 Mar 2025).
• In Evo-Memory streaming agent contexts, ExpRAG surpasses ReAct (no memory), Mem0 (adaptive store), and other memory baselines in both exact-match and API-based tasks.
• Ablation studies confirm that:
  • Complementary weighting (excluding the task-relevant code and upweighting orthogonal code types) can produce superior F1 scores on diagnosis/medication QA.
  • Retrieval set size ($k$) and retrieval method significantly affect performance, with excessive $k$ potentially introducing noisy context.
  • In multi-turn environments, ExpRAG substantially narrows the gap to more complex adaptive-memory methods, though it lacks iterative memory refinement and error-correction.

Baseline	Exact Match (EM)	API (avg)
ReAct (no memory)	0.37	0.57
Mem0 (adaptive store)	0.59	0.61
Dynamic Cheatsheet	0.56	0.57
ExpRAG (baseline)	0.60	0.73

In multi-turn agent benchmarks (Success/Progress), ExpRAG also demonstrates superior performance compared to Amem, Mem0, and AWM modules (Wei et al., 25 Nov 2025).

5. Practical Implications, Limitations, and Future Directions

• Implications: ExpRAG’s structured experience-driven retrieval mirrors domain expert workflows (e.g., clinicians reviewing analogous patient cases), enabling more contextually grounded and factually reliable LLM outputs in domains where case-based reasoning is critical (Ou et al., 23 Mar 2025). The approach is lightweight, scalable, and flexible, directly benefiting applications with large, heterogeneous memory stores (such as institutional EHRs or lifelong learning agents).
• Limitations:
  • Current implementations in clinical settings restrict the ranker to three code types (diagnosis, medications, procedures), omitting lower-level or free-text signals.
  • Most evaluations leverage multiple-choice or classification tasks; extension to fully generative settings is pending.
  • Off-the-shelf retrievers may lack domain specialization, and no active pruning or memory summarization is performed in agentic versions.
• Future Directions:
  • Incorporation of additional modalities (labs, imaging, richer notes) in clinical retrieval.
  • Extension to generative QA and open-ended reasoning.
  • Development of adaptive memory-management strategies and joint learning of retrieval and generation modules.
  • Empirical testing of case-augmented frameworks in real-world, generative, and long-horizon conditions.
  • Fine-tuning of dense retrievers and further integration with continual learning pipelines.

6. Relation to ExpertRAG and Other RAG Variants

“ExpRAG” is sometimes used interchangeably with “ExpertRAG,” notably in the theoretical literature (Gumaan, 23 Mar 2025). However, ExpertRAG refers specifically to models combining RAG with mixture-of-experts (MoE) architectures and dynamic retrieval gating. In this context, the architecture introduces a probabilistic latent-variable formulation in which retrieval and expert selection are governed by learned gates, optimizing both factuality (by selective invocation of retrieval) and compute cost (by exploiting MoE sparsity). This paradigm is distinct from the experience retrieval focus of ExpRAG, though related in spirit via the fusion of parametric (model weights) and non-parametric (retrieved experience) knowledge. Comparative analyses position ExpertRAG as interpolating between always-on RAG and pure MoE models, yielding a favorable balance of accuracy, efficiency, and adaptability (Gumaan, 23 Mar 2025).

• (Ou et al., 23 Mar 2025) "Experience Retrieval-Augmentation with Electronic Health Records Enables Accurate Discharge QA"
• (Wei et al., 25 Nov 2025) "Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory"
• (Gumaan, 23 Mar 2025) "ExpertRAG: Efficient RAG with Mixture of Experts -- Optimizing Context Retrieval for Adaptive LLM Responses"