Retrieval-Augmented Prompt Optimization

• RAPO is a family of methods that refine prompt contexts using retrieved evidence instead of updating model parameters.
• It integrates retrieval with transformation techniques to synthesize optimized prompts that improve reasoning, accuracy, and domain alignment.
• Empirical evaluations show RAPO boosts performance by up to 33% in QA tasks and significantly enhances multimodal generation quality.

Searching arXiv for the cited RAPO-related papers to ground the article with fresh metadata and identifiers. Retrieval-Augmented Prompt Optimization (RAPO) denotes a family of methods in which retrieval is used to improve the prompt itself, or to optimize prompt-adjacent context before generation, rather than only to supply evidence for a downstream answer. Across the literature, the retrieved object varies: raw evidence that is transformed before question answering, reference prompts from annotated corpora, reusable reasoning assets derived from scored examples, semantically relevant modifiers from training-caption graphs, annotated examples for few-shot prompting, or historical reasoning trajectories used for self-supervised refinement (Rodrigues et al., 2024, Lee et al., 2 Sep 2025, Seo et al., 18 Oct 2025, Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025, Duc et al., 22 Dec 2025, Soni, 27 Mar 2026). The unifying premise is that prompt quality is retrieval-dependent and can be optimized without retraining the primary task model, although the acronym has also been used in adjacent but distinct senses, including prompt-engineered emulation of RAG inside long-context LLMs and Retrieval-Augmented Policy Optimization for Agentic RL (Park et al., 18 Feb 2025, Zhang et al., 3 Mar 2026).

1. Conceptual scope and major variants

The literature uses RAPO to describe several related prompt-side interventions. In one line of work, RAPO inserts a refinement stage between retrieval and answer generation: external contents are retrieved, transformed by a second LLM under an optimized instruction, and only then passed to the answering model (Rodrigues et al., 2024). In another line, prompt optimization is cast as retrieval over high-quality prompt exemplars, followed by contrastive reasoning or multi-agent analysis that produces an improved prompt for the same query (Lee et al., 2 Sep 2025, Seo et al., 18 Oct 2025). In text-to-video generation, RAPO retrieves modifiers from a relation graph built from training prompts, then refactors the resulting prompt to better match the training prompt distribution (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025). In reasoning-oriented LLMs, retrieval targets prior trajectories and reasoning traces, which are distilled into prompt guidance and iteratively revised with self-supervised signals (Soni, 27 Mar 2026).

These variants differ in what is retrieved and what exactly is optimized, but they share a prompt-centric view of adaptation. The goal is not parameter update of the target model, and not merely improved retrieval ranking, but a better prompt or prompt-conditioned context for the task at hand.

Work	Retrieved artifact	Prompt-side objective
(Rodrigues et al., 2024)	Retrieved contents	Refine retrieved evidence before QA
(Lee et al., 2 Sep 2025)	Top-$k$ reference prompts from HelpSteer2	Synthesize an optimized prompt through contrastive reasoning
(Seo et al., 18 Oct 2025)	Reasoning assets $(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$	Apply evidence-grounded edits
(Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025)	Relation-graph modifiers from training prompts	Align prompts with T2V training distributions
(Duc et al., 22 Dec 2025)	Annotated examples and prompt components	Iterative prompt search for logistics frame detection
(Soni, 27 Mar 2026)	Historical reasoning trajectories $T_x$	Self-supervised prompt refinement

2. Retrieval targets and architectural patterns

A first architectural pattern is retrieved-evidence refinement. "Meta-prompting Optimized Retrieval-augmented Generation" inserts an explicit transformation stage into RAG: $$q \rightarrow \text{retrieve } c \rightarrow \text{transform/refine } c \rightarrow \text{LLM generates answer from } (q, tc)$$. The optimization target is not the answer prompt alone, but the instruction that tells a transformation LLM how to clean, summarize, filter, or reorganize retrieved contents before final generation (Rodrigues et al., 2024).

A second pattern is retrieved exemplar reasoning. CRPO retrieves top-$k$ prompts from the HelpSteer2 training split with BM25 and then uses either tiered contrastive reasoning over high-, medium-, and low-quality prompts or multi-metric contrastive reasoning over the best prompt for each of helpfulness, correctness, coherence, complexity, and verbosity. MA-SAPO pushes this further by transforming scored prompt-response examples into reusable reasoning assets: explanation, diagnosis, and edit directives are generated offline, stored, retrieved at test time, and used by Analyzer and Refiner agents to produce evidence-grounded prompt edits (Lee et al., 2 Sep 2025, Seo et al., 18 Oct 2025).

A third pattern is retrieval for distribution alignment. In text-to-video generation, the user prompt is treated as under-specified and often misaligned with the captions seen during training. RAPO therefore retrieves scene-relevant modifiers from a relation graph built from Vimeo25M, merges them into the prompt, and refactors the result with a fine-tuned LLM; RAPO++ retains this stage and then adds test-time iterative scaling and LLM fine-tuning of the rewriter (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025).

A fourth pattern is retrieval as internalized prompt behavior rather than external indexing. The long-context method in "Emulating Retrieval Augmented Generation via Prompt Engineering for Enhanced Long Context Comprehension in LLMs" instructs the model to act as both retriever and reasoner in a single forward pass: relevant spans are tagged, localized summaries are produced, chain-of-thought synthesis is performed, and a final answer is generated. The paper explicitly frames this as a prompt-engineered, single-pass emulation of RAG, with prompt structure and prompt order functioning as the optimization locus (Park et al., 18 Feb 2025).

A fifth pattern is retrieval from reasoning memory. RASPRef retrieves prior reasoning episodes, assembles an initial prompt from a base prompt plus retrieved trajectories, samples multiple traces under the current prompt, scores those traces using self-consistency, verifier feedback, self-critique, and retrieval alignment, and iteratively edits the prompt itself (Soni, 27 Mar 2026).

3. Optimization mechanisms and formal formulations

The formal machinery used in RAPO varies by domain, but the papers share an explicit optimization perspective. In the QA-oriented RAPO of (Rodrigues et al., 2024), the search variable is the refinement instruction $I$ used by the transformation LLM. The optimization is written as

$I^* = \arg\max_{I \in \mathcal{I}} S(I),$

where the score is induced by downstream task performance after retrieved contents are transformed and then used by the generation model. The meta-prompt stores candidate refinement instructions and their scores, and iterative meta-prompting updates that history by replacing poor instructions with better-performing ones.

CRPO gives a more structured retrieval-and-reasoning formalism. Given a query $q$, the system retrieves

$$\mathcal{R}(q)=\{p_1,\dots,p_k\}, \quad p_i \in \text{HelpSteer2},$$

with BM25 and $k=10$ by default. In the tiered variant, prompt quality is aggregated by

$(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$0

where $(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$1, and the optimized prompt is generated as

$(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$2

In the multi-metric variant, the best prompt per dimension is selected and integrated: $(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$3 (Lee et al., 2 Sep 2025).

RASPRef makes prompt-level optimization explicit through a self-supervised quality objective. After retrieval of relevant trajectories $(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$4, the system builds $(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$5 and then iterates: $(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$6

$(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$7

Prompt quality is scored by

$(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$8

with the optimization target

$(\mathcal{C}_i,\mathcal{D}_i,\mathcal{E}_i)$9

This formulation shifts the target of optimization from output traces to the prompt that induces those traces (Soni, 27 Mar 2026).

In text-to-video RAPO, the formal core is an iterative merge of retrieved modifiers into the prompt: $T_x$0 followed by sentence refactoring with a fine-tuned LLM $T_x$1 and a discriminator that selects between the retrieval-refactored prompt $T_x$2 and a direct rewrite $T_x$3. RAPO++ preserves this structure as Stage 1 and then adds Sample-Specific Prompt Optimization and LLM fine-tuning as later stages (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025).

4. Domains of application and reported empirical behavior

In multi-hop question answering, the original RAPO paper evaluates on StrategyQA and reports that plain retrieval helps Llama-2-70b-chat from 16.53% for query-only prompting to 26.12% for query plus contents, while RAPO further raises performance to 34.69% with refined query plus contents. The paper characterizes this as an improvement of about 33% relative over plain RAG and reports an unpaired $T_x$4-test with two-tailed $T_x$5 across three seeds (Rodrigues et al., 2024).

In long-context reasoning, the single-pass RAG-emulation method is evaluated on BABILong tasks QA2, QA7, and QA10 at 16k, 32k, and 64k context lengths using gpt-4o-mini and llama-3.1-8b-instruct. The paper reports that the proposed method generally beats naive RAG on tasks requiring multiple dispersed facts, especially QA2 and QA7, while QA10 is more mixed and naive RAG can be competitive when a single local snippet is sufficient. The work also emphasizes that prompt order—standard order, question first, and relevant first—materially affects performance (Park et al., 18 Feb 2025).

On automatic prompt optimization benchmarks, CRPO uses HelpSteer2 both as retrieval corpus and evaluation benchmark. For GPT-4o, the normalized average score rises from 0.6003 for RAG to 0.6355 for CRPO-Tiered; for LLaMA 3-8B, it rises from 0.5137 to 0.5654. The paper identifies CRPO-Tiered as the best-performing method overall in both model settings and reports that $T_x$6 is the best tradeoff between diversity and inference cost (Lee et al., 2 Sep 2025).

MA-SAPO reports stronger gains on HelpSteer1 and HelpSteer2 by retrieving reasoning assets rather than raw prompts alone. On HelpSteer2 with GPT-4o, MA-SAPO: 0.6490 Avg, compared with MARS: 0.5791 and RAG: 0.6003; with LLaMA-3-8B, MA-SAPO: 0.6065 Avg, compared with MARS: 0.5482 and RAG: 0.5245. The paper also reports that MA-SAPO is substantially more efficient than heavy multi-agent baselines at test time because reasoning assets are built offline (Seo et al., 18 Oct 2025).

In logistics frame detection, retrieval-guided auto-prompting is evaluated on 1,500 Vietnamese logistics messages with 73 unique hierarchical frame labels and compares manual prompts, auto prompts, and Auto Prompt + RAG. The paper reports that optimized prompts improve accuracy by up to 15% over weaker prompting baselines. For GPT-4o, Auto + RAG 6-shot reaches 90% test / 92% real-world; for LLaMA 3.1 70B and Qwen 2.5 72B, the corresponding result is 87% / 87% (Duc et al., 22 Dec 2025).

In text-to-video generation, RAPO is evaluated on LaVie and Latte and improves VBench totals from 80.89% to 82.38% for LaVie and from 77.03% to 79.97% for Latte. The paper highlights especially large gains on multiple-object generation, including 37.71% → 64.86% for LaVie and 29.55% → 52.78% for Latte on the multiple-objects metric (Gao et al., 16 Apr 2025). RAPO++ extends this to five T2V models and five benchmarks, raising LaVie further to 82.65% and Latte to 80.75% on VBench, and reporting that the multiple-objects score for LaVie increases to 71.89% under RAPO++ (Gao et al., 23 Oct 2025).

For reasoning-oriented black-box LLMs, RASPRef reports a prototype validation on a 500-example GSM8K-style arithmetic subset, with retrieval-augmented prompting improving accuracy from 85.6% under static prompting to 95.0% with retrieval-augmented prompting. The paper presents this as initial evidence for the retrieval-guided prompt construction component rather than a complete end-to-end validation of the full iterative loop (Soni, 27 Mar 2026).

5. Evaluation criteria, strengths, and recurring limitations

The evaluation of RAPO methods is heterogeneous because the optimized object differs across tasks. QA-oriented work uses task accuracy and exact-match style scoring; CRPO and MA-SAPO rely on normalized multi-objective reward scores over helpfulness, correctness, coherence, complexity, and verbosity; logistics frame detection uses test accuracy, real-world accuracy, and human expert review; RASPRef aggregates self-consistency, verifier feedback, self-critique, and retrieval alignment; T2V methods report VBench, T2V-CompBench, EvalCrafter, VideoPhy, and PhyGenBench metrics, including temporal coherence, semantic alignment, spatial fidelity, and physical plausibility (Rodrigues et al., 2024, Lee et al., 2 Sep 2025, Seo et al., 18 Oct 2025, Duc et al., 22 Dec 2025, Soni, 27 Mar 2026, Gao et al., 23 Oct 2025).

Several strengths recur. First, these methods are typically compatible with black-box or API-accessed models. CRPO is explicitly motivated by the limited applicability of white-box prompt optimization, RASPRef is designed for frozen black-box systems, and logistics auto-prompting describes its loop as a gradient-free optimization procedure in prompt space (Lee et al., 2 Sep 2025, Soni, 27 Mar 2026, Duc et al., 22 Dec 2025). Second, retrieval often makes prompt optimization more interpretable. MA-SAPO turns evaluation scores into explanation, diagnosis, and edit directives; CRPO reasons over high- and low-quality prompts; the long-context RAG-emulation paper exposes intermediate <relevant_section> tags that function as explicit evidence localization (Seo et al., 18 Oct 2025, Lee et al., 2 Sep 2025, Park et al., 18 Feb 2025). Third, retrieval can improve prompt faithfulness to domain structure. In T2V, relation-graph retrieval aligns prompts with the training distribution; in logistics, retrieved examples reduce ambiguity in actor/reason/cause labeling (Gao et al., 16 Apr 2025, Duc et al., 22 Dec 2025).

The limitations are equally consistent. Retrieval quality is a central failure mode: noisy or weakly related examples can degrade the prompt, and several papers note that sparse BM25 may miss semantically similar cases (Seo et al., 18 Oct 2025, Soni, 27 Mar 2026). Sensitivity to prompt arrangement remains high: the long-context paper reports that ordering of instructions, context, and question significantly affects outcomes, and CRPO observes that too large a retrieval pool can dilute signal, with performance improving up to a moderate pool and then declining (Park et al., 18 Feb 2025, Lee et al., 2 Sep 2025). Domain dependence is also prominent: CRPO depends on HelpSteer2, logistics auto-prompting is evaluated only on logistics, and RASPRef is tested narrowly on GSM8K-style arithmetic (Lee et al., 2 Sep 2025, Duc et al., 22 Dec 2025, Soni, 27 Mar 2026). Cost and latency remain nontrivial in iterative settings, particularly when multiple generations, validations, or video evaluations are required (Duc et al., 22 Dec 2025, Gao et al., 23 Oct 2025).

6. Security implications, nomenclature, and open directions

Prompt optimization also creates an attack surface. DeRAG shows that a black-box, gradient-free adversary can optimize very short adversarial suffixes for RAG systems so as to re-rank a chosen wrong target passage into the retriever’s top results. The method uses Differential Evolution, a ranking-based hinge loss, and short suffixes often averaging 2–3 tokens, with the abstract describing adversarial suffixes of $T_x$7 tokens. On dense-retrieval BEIR subsets, the paper reports, for example, SciFact: 0.198 / 0.573 / 0.739 Success@1 / Success@10 / Success@20 with 2.34 tokens, and on sparse retrieval SciFact Top-1 / Top-10 / Top-20 success of 0.250 / 0.890 / 0.970 for DE_seq_stop. A BERT-based / RoBERTa-based detector performs poorly, with AUROC 0.2023 and AUPRC 0.4665 at a stringent target FPR of 0.5%, suggesting that prompt-side retrieval manipulation can be both effective and difficult to detect (Wang et al., 20 Jul 2025).

The acronym itself is ambiguous. "RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization" uses RAPO to denote an RL training framework rather than prompt optimization. That work decomposes training into Hybrid-policy Agentic Rollout and Retrieval-aware Policy Optimization and reports an +5.0% average gain across fourteen datasets with 1.2x faster training efficiency, but its target is policy exploration in Agentic RL, not prompt refinement (Zhang et al., 3 Mar 2026). Any encyclopedic treatment of RAPO therefore requires disambiguation between prompt-optimization usage and policy-optimization usage.

The forward-looking directions stated in the prompt-optimization literature are pragmatic rather than purely formal. CRPO suggests that better retrievers may help beyond BM25; MA-SAPO proposes adding a Feedback Agent, using hybrid dense-sparse retrieval, and extending to multi-turn prompts; the long-context RAG-emulation paper points to scalability beyond tens of thousands of tokens as uncertain; RAPO++ identifies numeracy as a persistent weakness and suggests counting verifiers or numeracy-sensitive assessment; DeRAG argues that future defenses should consider prompt precision, embedding regularization, and anomaly or detection mechanisms tailored to retrieval-side attacks (Lee et al., 2 Sep 2025, Seo et al., 18 Oct 2025, Park et al., 18 Feb 2025, Gao et al., 23 Oct 2025, Wang et al., 20 Jul 2025). Taken together, these works suggest that RAPO is best understood not as a single algorithm but as a retrieval-conditioned optimization layer for prompts, spanning evidence transformation, exemplar selection, reasoning-memory reuse, distribution alignment, and adversarial manipulation.