RAPO++: Cross-Stage Prompt Optimization

Updated 4 July 2026

RAPO++ is a cross-stage prompt optimization framework that unifies data-aligned refinement, iterative scaling, and LLM fine-tuning for T2V generation.
It addresses the prompt–distribution mismatch by aligning user prompts with training captions using retrieval, refactoring, and feedback-driven refinement.
Empirical results demonstrate improved semantic alignment, temporal consistency, and object interactions across multiple T2V models and benchmarks.

Searching arXiv for papers explicitly using the term “RAPO++” and its precursor RAPO in text-to-video generation. RAPO++ is a cross-stage prompt optimization framework for text-to-video (T2V) generation that unifies training-data--aligned refinement, test-time iterative scaling, and LLM fine-tuning to improve video generation without modifying the underlying generative backbone. In the formal usage established by "RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling" (Gao et al., 23 Oct 2025), the term refers specifically to a three-stage system built on top of the earlier RAPO framework from "The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generation" (Gao et al., 16 Apr 2025). The framework is motivated by the observation that user prompts are often short, unstructured, and misaligned with training data, whereas effective prompts for diffusion-based T2V models tend to be more aligned with the prompt distribution seen during training (Gao et al., 23 Oct 2025).

1. Terminological scope and relation to earlier RAPO variants

RAPO++ is a formal method name only in the context of T2V prompt optimization. This is an important distinction because multiple contemporaneous papers use the acronym RAPO for unrelated reinforcement-learning or policy-optimization methods, including Rank-Aware Policy Optimization, Relative-Absolute Policy Optimization, Reward-Aware Policy Optimization, Retrieval-Augmented Policy Optimization, Role-Aware Policy Optimization, Reflection-aware Adaptive Policy Optimization, Reflection-anchor Policy Optimization, Robust Adversarial Policy Optimization, and Regret-Aware Policy Optimization in other problem domains (Jiang et al., 8 Sep 2025, Liu et al., 26 Sep 2025, Deng et al., 4 Oct 2025, Zhang et al., 3 Mar 2026, Xu et al., 25 Jun 2026, ai et al., 5 Jan 2026, Gong et al., 10 May 2026, Kim et al., 13 Apr 2026, Hiremath, 8 Apr 2026). By contrast, the “++” designation belongs to the prompt-optimization line introduced for T2V generation (Gao et al., 23 Oct 2025).

Within that line, RAPO denotes Retrieval-Augmented Prompt Optimization, while RAPO++ extends it by adding Sample-Specific Prompt Optimization (SSPO) and LLM fine-tuning on optimized prompt pairs (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025). The relation is explicit in the paper’s staging: Stage 1 is the original RAPO-style training-data-aligned prompt optimization, and Stages 2 and 3 constitute the “++” extension (Gao et al., 23 Oct 2025).

A common misconception is to interpret RAPO++ as a modification of a reinforcement-learning objective. In the T2V literature represented here, this is incorrect. RAPO++ is a prompt optimization framework, not a PPO-like or GRPO-like optimizer (Gao et al., 23 Oct 2025).

2. Motivation: prompt–distribution mismatch in text-to-video generation

RAPO++ is built on the premise that diffusion-based T2V models are highly sensitive to prompt formulation. The papers argue that natural user prompts are often short, underspecified, and structurally unlike the captions used to train T2V systems, whereas training prompts are typically longer, more descriptive, and more regular in structure (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025). This mismatch can limit semantic alignment, compositional reasoning, temporal stability, and physical plausibility (Gao et al., 23 Oct 2025).

The problem is presented as more acute in T2V than in text-to-image because video generation depends not only on static object and scene fidelity but also on temporal consistency, motion realism, object persistence, and interaction structure (Gao et al., 23 Oct 2025). The earlier RAPO paper also stresses that simply making prompts longer is not enough: manually adding random descriptions or relying on unconstrained LLM rewriting can introduce inaccurate or ambiguous detail and can drift away from the training prompt distribution (Gao et al., 16 Apr 2025).

RAPO++ addresses this by combining three distinct forms of optimization. First, it aligns prompts with the training distribution through retrieval and refactoring. Second, it performs closed-loop refinement using generated-video feedback. Third, it distills these optimized prompt transformations into a fine-tuned rewriter LLM (Gao et al., 23 Oct 2025). This suggests a layered view of prompt optimization: corpus-level alignment, sample-level correction, and reusable learned rewriting behavior.

3. Stage 1: Retrieval-Augmented Prompt Optimization

Stage 1 of RAPO++ is the original RAPO module, which is designed to solve the prompt–distribution mismatch problem by retrieving semantically relevant modifiers from a relation graph built from T2V training captions and then refactoring the enriched prompt into a form closer to the training distribution (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025).

The Stage 1 pipeline contains three modules: Word Augmentation Module, Sentence Refactoring Module, and Prompt Selection Module (Gao et al., 23 Oct 2025). Its retrieval substrate is a relation graph $\mathcal{G}$ constructed from Vimeo25M, a dataset of 25 million text-video pairs, with about 2.1M valid sentences filtered for graph construction (Gao et al., 23 Oct 2025). Each scene is treated as a core node, and its associated subject, action, and atmosphere descriptions are connected as sub-nodes (Gao et al., 23 Oct 2025). Retrieval uses a sentence transformer pre-trained model, specifically all-MiniLM-L6-v2, with cosine similarity (Gao et al., 23 Oct 2025).

Given a user prompt $x_i$ , the system retrieves top- $k$ relevant scenes, gathers connected modifiers, and selects the top- $k$ relevant modifiers $\{p_n|_{n=0}^{k-1}\}$ (Gao et al., 23 Oct 2025). These modifiers are merged iteratively into the prompt using an LLM-based merge function: $x^{m+1}_{i} = f(x^{m}_{i},p^m_i),$ where $m=0,1,\ldots,k-1$ (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025). The iterative structure is intended to preserve the original prompt semantics while gradually introducing relevant detail.

The enriched prompt is then passed to a fine-tuned Sentence Refactoring Module using $L_r$ , trained on approximately 86k prompt pairs with LoRA on LLaMA 3.1, 8 epochs, batch size 32, LoRA rank 64, and single A100 (Gao et al., 23 Oct 2025). The target of refactoring is not arbitrary elaboration but alignment with the training prompt distribution in terms of format and length while maintaining “original subject descriptions, actions, and scene descriptions” and appending “additional straightforward actions” only if necessary (Gao et al., 23 Oct 2025).

The final component is a Prompt Selection Module that compares the refactored RAPO candidate $x_r$ against a naively rewritten LLM candidate $x_n$ . A discriminator $x_i$ 0, also fine-tuned with LoRA on LLaMA 3.1, 3 epochs, batch size 32, LoRA rank 64, and single A100, selects the better prompt (Gao et al., 23 Oct 2025). The discriminator is trained using triples $x_i$ 1, where labels are derived by generating videos with both prompts and evaluating them on task-appropriate metrics (Gao et al., 23 Oct 2025).

This first stage already embodies the central data-alignment hypothesis. The papers report that RAPO’s optimized prompt-length distribution is closest to the training set, whereas user prompts are too short and other methods often generate overly long prompts with excessive detail and complex vocabulary (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025).

4. Stage 2: Sample-Specific Prompt Optimization

The “++” extension begins with Sample-Specific Prompt Optimization (SSPO), which introduces test-time scaling via iterative prompt rewriting (Gao et al., 23 Oct 2025). Unlike Stage 1, which is generic and corpus-driven, SSPO is explicitly sample-specific: it uses the generated video for a particular prompt to drive prompt refinement for that same instance (Gao et al., 23 Oct 2025).

SSPO starts from the Stage 1 refined prompt $x_i$ 2, generates a video $x_i$ 3, and evaluates it using multiple feedback sources. The first is a semantic misalignment signal $x_i$ 4, where $x_i$ 5 is the original user prompt. In implementation, this uses LLaVA-OneVision (Gao et al., 23 Oct 2025). The second is a multi-verifier score: $x_i$ 6 where the verifiers assess dimensions such as spatial fidelity, temporal coherence, and semantic alignment (Gao et al., 23 Oct 2025). For physical-aware video generation, SSPO can also add an optical-flow branch $x_i$ 7 to assess flow consistency, object trajectories, motion dynamics, and physical plausibility (Gao et al., 23 Oct 2025).

These signals are stored in a feedback memory database: $x_i$ 8 and the next refined prompt $x_i$ 9 is generated using the original prompt, current refined prompt, historical feedback, and current scores (Gao et al., 23 Oct 2025). The rewriting instruction explicitly asks the LLM to minimize semantic misalignment and improve temporal and spatial coherence and perceptual fidelity (Gao et al., 23 Oct 2025).

Candidate prompts are compared using an Average Ranking mechanism: each candidate receives metric-wise ranks, and the prompt with the lowest average rank is selected as the best prompt $k$ 0 (Gao et al., 23 Oct 2025). The paper characterizes this as a way to avoid over-optimizing a single metric and instead maintain balanced improvements across semantic, spatial, temporal, and physical criteria.

SSPO is therefore a gradient-free, black-box optimization loop over prompts, not over model weights or latents (Gao et al., 23 Oct 2025). This distinguishes it from RLHF-style prompt optimization and from latent-space test-time search. It also introduces a clear compute trade-off: several SSPO iterations require additional T2V generations and video evaluations, pushing inference time to roughly $k$ 1 that of a single-pass baseline, with about $k$ 2 additional memory for LLaVA-OneVision (Gao et al., 23 Oct 2025).

5. Stage 3: Fine-tuning the rewriter LLM

Stage 3 closes the loop by using optimized prompt pairs from SSPO to fine-tune the rewriter LLM, thereby internalizing the expensive iterative optimization patterns discovered at test time (Gao et al., 23 Oct 2025). The training data consist of prompt pairs: $k$ 3 where $k$ 4 is the original user prompt and $k$ 5 is the best optimized prompt selected after iterative SSPO (Gao et al., 23 Oct 2025).

The paper states that Stage 2 rewriting uses Qwen2.5-7B-Instruct, while the fine-tuning stage performs LoRA fine-tuning using LLaMA 3.1 (Gao et al., 23 Oct 2025). The reported training setup includes batch size 32, LoRA rank 64, single A100 GPU, and within 5 hours per iteration (Gao et al., 23 Oct 2025). The paper also notes a prompt pool of about 12k prompts generated by GPT-4 for analysis (Gao et al., 23 Oct 2025).

Conceptually, Stage 3 serves two functions. First, it improves the quality of the initial prompt rewriting before test-time optimization. Second, it reduces reliance on many SSPO rounds by distilling task-specific optimization behavior into the rewriter model itself (Gao et al., 23 Oct 2025). This suggests that RAPO++ is not merely a test-time search procedure; it is also a self-improving prompt-rewriting system.

6. Empirical performance and ablations

RAPO++ is evaluated on five T2V models—LaVie, Latte, HunyuanVideo, CogVideoX, and Wan2.1—and five benchmarks: VBench, T2V-CompBench, EvalCrafter, VideoPhy, and PhyGenBench (Gao et al., 23 Oct 2025).

On VBench, for LaVie, the baseline score is 80.89%, RAPO reaches 82.38%, and RAPO++ reaches 82.65% (Gao et al., 23 Oct 2025). For Latte, the corresponding scores are 77.03%, 79.97%, and 80.75% (Gao et al., 23 Oct 2025). The strongest submetric gains occur in Multiple Objects, Spatial Relationship, Object Class, and Human Action. For LaVie, Multiple Objects improves from 37.71% to 64.86% with RAPO and 71.89% with RAPO++; Spatial Relationship improves from 37.27% to 59.15% and 64.76% (Gao et al., 23 Oct 2025). Similar patterns hold for Latte (Gao et al., 23 Oct 2025). This suggests that the framework is especially effective for compositional and interaction-heavy prompts.

On EvalCrafter, for LaVie, Text-Video Alignment improves from 69.60 to 74.38 to 75.62, while Temporal Consistency improves from 60.87 to 61.29 to 66.80 (Gao et al., 23 Oct 2025). For Latte, Text-Video Alignment improves from 55.49 to 60.86 to 61.92 (Gao et al., 23 Oct 2025). These results indicate that the “++” stages contribute not only to static prompt compliance but also to video-level temporal quality.

On T2V-CompBench, RAPO already yields large gains, and RAPO++ usually improves further. For LaVie, Consistent Attribute Binding goes from 0.620 to 0.692 to 0.742, and Object Interactions from 0.760 to 0.839 to 0.849 (Gao et al., 23 Oct 2025). For Latte, Object Interactions goes from 0.792 to 0.847 to 0.856 (Gao et al., 23 Oct 2025). One nuance is that gains are not uniform on every submetric: on LaVie, Action Binding is 0.635 for RAPO and 0.632 for RAPO++ (Gao et al., 23 Oct 2025). This indicates that the cross-stage refinement improves overall quality but does not dominate on every individual metric.

In the physics-aware setting, across four SSPO rounds on PhyGenBench, quality increases steadily. For HunyuanVideo, PC improves from 0.38 to 0.57 and SA from 0.24 to 0.42; for CogVideoX-5B, PC improves from 0.34 to 0.53 (Gao et al., 23 Oct 2025). On VideoPhy, improvements are also reported across solid-solid, solid-fluid, and fluid-fluid conditions (Gao et al., 23 Oct 2025). These results support the claim that optical-flow-informed SSPO can refine prompts for physical plausibility.

The paper also reports compute-normalized gains for SSPO. Across LaVie and Latte, the additional inference cost of roughly $k$ 6 yields average gains of $k$ 7 on VBench and $k$ 8 on T2V-CompBench (Gao et al., 23 Oct 2025).

Ablations clarify component roles. For Stage 1, VBench total score on LaVie rises from 80.37% with word augmentation only and 79.75% with sentence refactoring only to 82.38% for full RAPO (Gao et al., 23 Oct 2025). For Stage 2 and Stage 3 on T2V-CompBench with LaVie, the progression is:

Variant	Consistent Attribute Binding	Dynamic Attribute Binding	Action Binding	Object Interactions
w/o fine-tuning $k$ 9, w/o SSPO	0.620	0.232	0.483	0.760
w/o fine-tuning $k$ 0, w/ SSPO	0.629	0.236	0.542	0.778
w/ fine-tuning $k$ 1, w/o SSPO	0.659	0.253	0.552	0.835
w/ fine-tuning $k$ 2, w/ SSPO	0.742	0.294	0.632	0.849

These results show that SSPO alone helps, fine-tuning alone helps more, and combining both is best (Gao et al., 23 Oct 2025).

7. Interpretation, model-agnosticism, and limitations

RAPO++ is repeatedly described as model-agnostic or “model-unaware,” meaning it does not modify T2V model weights or rely on architectural internals and can be used with both unet-based and DiT-based systems (Gao et al., 23 Oct 2025). The experiments across LaVie, Latte, HunyuanVideo, CogVideoX, and Wan2.1 support this portability claim (Gao et al., 23 Oct 2025). At the same time, the framework’s underlying rationale is explicitly training-data alignment, so its effectiveness depends on how well the prompt-optimization machinery matches the training prompt distribution relevant to the target generator (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025). This suggests that “model-agnostic” should be understood as interface-level portability, not as invariance to prompt-distribution differences.

The main explicit limitation discussed is numeracy. The paper presents failure cases such as “Five colorful parrots perch on a tree branch” and “Three majestic giraffes graze on the leaves of tall trees in the African savannah...” where the generated videos do not preserve the correct entity counts (Gao et al., 23 Oct 2025). The authors attribute this to T2V models’ tendency to blur numerical information into broader semantics and to SSPO’s lack of fine-grained count-aware feedback (Gao et al., 23 Oct 2025).

Other limitations are implied by the framework itself. RAPO++ depends on retrieval quality from the relation graph, LLM rewriting quality, and verifier quality in SSPO (Gao et al., 23 Oct 2025). Its gains are not perfectly uniform across all metrics, and the iterative stages incur substantial inference-time cost (Gao et al., 23 Oct 2025). The paper does not present a full failure taxonomy, but these trade-offs are visible in the ablations and compute analysis.

A second potential misunderstanding is to view RAPO++ as generic prompt beautification. The papers argue against this interpretation. The core mechanism is selective, semantically relevant, training-aligned enrichment, followed by feedback-driven correction, not indiscriminate elaboration (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025). This suggests that the method’s main contribution lies in controlling prompt distribution and prompt semantics jointly.

8. Significance within T2V prompt optimization

RAPO++ occupies a specific methodological position in T2V generation. Compared with standard prompt engineering, it replaces manual heuristics with a structured pipeline grounded in training captions, iterative feedback, and learned prompt rewriting (Gao et al., 23 Oct 2025). Compared with generic retrieval-augmented generation, its retrieval operates over scene–modifier relations derived from T2V training captions rather than over external knowledge documents (Gao et al., 16 Apr 2025, Gao et al., 23 Oct 2025). Compared with RLHF-style prompt optimization, it remains black-box with respect to the generator and optimizes prompts rather than model weights or latent trajectories (Gao et al., 23 Oct 2025).

Its most characteristic contribution is the unification of three temporal scales of prompt optimization: pre-inference corpus alignment, inference-time sample-specific correction, and cross-run distillation of optimized prompt behavior (Gao et al., 23 Oct 2025). This makes RAPO++ best understood not as a single rewriting model, but as a prompt-control layer around T2V systems.

In the formal literature represented here, RAPO++ therefore denotes a specific T2V framework: Stage 1 RAPO for retrieval-augmented prompt alignment, Stage 2 SSPO for iterative verifier-guided test-time refinement, and Stage 3 LLM fine-tuning on optimized prompt pairs (Gao et al., 23 Oct 2025). Its empirical profile is strongest on semantic alignment, multiple-object composition, spatial relations, object interactions, and temporal consistency, while its main practical trade-off is increased inference cost and its main open challenge is numeracy (Gao et al., 23 Oct 2025).