Continual Prompt Optimization (CPO)

• Continual Prompt Optimization (CPO) is a paradigm that dynamically refines LLM prompts in non-stationary settings using both negative and positive feedback.
• It employs iterated feedback aggregation methods to balance exploratory adjustments and maintain previously optimized prompt knowledge.
• CPO enables effective prompt migration across model upgrades, yielding significant accuracy improvements and API cost savings.

Continual Prompt Optimization (CPO) is a paradigm and set of methodologies for refining, adapting, and maintaining prompts for LLMs and foundation models in environments where tasks, data distributions, or underlying models evolve over time. CPO explicitly addresses the need to retain learned prompt knowledge, efficiently transfer or migrate prompts across model updates or task streams, and minimize computational and data costs, all while maintaining or enhancing task performance (Davari et al., 14 Jul 2025).

1. Formal Problem Statement and Motivation

CPO is formalized for scenarios where a black-box LLM or frozen pre-trained model $f$ processes user queries $u$ (or task inputs $x$) with a discrete prompt $x \in \{tokens\}^L$ of length $L$, returning a prediction $\hat{y} = f(x, u)$. The classical goal in prompt optimization is to identify $x^* = \arg\min_x L(x; f)$, with

$L(x;f) = \frac{1}{n} \sum_i \ell(f(x, u_i), y_i)$

where $(u_i, y_i)$ are labeled data and $\ell$ is a suitable loss function.

Continual Prompt Optimization generalizes this objective to non-stationary environments where the model itself, or the tasks/data, change over time: at time $u$0 with model $u$1, after an upgrade or domain shift, CPO seeks to find $u$2 minimizing $u$3, while leveraging the previously optimized prompt $u$4 (Davari et al., 14 Jul 2025). The motivating scenarios include API upgrades (e.g., GPT-3.5 $u$5 GPT-4o), adaptation to heterogeneous queries, or sequential arrival of new tasks/domains.

2. Core Update Mechanisms and Feedback Diversification

Update Strategy: CPO employs iterated, feedback-driven prompt modification using both negative and positive reinforcement textual gradients:

• $u$6: instructions abstracted from incorrect predictions (negative reinforcement).
• $u$7: instructions distilled from correct predictions (positive reinforcement).

A generic CPO update is

$u$8

with step size $u$9 and balancing weights $x$0 (Davari et al., 14 Jul 2025). In practice, both gradients are generated by prompting the LLM for feedback on the existing prompt's efficacy.

Feedback Diversification: To address the high variance and idiosyncrasy of LLM-generated feedback, CPO aggregates $x$1 feedback samples per update (for both $x$2 and $x$3), using robust operators such as token-wise median or attention-weighted sums. More concretely, each feedback ensemble $x$4 is processed by a median aggregator or a softmax attention-weighted sum parameterized by a learned query vector. This aggregation filters outliers and reduces variance, producing consistent and actionable update signals (Davari et al., 14 Jul 2025).

3. Prompt Migration and Algorithmic Implementation

CPO formalizes prompt migration—the efficient transfer and adaptation of expert prompts across model versions or providers—as a first-class objective. The standard migration algorithm proceeds as follows (Davari et al., 14 Jul 2025):

$x \in \{tokens\}^L$5 Key operational details:

• Positive reinforcement is introduced from a tuned iteration $x$5 onward (e.g., $x$6 for standard, $x$7 for migration).
• To contain API costs, $x$8 is often reduced (e.g., $x$9 for high-cost models).
• Prompt length and update granularity are selected to avoid saturation and over-generalization.

4. Empirical Results and Practical Impact

CPO has been empirically validated in both standard optimization and migration scenarios across a range of LLM tasks: causal judgment, geometric classification, biomedical sentence similarity, and natural language inference (Davari et al., 14 Jul 2025). Results on GPT-3.5-turbo and GPT-4o highlight:

Scenario	Accuracy Gain vs Baseline	API Calls Saved
APO (standard)	+4.9 to +21.5%	0.5%–3.3%
Prompt migration (GPT-3.5→4o)	+3.5 to +16.0%	4.2%–6.2%

Direct transfer of prompts without re-optimization yields only small gains or performance drop due to instruction loss, whereas CPO's re-optimization consistently outperforms classical methods both in accuracy and efficiency. Prompt adaptation converges in 8–15 iterations, though no formal convergence guarantees are provided.

5. Connections to Broader Continual Learning and Adaptive Prompting

CPO's principles are tightly connected to prompt-based continual learning in vision-language and NLP, where continual adaptation must balance plasticity (for new tasks or model versions) and stability (retaining previous knowledge).

Distinctive elements of CPO in the LLM and black-box context include:

• Direct aggregation of both corrective and preservative feedback, an extension over error-only correction in standard APO (Davari et al., 14 Jul 2025).
• Explicit prompt migration formulation for API/model evolution, a gap not addressed by prior classical prompt learning or fine-tuning methods.
• Robustness to feedback noise via ensemble aggregation, drawing parallels to attention-weighting and robust statistics.

While shared themes exist with other continual learning paradigms—such as prompt pools with dynamic retrieval (Wang et al., 2021), drift control through semantic unit attribution (Chen et al., 6 Jan 2026), and memory-based self-evolving strategies (Liang et al., 23 Mar 2026)—CPO's mathematical and empirical framework is specifically adapted for the high-variance, opaque-feedback environment of black-box LLM API application.

6. Recommended Practices, Limitations, and Future Directions

Essential best practices include:

• Tuning $x \in \{tokens\}^L$0 (feedback sample count) in $x \in \{tokens\}^L$1 to balance diversity and cost; excess $x \in \{tokens\}^L$2 can lead to over-generalization.
• Introducing positive reinforcement early in prompt migration tasks for faster stabilization.
• Careful grid search over $x \in \{tokens\}^L$3 (gradient weights), with $x \in \{tokens\}^L$4 as a robust initialization.
• Caching intermediate prompts/feedback to minimize redundant API calls.
• Early stopping on plateauing validation accuracy.

Limitations and proposed extensions:

• Current results are primarily for GPT-3.5/4 family; evaluation across more diverse LLMs is an open question.
• Fully automated hyperparameter tuning (possibly via Bayesian optimization) could enhance stability and generalization.
• Incorporation of more advanced aggregation strategies (voting, clustering) and adaptive or task-dependent reinforcement schedules.
• Potential for integration with memory-based and attribution-driven CPO mechanisms developed in adjacent work for enhanced interpretability and drift prevention (Chen et al., 6 Jan 2026, Liang et al., 23 Mar 2026).

CPO, by synthesizing both negative and positive explorative gradients and emphasizing noise-robust aggregation, establishes a rigorous and scalable foundation for prompt adaptation and migration in realistic, evolving LLM settings (Davari et al., 14 Jul 2025).